FME Form: 2024.2
Reading from a URL
Another variation on a dataset location is the ability to read data from a URL.
Defining a URL dataset to be read is as simple as entering the URL into the Dataset field within any of FME’s dataset selection tools. After reading, the data will be downloaded and used just as if it were a normal dataset.
URL datasets can be read using either HTTP or FTP.
To add a reader using a URL instead of a file location:
- Open the Add Reader dialog.
- Select the Format.
- Type or paste the URL of a dataset into the Dataset field
- Click OK to add the reader to the workspace.
Reading from Archive Files
Supported archiving files: 7z, 7zip, rar (except v5), rvz, tar, tar.bz2, tar.gz, tgz, zip
Supported compression-only files: bz2, gz
- To read all files of a specified extension at the top level of an archive file, use [*.<extension>]. For example:
- To read all files of a specified extension at the top level and in folders beneath the top level, use [**\*.<extension>]. For example:
- To read all files of a specified extension that are nested two folders deep into an archive file:
- To read all files of a specified extension under a relative path, use [<relative\path>\*.<extension>]. For example:
- If the file extensions inside an archive are different (for example, png and PNG), use *.* to open all files in the specified path. For example:
http://<hostname>/<filename>.zip[*.shp]
http://<hostname>/<filename>.zip[*.shp][**\*.shp]
http://<hostname>/<filename>.zip[*/*/*.shp]
http://<hostname>/<filename>.zip[*.shp][canada\lakes\*.shp]
http://<hostname>/<filename>.zip[*.*]
Reading from Cloud Storage URLs
FME can also read data from an archive file stored in Google Docs (or from other cloud storage facilities). For example, to read from a Google Doc:
- Share the file publicly and copy its ID number.
- Open the Add Reader dialog.
- In the Dataset Parameter, use a URL of the form:
https://docs.google.com/uc?export=download&id=XXXX
where XXXX is the ID of the Google document