S3Connector

Accesses the Amazon Simple Storage Service (S3) file storage service to upload, download, or delete files and folders or list file/folder information from an S3 account.

Jump to Configuration

Typical Uses

  • Manage datasets on S3 by uploading, downloading, and deleting files and folders
  • Transfer a file's contents (such as XML or raster) into or out of an attribute in FME
  • Read downloaded S3 data using the FeatureReader, or upload data written by the FeatureWriter to S3
  • Retrieve file and folder names, paths, links and other information from S3 to use elsewhere in a workspace.

How does it work?

The S3Connector uses your Amazon S3 account credentials (either via a previously defined FME web connection, or by setting up a new FME web connection right from the transformer) to access the file storage service.

Depending on your choice of actions, it will upload or download files, folders, and attributes; list information from the service; or delete items from the service. On uploads, link attributes can be added to the output features. On List actions, file/folder information are added as attributes.

Usage Notes

  • This transformer cannot be used to directly move or copy files between different S3 locations. However, multiple S3Connectors can be used to move or copy files with an intermediate download.
  • The FeatureReader can access S3 directly, without using the S3Connector. In this case, a local copy of the dataset will not be created.

Configuration

Input Ports

Output Ports

Parameters

The remaining parameters available depend on the value of the Request > Action parameter. Parameters for each Action are detailed below.

Editing Transformer Parameters

Using a set of menu options, transformer parameters can be assigned by referencing other elements in the workspace. More advanced functions, such as an advanced editor and an arithmetic editor, are also available in some transformers. To access a menu of these options, click beside the applicable parameter. For more information, see Transformer Parameter Menu Options.

Defining Values

There are several ways to define a value for use in a Transformer. The simplest is to simply type in a value or string, which can include functions of various types such as attribute references, math and string functions, and workspace parameters. There are a number of tools and shortcuts that can assist in constructing values, generally available from the drop-down context menu adjacent to the value field.

Dialog Options - Tables

Transformers with table-style parameters have additional tools for populating and manipulating values.

Reference

Processing Behavior

Feature-Based

Feature Holding

No

Dependencies S3 account
Aliases S3Deleter, S3Downloader, S3ObjectLister, S3Uploader
History Released FME 2019.0

FME Community

The FME Community is the place for demos, how-tos, articles, FAQs, and more. Get answers to your questions, learn from other users, and suggest, vote, and comment on new features.

Search for all results about the S3Connector on the FME Community.

 

Examples may contain information licensed under the Open Government Licence – Vancouver and/or the Open Government Licence – Canada.