Using the Amazon Simple Storage Service (S3), downloads data from an object in an S3 bucket.
Usage Notes
The performance of this transformer is dependent on the amount of memory allocated to the Java Virtual Machine (JVM). The following environment variables allow you to specify memory available to Java Plugins:
- FME_JVM_MIN_HEAP_SIZE: Initial heap size for initializing the JVM. Default value is 1024K.
- FME_JVM_MAX_HEAP_SIZE: Maximum heap size for initializing JVM. Default value is 16384K.
If unset, the JVM applies the default values. If set, the values must be multiples of 1024K (for example, 4M and 64M, or 4096K and 32768K).
Note: To pass additional parameters used by FME to the Java Virtual Machine, use the JAVA_TOOL_OPTIONS environment variable.
Input Ports
Input
This transformer accepts any feature.
Output Ports
Output
Features which have successfully downloaded their contents from S3 are output through this port.
<Rejected>
Features which have not successfully downloaded their contents from S3 are output through this port. Additionally, a message stored in an attribute specified in Error Attribute, will contain details about the failure.
Parameters
S3 Bucket
Bucket
The name of the Amazon S3 bucket.
Region
The AWS Region in which the specified Bucket resides. If the default value, US East, N. Virginia (us-east-1), is specified, and the specified Bucket does not reside there, the operation will still succeed. However, to optimize latency, it is best practice to specify the correct region.
Access Key
Specify a web connection to Amazon S3. Web connections can be reused in multiple workspaces, and connection parameters are hidden in the workspace. For more information, see Using Web Connections. Alternatively, check Embed Access Key (below) and specify the connection parameters manually.
Embed Access Key
When checked, you must specify the connection parameters to Amazon S3 manually. The connection parameters are visible in the workspace. To connect, specify:
Access Key ID
An access key associated with a user with permission to upload data to the specified bucket. If a pairing of an access key and a secret access key is not given, credentials will be searched in various locations listed here.
- In a file titled AwsCredentials.properties located in the Java classpath
- For the properties aws.accessKeyId and aws.secretKey in the Java system properties
- In the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_KEY
If credentials are still not found, the client will act as if it is in anonymous mode, where requests aren’t signed. This is useful if accessing a publicly accessible object or bucket.
Secret Access Key
A secret key paired with the access key provided. See the Access Key ID parameter for credential searching.
Download Options
Object Key
The name of the object to download. If the object is stored at the root of the bucket, specify only the file name (for example, myFile.txt). If it is stored inside a folder in the bucket, specify the path to the file (for example: FolderName/myFile.txt).
S3 Acceleration
If Yes, allows Amazon S3 Transfer Acceleration on the bucket, if enabled. To enable acceleration, see http://docs.aws.amazon.com/AmazonS3/latest/UG/enable-bucket-transfer-acceleration.html.
When allowing S3 Acceleration, keep in mind the following:
- The specified Bucket name must be DNS-compliant.
- Your Amazon account is charged for performing accelerated requests.
Output Options
Data Target
The target of the object data to be downloaded.
- Attribute: The downloaded object will be stored in the specified attribute. See Target Attribute.
- File: The downloaded object will be stored in the specified file. See Target File Path.
When working with large objects, File is the better choice as the data will be streamed directly to disk rather than stored entirely in memory on a feature.
Target File Path
Specify the path and name of the file that will store the downloaded object. The S3Downloader creates any folders in the specified path that do not currently exist.
Handle Existing File
Instructs the S3Downloader if Data Target is File, and a file already exists at the specified Target File Path:
- Reject: The download is rejected.
- Overwrite: The existing file is overwritten.
Target Attribute
Specify the attribute that will store the downloaded object.
Response Body Encoding
If Data Target is Attribute, you can specify a character set encoding to ensure the output is in a human-readable form.
Editing Transformer Parameters
Using a set of menu options, transformer parameters can be assigned by referencing other elements in the workspace. More advanced functions, such as an advanced editor and an arithmetic editor, are also available in some transformers. To access a menu of these options, click 
beside the applicable parameter. For more information, see Transformer Parameter Menu Options.
Transformer Categories
FME Licensing Level
FME Professional edition and above
Search FME Knowledge Center
Search for samples and information about this transformer on the FME Knowledge Center.