Using the Amazon Simple Storage Service (S3), list the keys of all objects contained in a folder or a bucket.
The performance of this transformer is dependent on the amount of memory allocated to the Java Virtual Machine (JVM). The following environment variables allow you to specify memory available to Java Plugins:
- FME_JVM_MIN_HEAP_SIZE: Initial heap size for initializing the JVM. Default value is 1024K.
- FME_JVM_MAX_HEAP_SIZE: Maximum heap size for initializing JVM. Default value is 16384K.
If unset, the JVM applies the default values. If set, the values must be multiples of 1024K (for example, 4M and 64M, or 4096K and 32768K).
Note: To pass additional parameters used by FME to the Java Virtual Machine, use the JAVA_TOOL_OPTIONS environment variable.
This transformer accepts any feature.
All object keys for a particular bucket, which can optionally be confined to a specific folder within that bucket, will be output through this port.
Input features are output through this port with an additional attribute (specified by Key Count Attribute) that contains the number of listed objects.
Features that generate an error are output through this port. The error messages are stored in an additional attribute specified by Error Attribute.
The name of the Amazon S3 bucket.
The AWS Region in which the specified Bucket resides. If the default value, US East, N. Virginia (us-east-1), is specified, and the specified Bucket does not reside there, the operation will still succeed. However, to optimize latency, it is best practice to specify the correct region.
Specify a web connection to Amazon S3. Web connections can be reused in multiple workspaces, and connection parameters are hidden in the workspace. For more information, see Using Web Connections. Alternatively, check Embed Access Key (below) and specify the connection parameters manually.
When checked, you must specify the connection parameters to Amazon S3 manually. The connection parameters are visible in the workspace. To connect, specify:
An access key associated with a user with permission to upload data to the specified bucket. If a pairing of an access key and a secret access key is not given, credentials will be searched in various locations listed here.
- In a file titled AwsCredentials.properties located in the Java classpath
- For the properties aws.accessKeyId and aws.secretKey in the Java system properties
- In the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_KEY
If credentials are still not found, the client will act as if it is in anonymous mode, where requests aren’t signed. This is useful if accessing a publicly accessible object or bucket.
A secret key paired with the access key provided. See the Access Key ID parameter for credential searching.
Whether to list keys of objects contained in the bucket or a specific folder within that bucket.
Limit the keys returned to those of objects contained within this folder. This can also be the path to a subfolder, where folder names are delimited by a ‘/’.
Whether or not the keys of objects contained within subfolders should be listed as well.
Specify the attribute that will store the output object key.
This attribute will be present on features passing through the Output port.
Specify the output attribute that will store the number of objects listed.
This attribute will be present on features passing through the Initiator port.
Editing Transformer Parameters
Using a set of menu options, transformer parameters can be assigned by referencing other elements in the workspace. More advanced functions, such as an advanced editor and an arithmetic editor, are also available in some transformers. To access a menu of these options, click beside the applicable parameter. For more information, see Transformer Parameter Menu Options.
FME Licensing Level
FME Professional edition and above
Search FME Knowledge Center
Search for samples and information about this transformer on the FME Knowledge Center.