S3Downloader

This transformer accepts any feature.

Features which have successfully downloaded their contents from S3 are output through this port.

Features which have not successfully downloaded their contents from S3 are output through this port. Additionally, a message stored in an attribute specified in Error Attribute, will contain details about the failure.

The name of the Amazon S3 bucket.

The AWS Region in which the specified Bucket resides. If the default value, US East, N. Virginia (us-east-1), is specified, and the specified Bucket does not reside there, the operation will still succeed. However, to optimize latency, it is best practice to specify the correct region.

Specify a web connection to Amazon S3. Web connections can be reused in multiple workspaces, and connection parameters are hidden in the workspace. For more information, see Using Web Connections. Alternatively, check Embed Access Key (below) and specify the connection parameters manually.

When checked, you must specify the connection parameters to Amazon S3 manually. The connection parameters are visible in the workspace. To connect, specify:

Access Key ID

An access key associated with a user with permission to upload data to the specified bucket. If a pairing of an access key and a secret access key is not given, credentials will be searched in various locations listed here.

1. In a file titled AwsCredentials.properties located in the Java classpath
2. For the properties aws.accessKeyId and aws.secretKey in the Java system properties
3. In the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_KEY

If credentials are still not found, the client will act as if it is in anonymous mode, where requests aren’t signed. This is useful if accessing a publicly accessible object or bucket.

Secret Access Key

A secret key paired with the access key provided. See the Access Key ID parameter for credential searching.

The name of the object to download. If the object is stored at the root of the bucket, specify only the file name (for example, myFile.txt). If it is stored inside a folder in the bucket, specify the path to the file (for example: FolderName/myFile.txt).

If Yes, allows Amazon S3 Transfer Acceleration on the bucket, if enabled. To enable acceleration, see http://docs.aws.amazon.com/AmazonS3/latest/UG/enable-bucket-transfer-acceleration.html.

When allowing S3 Acceleration, keep in mind the following:

• The specified Bucket name must be DNS-compliant.
• Your Amazon account is charged for performing accelerated requests.

The target of the object data to be downloaded.

• Attribute: The downloaded object will be stored in the specified attribute. See Target Attribute.
• File: The downloaded object will be stored in the specified file. See Target File Path.

When working with large objects, File is the better choice as the data will be streamed directly to disk rather than stored entirely in memory on a feature.

Specify the path and name of the file that will store the downloaded object. The S3Downloader creates any folders in the specified path that do not currently exist.

Instructs the S3Downloader if Data Target is File, and a file already exists at the specified Target File Path:

• Reject: The download is rejected.
• Overwrite: The existing file is overwritten.

Specify the attribute that will store the downloaded object.