Amazon DynamoDB Reader/Writer

FME can read and write DynamoDB attribute data.

DynamoDB is a managed NoSQL database service offered by Amazon Web Services. The DynamoDB reader/writer communicates using HTTP requests managed by the Amazon Web Services Java SDK.

Note  These help topics assume familiarity with DynamoDB, the attribute types supported, and its indexing mechanisms.

For more information, please visit the Amazon DynamoDB home page.

Usage Notes

The performance of this format is dependent on the amount of memory allocated to the Java Virtual Machine (JVM). The following environment variables allow you to specify memory available to Java Plugins:

FME_JVM_MIN_HEAP_SIZE

Initial heap size for initializing the JVM. If unset, the default value is 1024K.

This variable must be set in multiples of 1024 and greater than 1 MB. To indicate kilobytes, megabytes, or gigabytes, append k or K, m or M, or g or G, respectively. For example, any of these values is acceptable:

6291456

6144k

6m

FME_JVM_MAX_HEAP_SIZE

Maximum heap size for initializing the JVM. If unset, the default value is 16384K.

This variable must be set in multiples of 1024 and greater than 2 MB. To indicate kilobytes, megabytes, or gigabytes, append k or K, m or M, or g or G, respectively. For example, any of these values is acceptable:

83886080

81920k

80m

Tip  To pass additional parameters used by FME to the Java Virtual Machine, use the JAVA_TOOL_OPTIONS environment variable.
Memory Usage (applicable to writers only): Changing the value in the writer parameter Features Per Transaction can control the number of features that FME places in each transaction before a transaction is committed to the database.

Reader Overview

FME considers a DynamoDB dataset to be a database containing a collection of tables. The tables must be defined in FME Workbench before they can be read.

When reading from the DynamoDB database, each table is considered a feature type in FME and each row of a table one feature in FME.

Open the FME Workbench DynamoDB Reader Parameters to view the table list options.

Writer Overview

The DynamoDB writer module stores attributes into a DynamoDB database. The DynamoDB writer provides the following capabilities:

  • Index Creation: The DynamoDB writer can set up and populate indexes as part of the loading process. When creating a table, a partition key index must be specified. One additional column can be indexed using DynamoDB’s sort key index. The DynamoDB writer also supports the creation of a single global or local secondary index.
  • Batch Writes: The DynamoDB writer writes in batches of up to 25 items in order to ensure a faster data loading process. The maximum number of items processed in a single batch can be changed using the Batch Size writer parameter.