• All Files

Amazon Athena Reader

Licensing options for this format begin with FME Professional Edition.

The Amazon Athena reader module provides FME with access to attribute data held in Amazon Athena tables. FME provides read access via Amazon's Athena JDBC driver.

Overview

Amazon's Athena JDBC driver is not bundled with FME, but can be downloaded from http://docs.aws.amazon.com/athena/latest/ug/connect-with-jdbc.html and manually installed.

Installing the JDBC Driver

To view instructions for installing the JDBC driver, please see Getting Started with JDBC.

AWS fees are incurred for Amazon Athena usage in the following ways:

  1. For the amount of data scanned (all data is scanned for every query, unless the underlying data supports columnar storage).
  2. The underlying data is stored on S3.
  3. The result set of each query is stored on S3 (and will not be cleaned up by FME).

The Amazon Athena reader will not typically be an efficient way to retrieve entire datasets from S3; it is intended for retrieving subsets of datasets.

Usage Notes

The performance of this format is dependent on the amount of memory allocated to the Java Virtual Machine (JVM). The following environment variables allow you to specify memory available to Java Plugins:

  • FME_JVM_MIN_HEAP_SIZE: Initial heap size for initializing the JVM. If unset, the default value is 1024K.

  • FME32_JVM_MIN_HEAP_SIZE: Same as FME_JVM_MIN_HEAP_SIZE, but applies to 32-bit FME Desktop on Windows, and if set, takes precedence over FME_JVM_MIN_HEAP_SIZE.

These variables must be set in multiples of 1024 and greater than 1 MB. To indicate kilobytes, megabytes, or gigabytes, append k or K, m or M, or g or G respectively. For example, any of these values is acceptable:

6291456

6144k

6m

  • FME_JVM_MAX_HEAP_SIZE: Maximum heap size for initializing the JVM. If unset, the default value is 16384K.

  • FME32_JVM_MAX_HEAP_SIZE: Same as FME_JVM_MAX_HEAP_SIZE, but applies to 32-bit FME Desktop on Windows, and if set, takes precedence over FME_JVM_MAX_HEAP_SIZE.

These variables must be set in multiples of 1024 and greater than 2 MB. To indicate kilobytes, megabytes, or gigabytes, append k or K, m or M, or g or G respectively. For example, any of these values is acceptable:

83886080

81920k

80m

Note: To pass additional parameters used by FME to the Java Virtual Machine, use the JAVA_TOOL_OPTIONS environment variable.