Databricks Reader

FME can read attribute data held in Databricks database tables.

FME provides read access via Databricks’s JDBC driver, which is bundled with FME.

Databricks Product and System Requirements

Format	FME Platform			Operating System
Reader/Writer	FME Form	FME Flow	FME Flow Hosted	Windows 64-bit	Linux	Mac
Reader	Yes	Yes	Yes	Yes	Yes	Yes
Writer	See Databricks Writer (FME Form Package)

Getting Started with Databricks

This tutorial series details how to use the Databricks reader and writer, including creating database connections and example scenarios.

See the FME Knowledge Base article Getting Started with Databricks.

Usage Notes

The performance of this format is dependent on the amount of memory allocated to the Java Virtual Machine (JVM). The following environment variables allow you to specify memory available to Java Plugins:

FME_JVM_MIN_HEAP_SIZE

Initial heap size for initializing the JVM. If unset, the default value is 1024K.

This variable must be set in multiples of 1024 and greater than 1 MB. To indicate kilobytes, megabytes, or gigabytes, append k or K, m or M, or g or G, respectively. For example, any of these values is acceptable:

6291456

6144k

FME_JVM_MAX_HEAP_SIZE

Maximum heap size for initializing the JVM. If unset, the default value is 16384K.

This variable must be set in multiples of 1024 and greater than 2 MB. To indicate kilobytes, megabytes, or gigabytes, append k or K, m or M, or g or G, respectively. For example, any of these values is acceptable:

83886080

81920k

80m

Tip To pass additional parameters used by FME to the Java Virtual Machine, use the JAVA_TOOL_OPTIONS environment variable.

Memory Usage (applicable to writers only): Changing the value in the writer parameter Features Per Transaction can control the number of features that FME places in each transaction before a transaction is committed to the database.