Databricks Reader

FME can read attribute data held in Databricks database tables.

FME provides read access via Databricks’s JDBC driver, which is bundled with FME.

Databricks Product and System Requirements

Format

FME Platform

Operating System

Reader/Writer

FME Form

FME Flow

FME Flow Hosted

Windows 64-bit

Linux

Mac

Reader

Yes

Yes

Yes

Yes

Yes

Yes

Writer

See Databricks Writer (FME Form Package)

Getting Started with Databricks

This tutorial series details how to use the Databricks reader and writer, including creating database connections and example scenarios.

See the FME Community article Getting Started with Databricks.

Usage Notes

The performance of this format is dependent on the amount of memory allocated to the Java Virtual Machine (JVM). The following environment variables allow you to specify memory available to Java Plugins:

FME_JVM_MIN_HEAP_SIZE

Initial heap size for initializing the JVM. If unset, the default value is 1024K.

This variable must be set in multiples of 1024 and greater than 1 MB. To indicate kilobytes, megabytes, or gigabytes, append k or K, m or M, or g or G, respectively. For example, any of these values is acceptable:

6291456

6144k

6m

FME_JVM_MAX_HEAP_SIZE

Maximum heap size for initializing the JVM. If unset, the default value is 16384K.

This variable must be set in multiples of 1024 and greater than 2 MB. To indicate kilobytes, megabytes, or gigabytes, append k or K, m or M, or g or G, respectively. For example, any of these values is acceptable:

83886080

81920k

80m

Tip  To pass additional parameters used by FME to the Java Virtual Machine, use the JAVA_TOOL_OPTIONS environment variable.
Memory Usage (applicable to writers only): Changing the value in the writer parameter Features Per Transaction can control the number of features that FME places in each transaction before a transaction is committed to the database.