Databricks Writer (FME Form Package)

The Databricks Writer writes non-spatial data to Databricks Delta Lake tables.

The Databricks Reader is available in the FME 2023.0 release.

Databricks Product and System Requirements

Package Format	FME Platform			Operating System
Reader/Writer	FME Form	FME Flow	FME Flow Hosted	Windows 64-bit	Linux	Mac
Reader	See Databricks Reader
Writer	Yes	Yes	Yes	Yes	Yes	Yes

Integrating this Format with FME Form

This is a packaged format requiring a version of FME 2022.2 or greater (b22201).

Download from FME Hub

https://hub.safe.com/publishers/safe/packages/databricks

What is FME Hub?

Dependencies

Python 3.6+
Databricks workspace with Unity Catalog enabled

Writer Overview

The Databricks writer can be used to create or modify the contents of Databricks Delta tables.

The Databricks workspace clusters used with this writer should already be configured with direct access to a cloud storage location (an S3 bucket or an Azure Data Lake container). FME will upload files to this staging location before importing them into the specified Delta table.

Only Databricks workspaces which have Unity Catalog enabled are officially supported by this writer.