Pinecone Reader (FME Form Package)

Pinecone is a managed vector database for storing and searching vector embeddings.

The Pinecone reader reads records from Pinecone indexes. It can extract records, or run a similarity search that returns the records closest to a query vector or record.

Integrating this Format with FME Form

Download from FME Hub

https://hub.safe.com/publishers/safe/packages/pinecone

What is FME Hub?

FME Build

This is a packaged format requiring a version of FME 2026.2 or greater (build 26304).

Dependencies

Python 3.10

Product and System Requirements

Format

FME Platform

Operating System

Reader/Writer

FME Form

FME Flow

FME Flow Hosted

Windows 64-bit

Linux

Mac

Reader

✅ ✅ ✅ ✅ ✅ ✅

Reader Overview

Each Pinecone index is a feature type, and each record becomes a feature. The reader has two read modes:

  • Extract reads every record in a namespace, or only the records whose metadata matches a filter.
  • Similarity Search returns the records most similar to a query vector or an existing record, most similar first, with a similarity score.

The reader connects to Pinecone cloud, or to Pinecone Local for development. It works in a FeatureReader, where each incoming feature can supply its own query vector, query ID or namespace from its attributes.

See Pinecone documentation for details on indexes, namespaces and metadata filters. If a read fails or returns fewer features than expected, see Pinecone Troubleshooting.

Limitations

Document indexes – The reader reads Pinecone's vector indexes. Indexes with a document schema, which Pinecone reads through its Documents API, can't be read: Pinecone rejects the read with a message saying the index has a document schema.

Sparse vectors – The reader reads dense vectors only. It has no attribute for sparse values.

  • Hybrid indexes (records with dense and sparse values): pinecone_vector holds the dense values, and the sparse values are not read. Similarity Search by Query Vector compares dense values only, so sparse values don't affect the scores. By Query ID, Pinecone uses both the record's dense and sparse values.
  • Sparse-only indexes: Extract reads each record's ID, namespace and metadata, without pinecone_vector. Similarity Search by Query Vector skips these indexes. By Query ID, Pinecone searches them with the record's sparse values.

Integrated embedding – For indexes where Pinecone embeds text, the reader doesn't embed text itself: Similarity Search can't search by text, and searching by Query Vector skips these indexes, since a vector only matches if it came from the same embedding model. Search by Query ID instead. Extract reads the stored vectors, and the source text as a metadata attribute named after the index's text field (for example chunk_text).