FME Pluginbuilder API reference

This is the reference documentation for the FME Pluginbuilder API for Python.

The FME Pluginbuilder API contains the abstract base classes for implementing Python-based reader/writer plugins for FME.

FMEMappingFile	Mapping File Class
FMEReader	Reader Superclass
FMEWriter	Writer Superclass

Mapping File

class FMEMappingFile

Mapping File Class

Developers do not directly create instances of this class. FME will instantiate and provide access to the instance they should be using.

__init__(*args, **kwargs)

fetch(keyword)

Fetches the value for the passed in keyword.

Parameters:

keyword (str) – The keyword to fetch.

Return type:

string, int, float, list, None

Returns:

The value matching the keyword. None means there was no matching value.

fetchFeatureTypes(formatKeywordPrefix, formatTypePrefix, parameters, fetchMode)

Retrieve all the feature types for a reader by examining the parameters or the mapping file object and searching for +ID, _IDs and DEF keywords and finding the intersection between the IDs and DEFs.

Parameters:

• formatKeywordPrefix (str) – Input reader keyword prefix for fetching.

• formatTypePrefix (str) – Input reader type prefix for fetching.

• parameters (list[str]) – Open parameters for the reader.

• fetchMode (str) – kFetchFeatureTypes_IDsOnly, kFetchFeatureTypes_DEFsOnly, or kFetchFeatureTypes_IDsAndDEFs.

Return type:

list[str] or None

Returns:

The list of feature type names from the IDs and DEFs. If no IDs and DEFs exist, then None is returned.

fetchFlag(keyword)

Checks if the keyword was in the mapping file with a value of yes, YES, true, or TRUE.

Parameters:

keyword (str) – The string that is checked in the mapping file.

Return type:

bool

Returns:

True if the keyword was in the mapping file with a value of yes, YES, true, or TRUE. False otherwise.

fetchFlagDefaultTrue(keyword)

Checks if the keyword was in the mapping file with a value of yes, YES, true, or TRUE, or when the keyword was not found.

Parameters:

keyword (str) – The string that is checked in the mapping file.

Return type:

bool

Returns:

True if the keyword was in the mapping file with a value of yes, YES, true, or TRUE, or when the keyword was not found. False otherwise.

fetchFlagWithPrefix(readerKeywordPrefix, readerTypePrefix, keyword)

Checks if the keyword was in the mapping file with a value of yes, YES, true, or TRUE. First the mapping file is searched for the keyword with the reader keyword prefix, then it searches with the reader type prefix, and finally with no prefix.

Parameters:

• readerKeywordPrefix (str) – Reader keyword prefix str.

• readerTypePrefix (str) – Reader type prefix str.

• keyword (str) – The string that is checked in the mapping file.

Return type:

bool

Returns:

True if the keyword was in the mapping file with a value of yes, YES, true, or TRUE. False otherwise.

fetchFlagWithPrefixDefaultTrue(readerKeywordPrefix, readerTypePrefix, keyword)

Checks if the keyword was in the mapping file with a value of yes, YES, true, or TRUE, or when the keyword was not found. First the mapping file is searched for the keyword with the reader keyword prefix, then it searches with the reader type prefix, and finally with no prefix.

Parameters:

• readerKeywordPrefix (str) – Reader keyword prefix string.

• readerTypePrefix (str) – Reader type prefix string.

• keyword (str) – The string that is checked in the mapping file.

Return type:

bool

Returns:

True if the keyword was in the mapping file with a value of yes, YES, true, or TRUE, or when the keyword was not found. False otherwise.

fetchSearchEnvelope(formatKeywordPrefix, formatTypePrefix)

Retrieve a search envelope in the mapping file. This can be specified either by a single line: prefix_SEARCH_ENVELOPE minX minY maxX maxY

or in separate lines:

• prefix_MINX minX

• prefix_MINY minY

• prefix_MAXX maxX

• prefix_MAXY maxY

If a search for the single-line method succeeds, then no check will be made for the multi-line clauses.

Parameters:

• formatKeywordPrefix (str) – Input reader keyword prefix for fetching.

• formatTypePrefix (str) – Input reader type prefix for fetching.

Return type:

tuple[tuple[float]] or None

Returns:

The tuple of search envelope in the format ((minX, minY), (maxX, maxY)). If no search envelope exists, then None is returned.

fetchWithPrefix(readerKeywordPrefix, readerTypePrefix, keyword)

Fetches the value for the passed in keyword. First the mapping file is searched for the keyword with the reader keyword prefix, then it searches with the reader type prefix, and finally with no prefix.

Parameters:

• readerKeywordPrefix (str) – Reader keyword prefix string.

• readerTypePrefix (str) – Reader type prefix string.

• keyword (str) – The string that is checked in the mapping file.

Return type:

str, int, float, list[str], None

Returns:

The value matching the keyword. None means there was no matching value.

getMacroNames()

This method retrieves all the macro names that were defined at the end of the mapping file parsing. Each name can then be used in the fetchFlag method to get its actual value.

Return type:

list[str]

Returns:

The list of macro names.

getMacroValueExpanded(macroName)

This routine retrieves the value of a predefined FME macro.

Parameters:

macroName (str) – Name of the predefined FME macro.

Return type:

str or None

Returns:

The macro value is returned if the macro name is found in the dictionary, otherwise None is returned. If the value itself contained any macro references, these are expanded before the value is returned and if it referenced any undefined macros, they are replaced with a zero length string.

nextLine()

Gets the next logical line in the mapping file.

Return type:

list[str] or None

Returns:

The next logical line separated into individual components in the list. None is returned when there were no more lines available.

nextLineWithFilter(filter)

Gets the next logical line in the mapping file whose first string is equal to the filter string.

Parameters:

filter (str) – Value used for filtering.

Return type:

list[str] or None

Returns:

The next logical line separated into individual components in the list. None is returned when there were no more lines available.

startIteration()

Initializes the mapping file object to iterate through the entire mappingfile, one line at a time.

Return type:

None

Reader

class FMEReader

Reader Superclass

Developers MUST implement their own subclasses of this. FME will then use the subclass instance provided by the developer’s FME_createReader method in their plugin.

The protocol for using this class is:

1. An instance is constructed and initialized by the developer’s FME_createReader method in their plugin.

2. FME will call open when it is time for the dataset to be opened.

3. read will be called repeatedly, to retrieve features from the datastore. When the end-of-data is reached, read will return None indicating this.

4. FME will call close to shut down the reader.

5. At any time during (3), abort could be called to terminate the reading. Later the close method would be called to properly shut down the reader. For most implementations, abort will do nothing.

There is also an alternate protocol which is used when the reader provides schema information about its input data source to FME for mapping file generation purposes. In this case, the below protocol is used:

1. An instance is constructed and initialized by the developer’s FME_createReader method in their plugin.

2. FME will call open when it is time for the dataset to be opened.

3. readSchema will be called repeatedly. Each time it sends back another feature which holds ‘schema’ information (i.e. attribute names and their types) for the source dataset. When all the schema information has been sent, readSchema will return None indicating this.

4. FME will call close to shut down the reader.

A fmeobjects.FMEException may be raised at any time to indicate an error has occurred.

init(readerTypeName, readerKeyword, mappingFile)

Creates the reader with the FME provided members.

Parameters:

• readerTypeName (str) –

  The name used in the following contexts:

  • the name of the format’s .db file in the formats info folder

  • the format short name for the format within the .db file

  • FORMAT_NAME in the metafile

• readerKeyword (str) – a unique identifier for this reader instance.

• mappingFile (FMEMappingFile) – the FMEMappingFile

__init__(*args, **kwargs)

abort()

This method is called by FME immediately after a failed call to read or readSchema to perform any necessary clean up before close is called. Any special actions to shut down a reader not finished reading data should be taken in this method. For example, you could log a message or send an email, but normally it does nothing.

Return type:

None

Returns:

The return value is ignored.

Raises:

FMEException – The method should raise an fmeobjects.FMEException to signal that an error has occurred.

close()

This method is called by FME after the dataset has been exhausted (e.g. read indicated an EOF). It is also called after abort if an error occurs mid-stream.

WARNING: This method can be called multiple times for a single open call.

Return type:

None

Returns:

The return value is ignored.

Raises:

FMEException – The method should raise an fmeobjects.FMEException to signal that an error has occurred.

getProperties(propertyCategory)

This method should return a list of str pairs, as appropriate for that property category. Therefore the returned list should always have an even number of entries. The property category may choose to return an empty list if the property category was not recognized, the method itself will have a return value of None.

Currently this method handles the following property categories:

• 'fme_search_type': If the reader being used supports different search types in setConstraints, then the returned list should contain the property/value pairs for all the search types it can support. some example search type property/value pairs are:

  • ‘fme_search_type’ ‘fme_envelope_intersects’

  • ‘fme_search_type’ ‘fme_all_features’

  • ‘fme_search_type’ ‘fme_metadata’

  • ‘fme_search_type’ ‘fme_db_join’

  • ‘fme_search_type’ ‘fme_execute_sql’

  • If the reader does not know if it supports search types in setConstraints, it should return None.

• 'fme_envelope_intersects': If the reader being used supports the ‘fme_envelope_intersects’ search type in setConstraints, then the returned list should contain the property/value pairs for all the specific search type primitives it can support. Some example search type primitive property/value pairs in this case are:

  • ‘fme_envelope_intersects’ ‘fme_envelope_order_results’

  • ‘fme_envelope_intersects’ ‘fme_feature_type’

  • ‘fme_envelope_intersects’ ‘fme_where’

  • ‘fme_envelope_intersects’ ‘fme_test’

  • ‘fme_envelope_intersects’ ‘fme_dynamic_attr_list’

  • If the reader does not know if it supports search type primitives for this case in setConstraints, it should return None.

• 'fme_all_features': If the reader being used supports the ‘fme_all_features’ search type in setConstraints, then the returned list should contain the property/value pairs for all the specific search type primitives it can support. Some example search type primitive property/value pairs in this case are:

  • ‘fme_all_features’ ‘fme_feature_type’

  • ‘fme_all_features’ ‘fme_where’

  • ‘fme_all_features’ ‘fme_test’

  • ‘fme_all_features’ ‘fme_dynamic_attr_list’

  • If the reader does not know if it supports search type primitives for this case in setConstraints, it should return None.

• 'fme_metadata': If the reader being used supports the ‘fme_metadata’ search type in setConstraints, then the returned list should contain the property/value pairs for all the metadata it can return for a specific feature type. Some example property/value pairs in this case are:

  • ‘fme_metadata’ ‘fme_feature_identifier’

  • ‘fme_metadata’ ‘fme_geometry’

  • ‘fme_metadata’ ‘fme_contains_spatial_column’

  • ‘fme_metadata’ ‘fme_dimension’

  • ‘fme_metadata’ ‘fme_num_entries’

  • ‘fme_metadata’ ‘fme_min_x’

  • ‘fme_metadata’ ‘fme_min_y’

  • ‘fme_metadata’ ‘fme_min_z’

  • ‘fme_metadata’ ‘fme_max_x’

  • ‘fme_metadata’ ‘fme_max_y’

  • ‘fme_metadata’ ‘fme_max_z’

  • ‘fme_metadata’ ‘fme_feature_type’

  • ‘fme_metadata’ ‘fme_where’

  • ‘fme_metadata’ ‘fme_test’

  • ‘fme_metadata’ ‘fme_dynamic_attr_list’

  • If the reader does not know if it supports search type primitives for this case in setConstraints, it should return None.

• 'fme_db_join' - If the reader being used supports the ‘fme_db_join’ search type in setConstraints, then the returned list should contain the property/value pairs for all the specific search type primitives it can support. Some example search type primitive property/value pairs in this case are:

  • ‘fme_db_join’ ‘fme_select_columns’

  • ‘fme_db_join’ ‘fme_key_columns’

  • ‘fme_db_join’ ‘fme_key_values’

  • ‘fme_db_join’ ‘fme_feature_type’

  • ‘fme_db_join’ ‘fme_where’

  • ‘fme_db_join’ ‘fme_test’

  • ‘fme_db_join’ ‘fme_dynamic_attr_list’

  • If the reader does not know if it supports search type primitives for this case in setConstraints, it should return None.

• 'fme_execute_sql': If the reader being used supports the ‘fme_execute_sql’ search type in setConstraints, then the returned list should contain the property/value pairs for all the specific search type primitives it can support. Some example search type primitive property/value pairs in this case are:

  • ‘fme_execute_sql’ ‘fme_select_columns’

  • ‘fme_execute_sql’ ‘fme_sql_statement’

  • ‘fme_execute_sql’ ‘fme_feature_type’

  • ‘fme_execute_sql’ ‘fme_where’

  • ‘fme_execute_sql’ ‘fme_test’

  • ‘fme_execute_sql’ ‘fme_dynamic_attr_list’

  • If the reader does not know if it supports search type primitives for this case in setConstraints, it should return None.

• 'fme_get_version_list': If the reader being used supports the ‘fme_get_version_list’ search type in setConstraints, then the returned list should contain the property/value pairs for all the information that can be returned about a specific version. Some example property/value pairs in this case are:

  • ‘fme_get_version_list’ ‘fme_version_access’

  • ‘fme_get_version_list’ ‘fme_version_creation_time’

  • ‘fme_get_version_list’ ‘fme_version_description’

  • ‘fme_get_version_list’ ‘fme_version_name’

  • ‘fme_get_version_list’ ‘fme_version_parent_name’

  • If the reader does not know if it supports this search type, it should return None.

• '*' : Returns all properties for all known property categories. Each property is returned as a property/value pair of entries.

Refer to IFMEReader setConstraints() Documentation for additional information.

Return type:

list[str] or None

Returns:

Return a list of the property/value pairs for the specified property category. Return None if the property category is not recognized or support for the property category is unknown.

hasSupportFor(supportType)

Return whether this reader supports a certain type. Currently, the only supported type is fmeobjects.FME_SUPPORT_FEATURE_TABLE_SHIM.

When a reader supports fmeobjects.FME_SUPPORT_FEATURE_TABLE_SHIM, FME will copy the returned features from read() to a feature table feature. This will allow for significant performance gains when processing a large number of features.

Returns:

True if the passed in support type is supported.

Return type:

bool

open(datasetName, parameters)

This method opens the reader on a specified dataset using the parameters provided. A reader may be opened to read schema features or data features but not both. i.e. when a reader is opened to read schema features readSchema is called but read is not; when a reader is opened to read data features, read is called but readSchema is not. When a reader is opened to read schema features, there is no mapping file context (see comments for readSchema). In this situation, all the information needed by the reader is passed in through the parameters list The first entry of the parameters list will always contain a copy of the ‘datasetName’. The remaining entries contain:

• The name/value pairs for any reader keywords listed on the SOURCE_READER line metafile;

• A list of feature types specified as ‘ID’/value pairs, which are used to limit the number of schema features returned. For example, when open is called to read schema features, the parameters array might contain: (‘c:\data\parcels.tfs’, ‘GRID_SIZE’, ‘1000’, ‘RESOLVE_AGGREGATES’, ‘yes’).

When a reader is opened to read data features, the parameters list is empty and the reader is responsible for fetching its keywords and IDs from the mapping file object.

Parameters:

datasetName (str) – A folder, file or database to be read.

Return type:

None

Returns:

The return value is ignored.

Raises:

FMEException – The method should raise an fmeobjects.FMEException to signal that an error has occurred.

read()

This method is called repeatedly by FME until it returns None, indicating to FME that reading is complete.

The caller takes ownership of features returned by this method. Do not keep references to returned features.

Return type:

fmeobjects.FMEFeature or None

Returns:

Return the next feature in the dataset. If there are no more features to read, return None.

readSchema()

This method is called repeatedly by FME to get the next schema feature from the input dataset until it returns None, indicating that all schema features have been returned. It should provide a unique schema feature each time it is called. This method must be implemented by readers that support either of the following two options:

1. Automated translation support: i.e. mapping file generation and drag-and-drop translation.

2. FME Objects support: i.e. can be used from FME Objects applications such as the FME Universal Viewer.

In either of these two situations, a metafile is required.

A valid schema feature is built by specifying the following information:

• Feature type: The feature type described by the schema feature corresponds to theme, layer, level, file, or database table name depending on the input format. i.e. fmeobjects.FMEFeature.setFeatureType (‘MyFeatureType’).

• User attributes: If the format supports user-defined attributes, each user attribute that describes data features of this type is added using the sequencedAttribute property. i.e. fmeobjects.FMEFeature.setAttribute (‘name’, ‘type’) The format-specific types used to define user attributes are converted to generic FME types using the Attribute Type Map (ATTR_TYPE_MAP) provided in the metafile.

• Allowable geometries: The list of one or more geometry types that are valid for data features of this type is specified in a list attribute named ‘fme_geometry’ using the fmeobjects.FMEFeature.setAttribute (list) method.

Schema features can optionally specify a coordinate system, if it is known.

Return type:

fmeobjects.FMEFeature or None

Returns:

Return the next schema feature in the dataset. If there are no more schema features, return None.

Raises:

FMEException – The method should raise an fmeobjects.FMEException to signal that an error has occurred.

setConstraints(feature)

Specifies the spatial and attribute constraints to be used when reading the data. The spatial and attribute query is specified in the feature object. This only need be implemented which return spatialEnabled as True, otherwise they can do nothing. This can be called at any time after the reader is created. If any read is in progress then it is terminated and the next read will reflect the new constraints.

Parameters:

feature (fmeobjects.FMEFeature) – The constraint feature must have an attribute called ‘fme_search_type’ on it. The value of this attribute indicates the kind of search to be done. Refer to IFMEReader setConstraints() Documentation for more detailed information.

Return type:

None

Returns:

The return value is ignored.

Raises:

FMEException – The method should raise an fmeobjects.FMEException to signal that an error has occurred.

spatialEnabled()

Indicates whether or not this reader supports spatial constraints.

Return type:

bool

Returns:

By default this returns False. If a reader does support spatial constraints then it returns True.

Constants

Fetch Mode Constants

kFetchFeatureTypes_IDsOnly

kFetchFeatureTypes_DEFsOnly

kFetchFeatureTypes_IDsAndDEFs

Writer

class FMEWriter

Writer Superclass

Developers MUST implement their own subclasses of this. FME will then use the subclass instance provided by the developer’s FME_createWriter method in their plugin.

The protocol for using this class is:

1. An instance is constructed and initialized by the developer’s FME_createWriter method in their plugin.

2. FME will call open when it is time for the dataset to be opened.

3. write will be called repeatedly, to write features to the datastore.

4. When FME runs out of features to send to the writer, it will call close to shut it down.

5. At any time during (3), abort could be called to terminate the writing. Later the close method would be called to properly shut down the writer. For most implementations, abort will do nothing. A fmeobjects.FMEException may be raised at any time to indicate an error has occured.

init(writerType, destKeyword, mappingFile)

Creates the reader with the FME provided parameters.

Parameters:

• writerType (str) –

  The name used in the following contexts:

  • the name of the format’s .db file in the formats info folder.

  • the format short name for the format within the .db file.

  • FORMAT_NAME in the metafile.

• destKeyword (str) – a unique identifier for this writer instance.

• mappingFile (FMEMappingFile) – the FMEMappingFile.

__init__(*args, **kwargs)

abort()

This method is called by FME immediately after a failed call to write to perform any necessary clean up before close is called. Any special actions to shut down a writer not finished reading data should be taken in this method. For example, if your format requires a footer at the end of file, it would be written from this method.

Return type:

None

Returns:

The return value is ignored.

Raises:

FMEException – The method should raise an fmeobjects.FMEException to signal that an error has occurred.

addMappingFileDefLine(defLine)

This method adds a _DEF line to the writer. For formats or systems which do not support schema from schema features, nothing is done.

Return type:

None

Returns:

The return value is ignored.

Raises:

FMEException – The method should raise an fmeobjects.FMEException to signal that an error has occurred.

close()

This method is called by FME after it is done sending features to the writer. It is also called after abort if an error occurs mid-stream.

WARNING: This method can be called multiple times for a single open call.

Return type:

None

Returns:

The return value is ignored.

Raises:

FMEException – The method should raise an fmeobjects.FMEException to signal that an error has occurred.

commitTransaction()

This method commits the current Transaction. For formats or systems which do not have the notion of Transactions the nothing is done.

Return type:

None

Returns:

The return value is ignored.

Raises:

FMEException – The method should raise an fmeobjects.FMEException to signal that an error has occurred.

multiFileWriter()

In a single-writer architecture only one writer instance is created and opened. In a multi-writer architecture, an instance is created for each feature type and additional information, which is specific to each feature type, is passed in the open call. Refer to open for details regarding the additional information. In the following scenario:

• Single-writer architecture

• Mapping file _DEF lines used to retrieve schema feature information

The writer must explicitly handle ‘Feature Type Fanout By Attribute’ (see FME Help for details). When a writer is used in Fanout mode, the feature type of a data feature sent to the writer will not match the feature type defined in any _DEF line. To operate correctly in this scenario, a writer must retrieve _DEF line information based on the ‘fme_template_feature_type’ attribute. That matching _DEF line’s values should be used for the given data feature to determine the schema definition. If you require assistance for this advanced scenario contact support@fmeobjects.com. In the case of a multi-writer architecture, the ‘Feature Type Fanout By Attribute’ is handled automatically by FME. Typically, single-writer architecture is used for creating a single file writer and a multi-writer architecture is used for creating a directory writer. Database writers can use both single and multi-writer architecture depending on the implementation.

Return type:

bool

Returns:

Return True if the writer is using a multi-writer architecture and returns False if the writer is using single-writer architecture.

open(datasetName, parameters)

This method opens the writer on a specified dataset using the parameters provided. In a single-writer architecture, the parameters list is always empty and the writer can optionally retrieve context information (e.g. directives, schema, etc.) from FME. In a multi-writer architecture, a writer instance will be created for each feature type that has schema information. When this method is called for a writer instance, the parameters list contains the feature type followed by attribute name/type pairs. For example, when open is called in a multi-writer architecture, the parameters list might be: (‘featureType’, ‘attribute1’, ‘char(50)’, ‘attribute2’, ‘number(31,15)’).

Parameters:

datasetName (str) – A folder, file or database to be written.

Return type:

None

Returns:

The return value is ignored.

Raises:

FMEException – The method should raise an fmeobjects.FMEException to signal that an error has occurred.

rollbackTransaction()

This method rolls back the current Transaction. For formats or systems which do not have the notion of Transactions the nothing is done.

Return type:

None

Returns:

The return value is ignored.

Raises:

FMEException – The method should raise an fmeobjects.FMEException to signal that an error has occurred.

startTransaction()

This method starts a new Transaction. For formats or systems which do not have the notion of Transactions the nothing is done.

Return type:

None

Returns:

The return value is ignored.

Raises:

FMEException – The method should raise an fmeobjects.FMEException to signal that an error has occurred.

write(feature)

This method writes a feature to the output dataset.

Return type:

None

Returns:

The return value is ignored.

Raises:

FMEException – The method should raise an fmeobjects.FMEException to signal that an error has occurred.