fmetools.plugins

This module provides base classes for FME plugins such as transformers.

FMEEnhancedTransformer is the recommended base class for transformers. Transformer developers should subclass it to implement their own transformers.

class fmetools.plugins.FMEBaseTransformer

Bases: object

Base class that represents the interface expected by the FME infrastructure for Python-based transformer implementations. In particular, this is the class-based API required by the PythonFactory:

FACTORY_DEF {*} PythonFactory
    FACTORY_NAME { $(XFORMER_NAME) }
    INPUT { FEATURE_TYPE $(XFORMER_NAME)_READY }
    SYMBOL_NAME my_library.my_module.TransformerImpl
    OUTPUT { PYOUTPUT FEATURE_TYPE $(XFORMER_NAME)_PROCESSED }

When executed by FME, this is approximately equivalent to:

from my_library.my_module import TransformerImpl
transformer = TransformerImpl()

PythonFactory does not require Python classes to inherit from this base class, but it expects them to have the same interface as this class.

This class can be used as a context manager to guarantee that close() is called. This is useful for writing tests.

See also

PythonFactory in the FME Factory and Function Documentation.

__init__()

FME instantiates this class, so it must not require any constructor arguments.

close() → None

Called at the end of translation. Override this method to perform any necessary cleanup or finalization operations.

pyoutput() may be called from this method.

has_support_for(support_type: int) → bool

New in version 2022.0: This method and its corresponding constants in fmeobjects.

This method is used by FME to check whether the transformer claims support for certain capabilities. Currently, the only supported type is fmeobjects.FME_SUPPORT_FEATURE_TABLE_SHIM, which determines support for Bulk Mode.

Why support Bulk Mode

When a transformer supports Bulk Mode, FME may pass features to input() that come from a feature table object. This allows significant performance gains when processing many features, but requires the transformer to follow certain rules around how it handles features.

See also

How FME Improves Performance with Bulk Mode.

How to support Bulk Mode

  • Features received by input() must not be copied or cached for later use.

  • Features received by input() must not be read or modified after being passed to pyoutput().

  • pyoutput() should not be given new FMEFeature instances. Doing so will automatically downgrade feature processing to individual mode.

  • Override this method. When support_type is fmeobjects.FME_SUPPORT_FEATURE_TABLE_SHIM, return True.

Violating these requirements may result in undefined behavior.

Illegal Examples

Copy and access later:

def input(self, feature):
    self._cached_features.append(feature)

def close(self):
    for feature in self._cached_features:  # not allowed
        self.pyoutput(feature)

Access after output:

def input(self, feature):
    self.pyoutput(feature)
    feature.setAttribute("attr name", "attr val")  # not allowed

Group-by processing:

def input(self, feature):
    self._cached_features.append(feature)

def process_group(self):
    for feature in self._cached_features:  # not allowed
        self.pyoutput(feature)
Parameters:

support_type – The type of support to check for. Currently, the only supported type is fmeobjects.FME_SUPPORT_FEATURE_TABLE_SHIM.

Returns:

True if the passed in support type is supported. The default implementation returns False.

input(feature: fmeobjects.FMEFeature) → None

Receive a feature from the transformer’s single input port.

This method is used instead of input_from() if the transformer has no input tags, meaning that the transformer’s INPUT_TAGS parameter is listed as <BLANK>.

Transformers typically receive a feature through this method, process it, modify the feature by adding output attributes, and then output the feature using pyoutput(). However, transformers may output any number of features for each input feature, or none at all. Transformers may also create new FMEFeature instances and output them.

Parameters:

feature – The feature to process.

input_from(feature: fmeobjects.FMEFeature, input_tag: str) → None

Receive a feature from input_tag.

This method is used instead of input() if the transformer has defined input tags listed in the transformer’s INPUT_TAGS parameter and the PythonFactory’s PY_INPUT_TAGS clause. Introduced in FME 2024.0.

Example of a PythonFactory definition with two input tags:

FACTORY_DEF {*} PythonFactory
    FACTORY_NAME { $(XFORMER_NAME) }
    PY_INPUT_TAGS INPUT0 INPUT1
    $(INPUT_LINES)
    SYMBOL_NAME { symbol_name }
    PY_OUTPUT_TAGS Output <Rejected>
    OUTPUT { Output FEATURE_TYPE $(OUTPUT_Output_FTYPE)
        $(OUTPUT_Output_FUNCS) }
    OUTPUT { <Rejected> FEATURE_TYPE $(OUTPUT_<Rejected>_FTYPE)
        $(OUTPUT_<Rejected>_FUNCS) }

Transformers typically receive a feature through this method, process it, modify the feature by adding output attributes, and then output the feature using pyoutput(). However, transformers may output any number of features for each input feature, or none at all. Transformers may also create new FMEFeature instances and output them.

Parameters:

  • feature – The feature to process.

  • input_tag – The input tag that feature came from.

process_group() → None

If group processing is enabled, then this is called after all the current group’s features have been sent to input(). Can be left unimplemented if group processing is not supported.

pyoutput() may be called from this method.

pyoutput(feature: fmeobjects.FMEFeature, output_tag: str | None = None) → None

Output a feature from the transformer to an output tag.

Note

Do not implement this method. FME injects the implementation at runtime.

Parameters:

  • feature – The feature to output.

  • output_tag – The output tag to direct feature to. This argument is required if multiple output tags exist in the PythonFactory definition. Otherwise, it will default to PYOUTPUT or the single output tag if specified in the PythonFactory definition. Introduced in FME 2024.0.

total_features_passed_along() → int

Note

Do not implement this method. FME injects the implementation at runtime.

Returns:

A count of features that have been processed to date, in all groups.

factory_name: str

This is the FACTORY_NAME parameter of the PythonFactory that instantiated this class. Defaults to the name of this class.

Note

Do not modify this property. FME sets the value at runtime.

class fmetools.plugins.FMEEnhancedTransformer

Bases: FMEBaseTransformer

This is the recommended base class for transformer implementations. It exposes the FME log file through the log property, and adds methods to introduce more granularity to the transformer lifecycle to help developers organize their code.

input() is broken down to these methods for implementations to overwrite:
input_from() is broken down to these methods for implementations to overwrite:
close() is broken down to these methods for implementations to overwrite:

A typical transformer would implement:

Note

This class overrides has_support_for() to return True for Bulk Mode support. This means that the transformer cannot cache or copy features for later use, and cannot output new fmeobjects.FMEFeature instances. See FMEBaseTransformer.has_support_for() for details about these restrictions.

__init__()

FME instantiates this class, so it must not require any constructor arguments.

close() → None

Do not override this method.

finish() → None

Override this instead of close().

has_support_for(support_type: int) → bool

Overrides the default implementation to report support for Bulk Mode.

Returns:

True if support_type is fmeobjects.FME_SUPPORT_FEATURE_TABLE_SHIM. See FMEBaseTransformer.has_support_for() for more details.

input(feature: fmeobjects.FMEFeature) → None

Do not override this method.

input_from(feature: fmeobjects.FMEFeature, input_tag: str) → None

Do not override this method.

post_close() → None

Called after close().

post_input(feature: fmeobjects.FMEFeature) → None

Called after each input().

post_input_from(feature: fmeobjects.FMEFeature, input_tag: str) → None

Called after each input_from().

pre_close() → None

Called before close().

pre_input(feature: fmeobjects.FMEFeature) → None

Called before each input().

pre_input_from(feature: fmeobjects.FMEFeature, input_tag: str) → None

Called before each input_from().

receive_feature(feature: fmeobjects.FMEFeature) → None

Override this method instead of input(). This method receives all input features, including the first one that’s also passed to setup().

receive_feature_from(feature: fmeobjects.FMEFeature, input_tag: str) → None

Override this method instead of input_from(). This method receives all input features from input_tag, including the first one that’s also passed to setup_from().

reject_feature(feature: fmeobjects.FMEFeature, code: str, message: str) → None

Output a feature with conventional attributes that represent rejection.

Method will first attempt to output to <Rejected>. If the <Rejected> tag doesn’t exist on the PythonFactory definition, then feature will be directed to PYOUTPUT.

For transformers that only support FME 2024.0+, the transformer definition file should:

  • Specify a PY_OUTPUT_TAGS clause in the PythonFactory definition

  • Add <Rejected> to OUTPUT_TAGS and PY_OUTPUT_TAGS

  • Specify <Rejected> output tag in the PythonFactory definition

Example of a PythonFactory definition for a transformer with two output ports:

FACTORY_DEF {*} PythonFactory
    FACTORY_NAME { $(XFORMER_NAME) }
    $(INPUT_LINES)
    SYMBOL_NAME { symbol_name }
    PY_OUTPUT_TAGS Output <Rejected>
    OUTPUT { Output FEATURE_TYPE $(OUTPUT_Output_FTYPE)
        $(OUTPUT_Output_FUNCS) }
    OUTPUT { <Rejected> FEATURE_TYPE $(OUTPUT_<Rejected>_FTYPE)
        $(OUTPUT_<Rejected>_FUNCS) }

To support versions earlier than FME 2024.0, the transformer definition file needs to specify a <Rejected> output port, and its Execution Instructions need some corresponding lines:

  • A TestFactory definition that sends features with the fme_rejection_code attribute to the rejection port.

  • Handling for the possibility of the transformer’s initiator/input feature coming in with fme_rejection_code already defined. The transformer should not send features to the rejection port unless the feature was actually rejected by the transformer. If the input feature included rejection attributes, the transformer should pass them through in its output features. If the transformer happens to reject such a feature, it’s free to overwrite those existing attributes.

Example of a PythonFactory and TestFactory definition for a transformer with two output ports:

FACTORY_DEF {*} PythonFactory
    FACTORY_NAME { $(XFORMER_NAME) }
    INPUT { FEATURE_TYPE $(XFORMER_NAME)_READY }
    SYMBOL_NAME { symbol_name }
    OUTPUT { PYOUTPUT FEATURE_TYPE $(XFORMER_NAME)_PROCESSED }

# Removed all internal-prefixed attributes from output feature
# and emit to the correct output port based on value of fme_rejection_code.
FACTORY_DEF {*} TestFactory
    FACTORY_NAME { $(XFORMER_NAME)_ROUTER }
    INPUT { FEATURE_TYPE $(XFORMER_NAME)_PROCESSED }
    TEST &fme_rejection_code == ""
    OUTPUT { PASSED FEATURE_TYPE $(OUTPUT_Output_FTYPE)
        @RenameAttributes(FME_STRICT,fme_rejection_code,___fme_rejection_code___)
        @RemoveAttributes(fme_regexp_match,^<internal prefix>.*$)
        $(OUTPUT_Output_FUNCS) }
    OUTPUT { FAILED FEATURE_TYPE $(OUTPUT_<REJECTED>_FTYPE)
        @RemoveAttributes(___fme_rejection_code___)
        @RemoveAttributes(fme_regexp_match,^<internal prefix>.*$)
        $(OUTPUT_<REJECTED>_FUNCS) }
Parameters:

  • feature – Feature to reject. Rejection attributes are added to this feature. Then it is passed to pyoutput().

  • code – Value for the fme_rejection_code attribute.

  • message – Value for the fme_rejection_message attribute.

setup(first_feature: fmeobjects.FMEFeature) → None

This method is only called for the first input feature. Implement this method to perform any necessary setup operations, such as getting constant parameters.

Constant parameters are ones that cannot change across input features. For parameters defined using GUI Types, these are ones that do not include OR_ATTR. For parameters defined with Transformer Designer, these are ones with Value Type Level not set to “Full Expression Support”.

Parameters:

first_feature – First input feature.

setup_from(first_feature: fmeobjects.FMEFeature, input_tag: str) → None

This method is only called for the first input feature from each unique input tag. Implement this method to perform any necessary setup operations, such as getting constant parameters.

Constant parameters are ones that cannot change across input features. For parameters defined using GUI Types, these are ones that do not include OR_ATTR. For parameters defined with Transformer Designer, these are ones with Value Type Level not set to “Full Expression Support”.

Parameters:

  • first_feature – First input feature.

  • input_tag – Input tag that first_feature came from.

property log: LoggerAdapter

Provides access to the FME log.