fmetools.plugins

This module provides base classes for FME plugins such as transformers.

FMEEnhancedTransformer is the recommended base class for transformers. Transformer developers should subclass it to implement their own transformers.

class fmetools.plugins.FMEEnhancedTransformer(*args: Any, **kwargs: Any)

Bases: BaseTransformer

This is the recommended base class for transformer implementations. It exposes the FME log file through the log property, and adds methods to introduce more granularity to the transformer lifecycle to help developers organize their code.

In FME versions prior to 2024.2, this will subclass fmetools._deprecated.FMEBaseTransformer instead.

input() is broken down to these methods for implementations to overwrite:

pre_input()
- setup() which is only called for the first input feature
receive_feature() which should contain the bulk of the main logic
post_input()

input_from() is broken down to these methods for implementations to overwrite:

pre_input_from()
- setup_from() which is only called for the first input feature from each input tag
receive_feature_from() which should contain the bulk of the main logic
post_input_from()

close() is broken down to these methods for implementations to overwrite:

pre_close()
finish() which should contain any cleanup tasks
post_close()

A typical transformer would implement:

setup() or setup_from() to collect constant transformer parameters off the first input feature and use them to do any initialization steps.

receive_feature() or receive_feature_from() to process each input feature.

finish() to delete any temporary files or close any connections.

Note

This class overrides has_support_for() to return True for Bulk Mode support. This means that the transformer cannot cache or copy features for later use, and cannot output new fmeobjects.FMEFeature instances. See fme.BaseTransformer.has_support_for() for details about these restrictions.

__init__()

close() → None: Do not override this method.

finish() → None: Override this instead of close().

has_support_for(support_type: int) → bool

Overrides the default implementation to report support for Bulk Mode.

Returns:: True if support_type is fmeobjects.FME_SUPPORT_FEATURE_TABLE_SHIM. See fme.BaseTransformer.has_support_for() for more details.

input(feature: fmeobjects.FMEFeature) → None: Do not override this method.

input_from(feature: fmeobjects.FMEFeature, input_tag: str) → None: Do not override this method.

post_close() → None: Called after close().

post_input(feature: fmeobjects.FMEFeature) → None: Called after each input().

post_input_from(feature: fmeobjects.FMEFeature, input_tag: str) → None: Called after each input_from().

pre_close() → None: Called before close().

pre_input(feature: fmeobjects.FMEFeature) → None: Called before each input().

pre_input_from(feature: fmeobjects.FMEFeature, input_tag: str) → None: Called before each input_from().

receive_feature(feature: fmeobjects.FMEFeature) → None: Override this method instead of input(). This method receives all input features, including the first one that’s also passed to setup().

receive_feature_from(feature: fmeobjects.FMEFeature, input_tag: str) → None: Override this method instead of input_from(). This method receives all input features from input_tag, including the first one that’s also passed to setup_from().

reject_feature(feature: fmeobjects.FMEFeature, code: str, message: str) → None

Output a feature with conventional attributes that represent rejection.

Method will first attempt to output to <Rejected>. If the <Rejected> tag doesn’t exist on the PythonFactory definition, then feature will be directed to PYOUTPUT.

For transformers that only support FME 2024.0+, the transformer definition file should:

Specify a PY_OUTPUT_TAGS clause in the PythonFactory definition
Add <Rejected> to OUTPUT_TAGS and PY_OUTPUT_TAGS
Specify <Rejected> output tag in the PythonFactory definition

Example of a PythonFactory definition for a transformer with two output ports:

FACTORY_DEF {*} PythonFactory
    FACTORY_NAME { $(XFORMER_NAME) }
    $(INPUT_LINES)
    SYMBOL_NAME { symbol_name }
    PY_OUTPUT_TAGS { Output <Rejected> }
    OUTPUT { Output FEATURE_TYPE $(OUTPUT_Output_FTYPE)
        $(OUTPUT_Output_FUNCS) }
    OUTPUT { <Rejected> FEATURE_TYPE $(OUTPUT_<Rejected>_FTYPE)
        $(OUTPUT_<Rejected>_FUNCS) }

To support versions earlier than FME 2024.0, the transformer definition file needs to specify a <Rejected> output port, and its Execution Instructions need some corresponding lines:

A TestFactory definition that sends features with the fme_rejection_code attribute to the rejection port.
Handling for the possibility of the transformer’s initiator/input feature coming in with fme_rejection_code already defined. The transformer should not send features to the rejection port unless the feature was actually rejected by the transformer. If the input feature included rejection attributes, the transformer should pass them through in its output features. If the transformer happens to reject such a feature, it’s free to overwrite those existing attributes.

Example of a PythonFactory and TestFactory definition for a transformer with two output ports:

FACTORY_DEF {*} PythonFactory
    FACTORY_NAME { $(XFORMER_NAME) }
    INPUT { FEATURE_TYPE $(XFORMER_NAME)_READY }
    SYMBOL_NAME { symbol_name }
    OUTPUT { PYOUTPUT FEATURE_TYPE $(XFORMER_NAME)_PROCESSED }

# Removed all internal-prefixed attributes from output feature
# and emit to the correct output port based on value of fme_rejection_code.
FACTORY_DEF {*} TestFactory
    FACTORY_NAME { $(XFORMER_NAME)_ROUTER }
    INPUT { FEATURE_TYPE $(XFORMER_NAME)_PROCESSED }
    TEST &fme_rejection_code == ""
    OUTPUT { PASSED FEATURE_TYPE $(OUTPUT_Output_FTYPE)
        @RenameAttributes(FME_STRICT,fme_rejection_code,___fme_rejection_code___)
        @RemoveAttributes(fme_regexp_match,^<internal prefix>.*$)
        $(OUTPUT_Output_FUNCS) }
    OUTPUT { FAILED FEATURE_TYPE $(OUTPUT_<REJECTED>_FTYPE)
        @RemoveAttributes(___fme_rejection_code___)
        @RemoveAttributes(fme_regexp_match,^<internal prefix>.*$)
        $(OUTPUT_<REJECTED>_FUNCS) }

Parameters:

feature – Feature to reject. Rejection attributes are added to this feature. Then it is passed to pyoutput().
code – Value for the fme_rejection_code attribute.
message – Value for the fme_rejection_message attribute.

setup(first_feature: fmeobjects.FMEFeature) → None

This method is only called for the first input feature. Implement this method to perform any necessary setup operations, such as getting constant parameters.

Constant parameters are ones that cannot change across input features. For parameters defined using GUI Types, these are ones that do not include OR_ATTR. For parameters defined with Transformer Designer, these are ones with Value Type Level not set to “Full Expression Support”.

Parameters:: first_feature – First input feature.

setup_from(first_feature: fmeobjects.FMEFeature, input_tag: str) → None

This method is only called for the first input feature from each unique input tag. Implement this method to perform any necessary setup operations, such as getting constant parameters.

Constant parameters are ones that cannot change across input features. For parameters defined using GUI Types, these are ones that do not include OR_ATTR. For parameters defined with Transformer Designer, these are ones with Value Type Level not set to “Full Expression Support”.

Parameters:

first_feature – First input feature.
input_tag – Input tag that first_feature came from.

property log: LoggerAdapter: Provides access to the FME log.

class fmetools._deprecated.FMEBaseTransformer

Bases: object

In FME 2024.2, this base class was deprecated and replaced with fme.BaseTransformer.

Base class that represents the interface expected by the FME infrastructure for Python-based transformer implementations. In particular, this is the class-based API required by the PythonFactory:

FACTORY_DEF {*} PythonFactory
    FACTORY_NAME { $(XFORMER_NAME) }
    INPUT { FEATURE_TYPE $(XFORMER_NAME)_READY }
    SYMBOL_NAME my_library.my_module.TransformerImpl
    OUTPUT { PYOUTPUT FEATURE_TYPE $(XFORMER_NAME)_PROCESSED }

When executed by FME, this is approximately equivalent to:

from my_library.my_module import TransformerImpl
transformer = TransformerImpl()

PythonFactory does not require Python classes to inherit from this base class, but it expects them to have the same interface as this class.

This class can be used as a context manager to guarantee that close() is called. This is useful for writing tests.

How to support Bulk Mode

Features received by input() must not be copied or cached for later use.
Features received by input() must not be read or modified after being passed to pyoutput().
pyoutput() should not be given new FMEFeature instances. Doing so will automatically downgrade feature processing to individual mode.
Override this method. When support_type is fmeobjects.FME_SUPPORT_FEATURE_TABLE_SHIM, return True.

Violating these requirements may result in undefined behavior.

Illegal Examples

Copy and access later:

def input(self, feature):
    self._cached_features.append(feature)

def close(self):
    for feature in self._cached_features:  # not allowed
        self.pyoutput(feature)

Access after output:

def input(self, feature):
    self.pyoutput(feature)
    feature.setAttribute("attr name", "attr val")  # not allowed

Group-by processing:

def input(self, feature):
    self._cached_features.append(feature)

def process_group(self):
    for feature in self._cached_features:  # not allowed
        self.pyoutput(feature)

Parameters:: support_type – The type of support to check for. Currently, the only supported type is fmeobjects.FME_SUPPORT_FEATURE_TABLE_SHIM.
Returns:: True if the passed in support type is supported. The default implementation returns False.

input(feature: fmeobjects.FMEFeature) → None

Receive a feature from the transformer’s single input port.

This method is used instead of input_from() if the transformer has no input tags, meaning that the transformer’s INPUT_TAGS parameter is listed as <BLANK>.

Transformers typically receive a feature through this method, process it, modify the feature by adding output attributes, and then output the feature using pyoutput(). However, transformers may output any number of features for each input feature, or none at all. Transformers may also create new FMEFeature instances and output them.

Parameters:: feature – The feature to process.

input_from(feature: fmeobjects.FMEFeature, input_tag: str) → None

Receive a feature from input_tag.

This method is used instead of input() if the transformer has defined input tags listed in the transformer’s INPUT_TAGS parameter and the PythonFactory’s PY_INPUT_TAGS clause. Introduced in FME 2024.0.

Example of a PythonFactory definition with two input tags:

FACTORY_DEF {*} PythonFactory
    FACTORY_NAME { $(XFORMER_NAME) }
    PY_INPUT_TAGS { INPUT0 INPUT1 }
    $(INPUT_LINES)
    SYMBOL_NAME { symbol_name }
    PY_OUTPUT_TAGS { Output <Rejected> }
    OUTPUT { Output FEATURE_TYPE $(OUTPUT_Output_FTYPE)
        $(OUTPUT_Output_FUNCS) }
    OUTPUT { <Rejected> FEATURE_TYPE $(OUTPUT_<Rejected>_FTYPE)
        $(OUTPUT_<Rejected>_FUNCS) }

Transformers typically receive a feature through this method, process it, modify the feature by adding output attributes, and then output the feature using pyoutput(). However, transformers may output any number of features for each input feature, or none at all. Transformers may also create new FMEFeature instances and output them.

Parameters:

feature – The feature to process.
input_tag – The input tag that feature came from.

process_group() → None

If group processing is enabled, then this is called after all the current group’s features have been sent to input(). Can be left unimplemented if group processing is not supported.

pyoutput() may be called from this method.

pyoutput(feature: fmeobjects.FMEFeature, output_tag: str | None = None) → None

Output a feature from the transformer to an output tag. If an output tag is specified and does not exist on the PythonFactory, an error will be raised.

Note

Do not implement this method. FME injects the implementation at runtime.

Parameters:

feature – The feature to output.
output_tag – The output tag to direct feature to. If multiple output tags exist but this argument is not specified, PYOUTPUT will be used as a fallback value. If the PythonFactory definition has a single output tag, this tag will be the default. Introduced in FME 2024.0.

total_features_passed_along() → int

Note

Do not implement this method. FME injects the implementation at runtime.

Returns:: A count of features that have been processed to date, in all groups.

factory_name: str: This is the FACTORY_NAME parameter of the PythonFactory that instantiated this class. Defaults to the name of this class.

Note

Do not modify this property. FME sets the value at runtime.