Overview
This section documents the format for the transformer files (*.fmx). These
files live in your install/transformers directory, and are read by Workbench
on each startup.
Versioning:
- Transformers are versioned. We add functionality to functions/factories
all the time, but we want to make sure old workspaces act exactly as they
always have, so each transformer definition has a VERSION clause.
The following types of changes can be made without making a new version:
- Adding/renaming ports
- Adding preserved or added attributes on output ports
- Adding a new parameter when the default value doesn't change the semantics or behavior of the transformer
- Removing a parameter (if it no longer makes sense)
- Changing the default value of a parameter, or a parameter prompt
- Changing the type of a parameter (as long as ALL existing values are compatible with the new type)
- Adding values to a choice type parameter (eg. CHOICE, ACTIVECHOICE, LISTBOX, etc)
- Changing the transformer execution template, if the change does not change the default operation of the transformer
- Changing the transformer's category or aliases
- Renaming the transformer
The following types of changes require making a new version:
- Removing ports
- Removing preserved or added attributes from output ports (debatable - use judgment)
- Changing the behavior of the transformer for a given value of a parameter
- Changing the type of a parameter from not encoded to encoded (ie. adding _ENCODED). See PR76931)
- Changing/removing values in a choice type parameter (eg. CHOICE, ACTIVECHOICE, LISTBOX, etc)
- Changing the transformer execution template, if the change does change the default operation of the transformer
When creating a new version and changing a parameter, you should consider if
the parameter you change is backwards compatible with older presets (see FMEENGINE-48843)
In the event that you do not want an older preset to be 'pulled' into this version, you
need to rename the parameter. Note this is separate from 'upgrade' transformer infrastructure.
That is, you can still map the old to the new name. The restoration of presets does NOT
go through any parameter upgrade infrastructure. It simply replaces the old value with the new
one for a given parameter.
Refreshing transformer in Workbench
IMPORTANT: In order to see changes in your transformer in Workbench, you
have to:
- delete the transformer from the workspace
- refresh the transformer list (there is a refresh button at the bottom of
the Transformer Gallery or the Tools->"Reload Transformers" menu option)
- undo delete (Ctrl+Z) of transformer.
Structure
TRANSFORMER_NAME: <name>
VERSION: <version>
[ALIASES: <alias1> <alias2> ...]
[KEYWORDS: <keyword1> <keyword2> ...]
CATEGORY: <category1>[,<category2>]+ -> see comment below for category list & descriptions.
INPUT_TAGS: <inputTags>
OUTPUT_TAGS: <outputTags>
DYNAMIC_INPUT_TAGS: [<tagName> [<tagName>]+]
[MULTIPROCESS: <groupby param>]
[TAG_DESCRIPTION <tagName>: <description>]
PRESERVES_ATTRIBUTES: <preservesAttributes>
[PRESERVES_GEOMETRY: <preservesGeometry>]
FEATURE_HOLDING: [NONE] [GROUPED] [ALL]
ATTRIBUTES_ADDED: <addedAttributes>
[GEOMETRY_ADDED: [<tagName> [<tagName>]+]]
LIST_ACTION: <listAction>
[VISIBLE: <yes|no|if_licensed>]
[REPLACED_BY: <transformerName>[:<version>]]
[PARAMETER_MAPPING: [<newParameter>;$(<oldParameter>)]* [<newParameter>;<constantValue>]*]
[TAG_MAPPING: [IN;<oldInputTag>;<newInputTag>]* [OUT;<oldOutputTag>;<newOutputTag>]*]
[NUKERS_REQUIRED: <yes|no>]
[DECORATION_NAME: <decoration image base name>]
[DECORATION_LEFT: <yes|no>]
[CANVAS_PROPERTIES: BANNER_NAME;<banner image base name> [prop;value]*]]
[ARCGIS_COMPATIBILITY: <ARCGIS_AUTO|ARCMAP|ARCPRO>]
[AUTO_PUBLISH: <parmName>[ <parmName>]+]
[OUTPUT_PORT_CONFIG: <JSON outputPorts spec>]
(This block is repeated for each parameter:)
PARAMETER_NAME: <parmName>
PARAMETER_TYPE: [<flags>] <parmType> <paramConfig>
PARAMETER_PROMPT: <parmPrompt>
PARAMETER_DEFAULT: <parmDefault>
[PARAMETER_LOOKUP: <lookupParamName> <choice1>%<choice2>%choice3>]
# Note: rather than parameter_lookup, consider using CHOICE_LOOKUP variants of gui types. They are more robust when terminology changes.
# because they store the looked up rather than the string value of the choice
CHANGE_LOG_START
<changes>
CHANGE_LOG_END
TEMPLATE_START
<template>
TEMPLATE_END
<name> and <category> can be any text string.
<version> can be any integer and should start at 1. Only the highest-
numbered version is made available for adding into a workspace, but old
versions should be left in the file for backwards-compatibility
with older workspaces. See the 'Transformer Versions' section below for
more information on why you would want to make a new version of a
transformer.
ALIASES specifies a list of alternate names for this transformer. They're
separated by spaces. This is useful if a transformer is renamed so customers can continue to
find the transformer using the old name.
KEYWORDS specifies a list of keywords related to this transformer. They're separated by spaces.
This is useful for searching for transformers in various contexts (e.g. Quick Add) where we can make
search terms not explicitly linked to a transformers name or alias still match to the given
transformer.
VISIBLE [<yes|no|if_licensed>
This optional directive can be used to mark the transformer as deprecated
by setting its value to "no". Its main purpose although is to control if a
transformer will be visible in GUI for adding to canvas using Quick Add or
Transformer Gallery.
yes [Default] : This is the default, meaning transformer is visible in GUI
and can be selected from Quick Add or Transformer Gallery.
no : Transformer is not visible in GUI from Quick Add or
Transformer Gallery and cannot be added to the canvas.
Existing workspaces using that transformer will still
work as expected.
if_licensed : Transformer is only visible if it is licensed to use on
the platform
REPLACED_BY: <transformerName>[:<version>]
This optional directive is used when a transformer is deprecated by setting
VISIBLE to NO. Its value is the name of the replacing transformer along
with its (replacing transformer) current version.
PARAMETER_MAPPING: [<newParameter>;<new value>]*[;<valueMap>]
This optional directive can be used when parameter is renamed within the
same transformer or when a transformer is deprecated. When transformer is
deprecated this can help map over the old parameters to parameters in the
new transformer. This mapping would be specified in the version previous
to where the changes are made. E.g. If parameter is renamed in version 3
and a new version 4 of transformer is created. Then PARAMETER_MAPPING
directive will go in version 3 of the transformer indicating how to
upgrade from version 3 to 4.
<new value> can be:
- parameter name from the old version E.g. NEW_PARM;$(OLD_PARM)
- FMEParsableText encoded literal string E.g. NEW_PARM;Replace<space>with<space>Point
- combination of both E.g. NEW_PARM;[<literal-string>]$(OLD_PARM)[<literal-string>]
<valueMap> syntax is [<oldValue>,<newValue>]*
Is for CHOICE* types where old choice value could be mapped to
new choice value. E.g.
MODE;$(ALGORITHM_NAME);Five<space>Color,Map<space>Coloring:<space>Five<space>Color,Simple,Map<space>Coloring:<space>N<space>Color<space>(Fast)
<oldValue> could also be one of these when we need to map the output value
based on if the old value was empty or not.
__fme_upgrade_empty_value__
__fme_upgrade_non_empty_value__
__fme_upgrade_unused_value__
E.g.
POLY_GENERATE_LIST_GROUP;$(POLY_LIST_NAME);__fme_upgrade_empty_value__,No,__fme_upgrade_non_empty_value__,Yes
The above line will amount to checking/unchecking an active group (POLY_GENERATE_LIST_GROUP)
based on if the list name (POLY_LIST_NAME) parameter was set or not
TAG_MAPPING: [IN;<oldInputTag>;<newInputTag>]* [OUT;<oldOutputTag>;<newOutputTag>]*]
If input or output tags get renamed from one version to other, this directive
specifies how the old tags be mapped to new ones when upgrading transformers.
Just like PARAMETER_MAPPING, this directive is specified in the version
previous to where the change is made.
AUTO_PUBLISH: <parmName>[ <parmName>]+
Any parameters specified by this directive will be published automatically
when the transformer is added to a workspace. Does nothing if a given parameter
does not support publishing.
Inputs and Outputs
<inputTags> is a list of all input tags for the transformer.
<outputTags> is a list of all output tags for the transformer.
Each have a syntax of form:
[<alias>:]<actualname>[:<condition>] [[<alias>:]<name>[:<condition>]]*
<alias> is the aliased name that will show in workbench and <actualname>
is the tag used inside of FME. Either the alias or actualname must be specified.
For input tags, the <actualname> may be left empty if you don't want to pipe an input
directly into a factory, e.g. NeighborPairFinder.fmx uses "CANDIDATE:" as an input tag.
If a given input has a blank input type, this can be denoted by <BLANK>. An input tag of
<NO_INPUT> means the transformer takes no inputs.
For output tags, <BLANK> means there are NO outputs, as opposed to an output without a name.
For this, aliasing is for display purposes only.
<condition> indicates an optional condition must be true for the port to be shown the user
in the gui. As a result of not showing in the GUI, the port will also not be written to the mapping file (unless it has a nuker).
Condition syntax is:
PARM,[<boolOp>],<parm[.#]>,<FMEParsableTextvalue>,[[<boolOp>],<parm[.#]>,<FMEParsableTextvalue>]* - the port will only be included
if the value of one of the <param> matches the cooresponding <FMEParsableTextvalue>.
A <FMEParsableTextvalue> of FME_RUNTIME_VALUE indicates the port should only be included
when the value of the parameter is unknown until runtime (ie. is set to an expression or published
parameter). This is useful for <Rejected> ports where the rejection can only occur if the
user inputs an expression that results in invalid data (eg. cloner, decelerator).
For LISTBOX types, the check for equality will check against each item that has been selected
by the user.
<boolOp> values are NOT, OR, AND, OR_NOT, AND_NOT. Allows the condition to be negated
and joined. If nothing is specified for a given param-value pair, then OR is assumed
Examples with conditions:
from cloner <REJECTED>:PARM,COPIES,FME_RUNTIME_VALUE
will only include the <Rejected> port if COPIES parameter is an expression
from Intersector COLLAPSED:PARM,NOT,CLEANING_TOLERANCE,None
will only include the COLLAPSED port if CLEANING_TOLERANCE parameter is not 'None'
OUTPUT_PORT_CONFIG: <JSON outputPorts spec>
This provides output port information in JSON list, as described for
the "outputPorts" key in [transformer.md]. Any newly defined ports are
added as defined to the list of OUTPUT_TAGs. Information for known ports
is merged with the existing port information, such as from PRESERVES_ATTRIBUTES
and ATTRIBUTES_ADDED. This tag is specifically useful for defining output
ports' preservesGeometry and geometryAdded.
DYNAMIC_INPUT_TAGS
These are special input tags, that will never end up in the mapping file but
are there for the benefit of the view. These tag names can have spaces,
hence are FMEParsableText encoded. There can be no link made to these tags, rather
the process of making a link to this tag will invoke a custom behaviour
that is to be determined by the transformer. Mostly this is used to create
new ports whose definition is based on the schema from the incoming link.
(See InlineQuerier or MaptextLabeller transformers)
MULTIPROCESS
<groupby param> is the name of the group by parameter for use when doing
the multiprocess. In general, this flag should only be added if the
transformer is CPU intensive. For multiprocess to make sense to use, the
overhead of launching a new process and transferring the features back
and forth should be insignificant compared to the processing time for
each group. See dissolver for example. When using this option, inform
the tech pubs team to add the multiprocess documentation to your
transformer.
Schema Transformation
PRESERVES_ATTRIBUTES
<preservesAttributes> specifies whether the attributes on the input side
of the transformer will be propagated through to the outputs. This can
be yes (meaning all attributes are propagated to all outputs), no
(meaning none are propagated), or a list of the form:
OUTPUT1:INPUT1[:<condition>] OUTPUT2:INPUT2[:<condition>] ...
Which specifies that the output tag OUTPUT1 takes attributes from
INPUT1, and so on. <BLANK> inputs can be specified by using OUTPUT1:. *
can be used as a wildcard in the specification of input and/or output
(but not as a substring wildcard).
<condition> indicates some optional condition for the schema transformation.
This can be:
- PARM:<parm>[:<parm>]* - the attributes will only be preserved if logical
AND operations on >=1 bool parameters, <parm>, equals 'yes' or 'true'
- LIST_PARM:<listParm> - the attributes will be preserved if the list
parameter <listParm> on the transformer is not blank.
- PREFIX:<prefixParm> - all attributes will be prefixed with the value from
parameter <prefixParm>
- PREFIX:<FMEParsableTextPrefixString> - all attributes will be prefixed with the decoded value of <FMEParsableTextPrefixString>
- PARM_WITH_PREFIX:<parm>[:<parm>]*:<prefixParm> - the attributes will only be
preserved if logical AND operations on >=1 bool parameters, <parm>, equals 'yes'
or 'true', and all attributes will be prefixed with the value from
parameter <prefixParm>
- PARM_WITH_PREFIX:<parm>[:<parm>]*:<FMEParsableTextPrefixString> - the attributes will only
be preserved if logical AND operations on >=1 bool parameters, <parm>, equals 'yes'
or 'true', and all attributes will be prefixed with the decoded value of <FMEParsableTextPrefixString>
- GROUP_BY:<groupByParm> - the attributes included in the value of the
parameter <groupByParm> will be preserved.
- PARM_WITH_GROUP_BY:<parm>:<groupByParm> - the attributes will only be
preserved if the parameter <parm> is set to 'yes' or 'true'. In
addition, all attributes included in the value of the parameter
<groupByParm> will be preserved. e.g. Dissolver.fmx
- LIST_PARM_WITH_GROUP_BY:<listParm>:<groupByParm> - the attributes
will only be preserved if the parameter <listParm> is not blank.
In addition, all attributes included in the value of the parameter
<groupByParm> will be preserved.
Note that if you have aliased the input, the INPUT tag should use the
alias tag not the real tag.
ATTRIBUTES_ADDED
<addedAttributes> specifies the attributes that will be added to output
tags. The syntax is:
Per output port: [<portname>]:<attrdef>[:PARM | LIST_PARM | PARM_NULL]:[<condition>]
All output ports: <attrdef>[ <attrdef>]
<portname> : Name of the output port
<attrdef> : Attribute definition specified as comma separated list
<attr1>[;<attr_type>][,<attr2>,<macro>,<attr3>,...[;<attr_type>]]*
<attr_type>: Can be any one of the type defined for FFS format. Left hand side
of the ATTR_TYPE_MAP in ffs.fmf.
List of available type names:
- char(width)
- varchar(width)
- buffer
- xml
- json
- binary(width)
- varbinary(width)
- binarybuffer
- datetime
- time
- date
- real64
- real32
- int32
- uint32
- int64
- uint64
- logical
- int16
- uint16
- int8
- uint8
- number(width,decimal) : Avoid using this type, but if you have to then
make sure it is encoded like number(20<comma>10)
As a shortcut, type name can be specified for the first attribute in the list and
it will apply to all the following attributes in the list until a new type is
specified for the attribute in that list. E.g.
fme_primary_axis;real64 fme_secondary_axis fme_rotation;real32 fme_start_angle fme_sweep_angle
fme_primary_axis and fme_secondary_axis will be of type real64 and all the other types will be real32
<macro> can be the value of a parameter. A macro can also be combined
with a constant string (eg. $(FOO)Hi). If a list parameter is specified,
each entry in the list is evaluated separately and added to the list of
output attributes.
<condition> indicates some optional condition to satisfy in order for
attributes to be added. This can be:
- PARM_COND:[<boolOp>],<parm[.#]>,<FMEParsableTextvalue>,[[<boolOp>],<parm[.#]>,<FMEParsableTextvalue>]* -
the attributes will only be added if the condition evaluates to true.
If nothing is specified for a given param-value pair, then OR is assumed.
If the specified parameter cannot be found, that part of the condition will be false.
See Inputs and Outputs for more detail, as is the same evaluation syntax.
The following conditions are deprecated. Please use PARM_COND instead.
- PARM:<parm> (deprecated) - the attributes will only be added if the parameter
<parm> is set to 'yes' or 'true'.
- PARM:<parm>:<val1>,<val2> (deprecated) - the attributes will only be added if
the parameter <parm> is set to <val1> or <val2>
- LIST_PARM:<listParm> (deprecated) - the attributes will be added if the list
parameter <listParm> on the transformer is not blank.
- PARM_NULL:<parm>:<yes | no > (deprecated) - if 'yes', the attributes will be added
only if the parameter is null. If 'no', the attributes will be added
only the parameter is not null.
If a parameter refers to a global parameter, the attributes will always
be added. This is because we do not know the value of the parameter
until runtime. Better to show too much than too little.
Note that when using conditions, if the parameter specified in the
condition refers to a global parameter, the attributes will always be
added. this is because we do not know the value of the attribute until
runtime, thus must show the user ALL possibilities at the workspace
building phase.
For backward compatibility, a list of values separated by a ' ' is also
valid: <val1> <val2>
Examples of ways to specify the added attributes:
<attribute> [...]
e.g. ATTR1 ATTR2
The example above will added ATTR1 and ATTR2 to... all outputs?.
<output tag>:<attribute> [<output tag>:<attribute>]
e.g. OUTPUT1:ATTR1 OUTPUT1:ATTR2 OUTPUT2:ATTR2 ...
The example above will add attribute ATTR1 and ATTR2 on OUTPUT1 tag and
ATTR2 on OUTPUT2 tag
<output tag>:{}.<attribute> [<output tag>:{}.<attribute>]
e.g. OUTPUT1:{}.ATTR1 OUTPUT1:{}.ATTR2 OUTPUT2:ATTR2 ...
The example above will add ATTR1 and ATTR2 to the list name if there was
one specified. So if the list name was "myList" then OUTPUT1 will have
two attributes myList{}.ATTR1 and myList{}.ATTR2. See its use in
NeighborFinder.fmx
Example From AttributeExposer:
ATTRIBUTES_ADDED: OUTPUT:$(NEW_ATTRIBUTES)
....
PARAMETER_NAME: NEW_ATTRIBUTES
PARAMETER_TYPE: OPTIONAL USER_DEFINED_LIST
In this case, the FMEParsableText space delimited values
specified in NEW_ATTRIBUTES will be added to the output port OUTPUT.
Example From PointCloudCoercer
COERCED:$(POINT_COMPONENTS_ATTR):PARM:POINT_COMPONENTS_PRESERVE_TYPE:ATTRIBUTES
The example above will add the values in $(POINT_COMPONENTS_ATTR) as
attributes only if the macro POINT_COMPONENTS_PRESERVE_TYPE has value
ATTRIBUTES.
Example from StreamOrderCalculator
ATTRIBUTES_ADDED: NETWORK:$(STRAHLER_ATTR):PARM:STREAM_ORDER_TYPE:Strahler NETWORK:$(STRAHLER_ATTR),$(HORTON_ATTR):PARM:STREAM_ORDER_TYPE:Horton
This says: On the NETWORK output port, display the attribute
$(STRAHLER_ATTR) if "Strahler" is specified as the STREAM_ORDER_TYPE.
Also, on the NETWORK output port, display the attributes
$(STRAHLER_ATTR) and $(HORTON_ATTR) if "Horton" is specified as the
STREAM_ORDER_TYPE.
PRESERVES_GEOMETRY
<preservesGeometry> specifies whether the geometry on the input side
of the transformer will be propagated through to the outputs.
This should be a list of the form:
OUTPUT1:INPUT1[:Yes|No] [OUTPUT2:INPUT2[:Yes|No]] ...
Which specifies that the output tag OUTPUT1 does or doesn't preserve geometry
from INPUT1, and so on. * can be used as a wildcard in the specification of input ports.
If not specified, geometry from all input ports is assumed to be
preserved on all output ports.
GEOMETRY_ADDED
specifies a list of output tags that should have an unnamed geometry added to their output schema.
LIST_ACTION
<listAction> can take any of the following forms and may be repeated
multiple times(e.g as in NeighborFinder.fmx), one for each list
parameter:
- BUILD_ALL <list_parm> [TRANS_LIST]
- EXPLODE <list_parm> [TRANS_LIST]
- REMOVE <list_parm> - KEEP <list_parm>
REMOVE removes all list attributes specified in <list_parm>
KEEP removes all list attributes except those specified in <list_parm>
BUILD_ALL makes a list (of name specified in parameter list_parm) adding incoming
attributes to the list based on the [TRANS_LIST] specification
EXPLODE explodes the list in list_parm into its elements.
TRANS_LIST specifies the transfer of list attributes from input to
output and has the format:
OUTPUT1:INPUT1[INCLUDE_ATTRS:parmName] OUTPUT2:INPUT2[INCLUDE_ATTRS:parmName] ...
If TRANS_LIST is empty, every output takes from every input. If INCLUDE_ATTRS is specified,
only the attributes indicated in parmName are transferred.
NUKERS_REQUIRED
When set to yes, this forces the OUTPUT clause to always be written out,
and clean up any features that will come out. Only required for certain
factories that require an OUTPUT clause, or for any transformer that
does its work on the OUTPUT clause of a TeeFactory.
FEATURE_HOLDING [NONE|GROUPED|ALL]
- NONE: The transformer has a mode where features will be fully
processed as they enter.
- GROUPED: This tag means the transformer has a Group By parameter,
and is expected to hold features if that parameter is populated.
Unclear if it is supposed to imply that features are expected to
be ordered and will be released as soon as a feature in a different
group is seen.
- ALL: The transformer has a mode where input features will be
stored until the transformer is notified that the final feature
has been received.
Explanation for the Clipper transformer:
The Clipper for example by default for both vector and raster input
(which dictates use of two different factories) is ALL because the
Clipper Type is Multiple Clippers and it must wait until all input
clipper features are known. Setting the Group By simply changes this
to GROUPED as expected. However, setting the Clipper Type to Single
Clipper or Clippers First will allow the clipping to occur
immediately on subsequent input clippee features, thus rendering the
feature holding mode NONE. GROUPED is ambiguous in that a factory
with a Group By parameter can either hold all feature or hold no
features if the Group By is not set. So GROUPED
should probably always appear with at least one of NONE or ALL.
Parameters
The PARAMETER_* block defines input parameters for the transformer. This
block may be repeated multiple times, one for each parameter. <parmName>
can be any valid name. <parmType> is the type of the parameter, more on
that below. <parmPrompt> is a prompt shown to the user when this
parameter is asked for. <parmDefault> is the default value for the
parameter.
For a list of valid values for <parmType>, see the GUI Type
Documentation on http://docs.safe.com/fme/html/FME_GuiType/
The parameter type also has several optional flags:
OPTIONAL - specifies a blank value is valid for this parameter.
NO_CONDTIONAL - specifies that the conditional option will not be
available on the parameters drop down menu
DISABLED_ONLY
ALWAYS_ENABLED
FORCE_DEFS
IGNORE - Used when the parameter is only required by the <xformer>cfg.cpp class and not by the GUI.
The parameter will exist
but not be shown in the navigator, tooltips
or included in the qfmequery dialog instance.
NO_SORT
NULLABLE - parameter accepts FME_NULL_VALUE and parameter menu should have 'set to null' option
NO_OP - parameter menu should have the 'no op' option
ABORTABLE - parameter menu should have the abort option
SHOW_ZIP_BUTTON - parameter will allow zip files
LITERAL - parameter only accepts literal values
IGNORE_VALUE
NO_EDIT - parameter will be in the query but will not be visible to the user.
It will be shown in the nav tree and in the tooltip UNLESS
its type or description precludes doing so. For example,
GROUP never appears in nav tree and a TEXT field that has
no prompt specified will never appear in the tooltip or nav tree.
If you are using an OR_ATTR type or de-align with attribute names, be
sure that your transformer can handle attribute names containing spaces.
Parameter values are referenced by enclosing the parameter name in
$(). E.g. For a parameter with a name of A_PARAMETER, it is referenced
as $(A_PARAMETER). In order to preserve spaces in parameter values,
the parameter name references should be enclosed in "".
E.g. The parameter reference $(A_PARAMETER) should be "$(A_PARAMETER)".
The PARAMETER_LOOKUP directive should be used sparingly and is only used if the
parameter is a CHOICE or LISTBOX type. Instead, consider using
LOOKUP_CHOICE or LOOKUP_LISTBOX variants of gui types. They store the looked up rather than
the string value of the choice so are more robust as terminology changes. However, this directive
when a single choice maps to >1 value. It allows you to use full-length
English versions of each option in the CHOICE line, providing a nice
alternative to obscure factory directives. Macro name used in
PARAMETER_LOOKUP directive cannot be the same as the name used for
PARAMETER_NAME directive. Once PARAMETER_LOOKUP is defined, then the
template section of the transformer should use the macro name defined in
PARAMETER_LOOKUP directive.
Here's an example:
PARAMETER_NAME: OBSCURE_PARAM
PARAMETER_TYPE: CHOICE "A description of an obscure option"%"Another description"
PARAMETER_LOOKUP: OBSCURE_PARAM_LOOKUP
OBSCURE_DIRECTIVE%ANOTHER_OBSCURE_DIRECTIVE . . . TEMPLATE_START
FACTORY_DEF * TestFactory FACTORY_NAME $(XFORMER_NAME)_filter
$(INPUT_LINES) TEST @TCL("AS_remove_empty_attributes
$(OBSCURE_PARAM_LOOKUP)") == "0" OUTPUT PASSED FEATURE_TYPE
$(OUTPUT_OUTPUT_FTYPE)
Changes
Changes to the transformer are listed between the CHANGE_LOG_START and
CHANGE_LOG_END section. Log message can include url which will be converted
to clickable hyperlink when displayed in help browser.
Decoration
The Transformer Decoration is an optional extra image added to the
left or right side of the Transformer node in the Workbench Canvas.
DECORATION_NAME is the base name of the decoration image found in the
active Workbench theme. Supported file type is .svg
DECORATION_LEFT is true if the decoration is to be displayed on the
left side of the Transformer node and false otherwise.
This has no effect if DECORATION_NAME is not set or the file is not found.
CANVAS_PROPERTIES: [<prop name>;<prop value>]* (Internal Only)
NOTE: This is strictly for Internal Use Only at this time.
Value of this tag is expected to be space delimited string with one or
more of <prop name>;<prop value>.
Following properties are currently supported:
BANNER_NAME;<image base name>
SHOW_IN_NAVBAR;[Yes|No]
EXCLUDE_ACTIONS; <List of actions to exclude from context menu>
OPEN_FOLDER;PATH,<macro>[,FORMAT,<macro>][,SUPPORTS_IF,<cond>]
INSPECT;FORMAT,<macro>,DATASET,<macro>,RUNTIME_MACROS,<macro>[,COORDSYS,<macro>,FEATURE_TYPES,<macro>][,SUPPORTS_IF,<cond>]
BANNER_NAME:
Using this tag we can configure transformer node's appearance and behaviour
on the canvas and navbar. The transformer banner is generally fixed to
use a standard image whose base name is set to 'banner_xformer'. In some
rare cases where we want to have custom banner image, it can be specified
using BANNER_NAME property. Currently if this tag is set then we assume
the following:
- Input and Output ports are hidden, that means the input and output
ports (triangles) will show up at the banner level
- Node is not editable
- Node will not have any visible text item
SHOW_IN_NAVBAR:
If its value is set to No (default is Yes if not specified) then the
transformer node won't show in the navbar except when performing
workspace search.
EXCLUDE_ACTIONS:
Value is a comma delimited list of action names to exclude from node's
context menu.
OPEN_FOLDER:
Requires PATH parameter and optionally a FORMAT can be specified. When
FORMAT is specified then the context menu will be disabled under certain
circumstances where PATH is not a valid file system path. Formats with
dataset type (formats.db) of DATABASE, ODBC and NONE will cause the context
menu option to be disabled
INSPECT:
Expected value for FORMAT is the format short name (formats.db). DATASET
is expected to be a valid file system path name. RUNTIME_MACROS is a
comma delimited SDF encoded string of parameters as name-value pairs
SUPPORTS_IF:
valid for INSPECT and OPEN_FOLDER - indicates a condition which
must be met for the inspect or open folder to be supported.
Current syntax supports form PARMNAME=VAL only which would mean
the open/inspect is only supported when PARMNAME is equal to VAL
CATEGORY: The valid categories are:
3D Operations for 3D formats and 3D geometry including appearances
Attributes Operations for attribute/list management
Values Operations that return a calculated value
Cartography and Reports Operations that prepare and style data for cartographic purposes
Coordinates Operations for coordinate reprojection and warping
Data Quality Operations for attribute/geometric data QA and validation
Filters and Joins Operations for dividing and merging data flows
Format Specifics Operations that are related to a specific format
Geometries Operations that create geometry or transform it to a different geometry type (usually independent of other features)
Integrations Operations that integrate FME with another tool, be it R or a web service
Point Clouds Operations relating specifically to Point Cloud features
Rasters Operations relating specifically to Raster features
Spatial Analysis Operations that return the result of a spatial analysis
Strings Operations that manipulate string contents, including dates
Web Operations specific to web-related issues such as web services, or web-based usage of a language like XML or JSON
Workflows Operations that control workflow, such as Job submitters
Samples Transformers in the plugin directory that are example so users can write their own transformers
TestSuite Tools (Safe Internal) Transformers in the testsuite directory used by Safe Developers to write tests
Template
Everything in-between TEMPLATE_START and TEMPLATE_END is what is
actually inserted into the mapping file. TEMPLATE_END is the end of the
transformer definition (and beginning of another). The transformer name
can be inserted using the tag $(XFORMER_NAME) and you'll need it as
almost everything is prepended with it for namespacing.
Input tags can be specified using $(INPUT_<type>_LINES) to insert tags
of a given <type>, or $(INPUT_LINES) to put all input lines at once.
Output tags can be specified using $(OUTPUT_<type>_FTYPE) to insert
output tags for the feature type <type>. Output functions can be
inserted by specifying $(OUTPUT_<type>_FUNCS)
Parameters can be substituted into the template by encasing it in $()
For example $(MYPARM), to substitute the parameter named MYPARM.
If you have optional parameters do not rely on WHOLE_LINE to do the removal. Instead,
use the {*} rather than * in the template start. This indicates that { } will be the indicator of
the keyword value rather than space. Otherwise, if the user publishes the
parameter and leaves it blank, the parser will fail because the keyword has no value.
FACTORY_DEF {*} MyFactory
....
KEYWORD { $(MYPARM) }
You can write all parameters and their values using the
$(FME_BRACED_PARM_VAL_LIST) keyword (formally FME_PARM_VAL_LIST).
This is handy if you have a factory whose keywords exactly match all the parameters
in your GUI. In this case, the data is written as:
parm1 { val1 }
parm2 { val2 }
....etc...
Similarly, you can get a list of all parameters using the FME_PARM_LIST
keyword. This will give a comma delimited list of all parameters in the
form:
parm1, parm2, etc
The template is basically a list of factories that work with each other
to process the features coming from the ports specified in INPUT_TAGS
and finally output features to the ports specified in OUTPUT_TAGS (there
is a default output tag of OUTPUT). Look at the header files for the
different factories to see how their specific syntax works. Some common
ones are TeeFactory (used to route output) and TestFactory(used to route
output based upon some test). Functions can be applied on features by
using the @Function syntax either upon input (on the INPUT FEATURE_TYPE
line) or upon output(on the OUTPUT FEATURE_TYPE line). For example, the
following line:
OUTPUT FEATURE_TYPE ___BOUNDING_LINE___
@Coordinate(REMOVE,4)
@Coordinate(REMOVE,3)
@Coordinate(REMOVE,1)
... will output ___BOUNDING_LINE___ features after applying the
Coordinate(REMOVE, ...) function on them 3 times.
To execute a function from tcl, you will need to use FME_Execute. For
example:
FME_Execute Geometry GET_CONVEXITY_CLASS
A more involved example that sets a global variable:
Tcl2 proc $(XFORMER_NAME)_setFeatType {} {
global FME_FeatureType;
set ft {$(XFORMER_NAME)_};
append ft [FME_Execute Geometry GET_CONVEXITY_CLASS];
set FME_FeatureType $ft;
}
Then we can use this function like this: @Tcl2($(XFORMER_NAME)_setFeatType)
To dynamically change the output feature type, you need to modify the
tcl global variable FME_FeatureType. You can easily do that using the
function @FeatureType. Here is an example from the ConvexityFilter
transformer. This transformer routes features to different ports by
depending on what the result of applying @Geometry(GET_CONVEXITY_CLASS)
is:
FACTORY_DEF * TeeFactory
FACTORY_NAME $(XFORMER_NAME)
$(INPUT_LINES)
OUTPUT FEATURE_TYPE * @FeatureType(@Concatenate($(XFORMER_NAME)_,@Geometry(GET_CONVEXITY_CLASS)))
Good luck!
Upgrading Transformers
Transformers can now be upgraded to their latest version. Here are few things
to keep in mind when transformer is updated in any form:
- Briefly describe the changes done to the transformer in CHANGE_LOG section
- If any of the input or output tags were renamed/removed then specify
TAG_MAPPING clause. This is required so that the links can be maintained
after upgrading the transformer
- If parameters were renamed/removed then specify PARAMETER_MAPPING clause
if parameters values can be mapped over to new parameter. If the change is
complex then you may be required to implement upgradeTransformer()
to correctly map over the parameters from old to new version
Transformer Template Keywords
The following are built in substitutions for mapping templates:
XFORMER_NAME_ENCODED & XFORMER_NAME - the transformer name input by the user
XFORMER_VERSION - the transformer version
These are useful to avoid having to keep your factory template in sync with your parameter listings:
FME_BRACED_PARM_VAL_LIST - parameters and their values written in 'key { value }' form assuming {*} syntax is used on the transformer
FME_PARM_VAL_LIST - parameters and their values written in 'key value'
FME_PARM_LIST - list of parameter names separated by ,. No values included
Renaming Transformers
Follow these steps to rename an existing "OldName.fmx" transformer to
"NewName.fmx"
- Rename "OldName.fmx" to "NewName.fmx". This will ensure we don't lose history
for the "OldName.fmx"
- Now in "NewName.fmx" add your new version if necessary, leaving the old versions
around for backwards compatibility
- For all of the versions in NewName.fmx, add the line "ALIASES: OldName"
- Update the list of transformers in "bundles/esridesktop/esri-interop-transformers.txt"
by removing OldName.fmx and adding NewName.fmx
- Commit changes and you are done
Transformer Gotchas
The following is a list of gotchas and best practices to consider when
writing transformer templates:
- When in doubt ensure your parameter reference is in quotes "$(myParam)".
String values that contains spaces and are not quoted will have the spaces
removed.
- make all parameters OR_ATTR types whenever reasonably possible.
- don't use WHOLE_LINE, it is is deceiving. Always handle empty value in your
function/factory. This is because if the parameter is published the value
is only known at runtime thus whole_line is not done and the line is written to the mapping file.
At runtime, the line is not removed but the parameter set to the value of "".
See PR8182
- only use the transformer name if you really need a transformer name. If it
is just to have a unique name and the user does not see it, use the
FME_UUID macro instead (e.g. for Tcl function names)
- If at all possible do NOT look for 'gui-isms' in fmx or C++ code.
- Do not check for <Unused>! Instead check the value of the MACRO (active*) that made
the parameter unused. For example, if the user publishes the active choice
parameter, then <Unused> will not be written out because we do not know at
mapping file generation time whether or not the parameter actually is unused.
- @Value() - instead of using @Value, pass the attribute value into the
tcl function as a parameter (see stylers).
- be aware that if the ACTIVECHOICE or ACTIVECHECK parameter is linked to
a user parameter, the GUI can NOT know the value of the choice until
runtime. As a result, it will write out ALL the dependent parameters to
the mapping file. this is a good example why you should not check for <Unused> (see above)
- if you change the choice values seen by the user, you almost always have
to update the version of the transformer
- when writing TCL that access macros directly, ensure that special characters
are handled correctly since the FME parser can do strange things. Test with
a string like: "PARAN (and spaces)".
- when setting the default values for your transformer with ACTIVECHOICE,
set them consistently to what the GUI would normally save. This
probably means the default value for some dependent parameters will be
<Unused>.
- [common and counter-intuitive] Up the transformer version when changing the
list of displayed values for CHOICE or LISTBOX (or similar) types's
configuration on the PARAMETER_TYPE line. (This is required because the
displayed value is used to store the parameter value in the workspace.)
Deprecated Sections:
WHOLE_LINE - don't use this flag. See PR8182 for good examples why. the gotchas section
also covers why.
TOOLTIP_START
<tooltip> - you only need this if you have a non-standard tooltip. Workbench will generate a nice looking tooltip for you.
TOOLTIP_END