Merges the attributes and/or geometry of one set of features onto another set of features, based on matching list attribute values with key attribute values and expressions.
Typical Uses
- Combining attributes and/or geometry from two different streams of features, based on a common key attribute value or expression, where the key of the receiving feature (Requestor) is stored as a list attribute.
- Referencing a lookup table to a list attribute
How does it work?
The ListBasedFeatureMerger joins two feature streams based on a common key attribute or expression, where one of the key values is a list attribute. Every value in the list (on every feature) is considered for matching, as the transformer tests each list element in turn for a match.
It receives two the streams of features via its input ports.
Requestor: Requestors are the features that will receive new attributes and/or geometry. The Requestor must have a list attribute that will be used as a key to join the Supplier(s).
Supplier: Suppliers provide attributes and/or geometry to be merged onto the Requestors.
Matches between Requestor and Supplier are identified according to the Join Attributes configuration in the parameters dialog. A Requestor feature matches a Supplier feature when a key in the specified Requestor List Attribute (for example, list{}.key) matches the specified Supplier Attribute parameter (for example, supplierKey).
When a Requestor finds a matching Supplier, the attributes and/or geometry from the Supplier are merged with the Requestor.
The Incomplete Requestors parameter specifies whether or not to merge Suppliers onto partially matching Requestors when there are keys in the Requestor List Attribute that have no matching Suppliers. A Requestor feature is considered fully matched if every key in the Requestor List Attribute matches one or more Supplier features.
- If the Requestor already has an attribute that the Supplier also has, the Requestor's original value for that attribute can be preserved or overridden.
- A single Supplier may be used by many Requestors.
- Many Suppliers can be merged onto a single Requestor.
- When attribute names conflict, you can choose whether Requestor or Supplier attribute values are maintained by using the Conflict Resolution parameter, and whether null values follow the same behavior by using the Ignore Nulls parameter.
Example: Using a list attribute to merge features
In this example, we have a set of polygons representing neighborhoods. They have a list attribute - FoodVendors{} - that contains unique IDs (KEY) for the food trucks located in the area. The task is to merge additional attributes for each vendor from a spreadsheet.
In this portion of a workspace, the area (with list) features are routed into the Requestor port of a ListBasedFeatureMerger. The spreadsheet is attached to the Supplier port.
In the parameters dialog, the Join Attributes are set to match the FoodVendors{}.KEY list attribute (Requestor) with the KEY attribute from the spreadsheet (Supplier).
In Merge Parameters, we choose to merge Attributes Only, and (importantly) to Process Duplicate Suppliers. This will ensure that all food vendors in the list are matched, not just the first one encountered.
And finally, we enable Generate List, to create new list attributes on the output features. It will contain all matching Business names and descriptions, via Selected Attributes. Note the list name entered - FoodVendors - is the same as the existing list name. This will result in the new attributes being added to (and expanding) the current list. If we entered a different name here, a new list attribute would be created.
Inspecting the features output from the Merged port, we see that BUSINESS_NAME and DESCRIPTION have been added to the FoodVendors{} list and can now be used elsewhere in the workspace.
Usage Notes
- The ListBasedFeatureMerger is a very specific instance of the FeatureMerger dealing with the case where the Requestor attribute is a list. If the requestor attribute is NOT a list, use the FeatureMerger or another joining method.
Choosing a Feature Joining Method
Many transformers can perform data joining based on matching attributes, expressions and/or geometry. When choosing one for a specific joining task, considerations include the complexity of the join, data format, indexing, conflict handling, and desired results. Some transformers use SQL syntax, and some access external databases directly. They may or may not support list attribute reading and creation.
Generally, choosing the one that is most specific to the task you need to accomplish will provide the optimal performance results. If there is more than one way to do it (which is frequently the case), time spent on performance testing alternate methods may be worthwhile. Performance may vary greatly depending on the existence of key indexes when reading external tables (as opposed to features already in the workspace).
Joining Transformers Comparison Matrix
|
Transformer |
Match By |
Uses SQL Statements |
Can Create List |
Input Type |
Notable |
Description |
|---|---|---|---|---|---|---|
| FeatureJoiner | Attributes | No | No | Features |
|
Joins features by combining the attributes and/or geometry of features based on common key attribute values. Performs the equivalent of Inner, Left, and Full SQL joins. |
| FeatureMerger | Attributes | No | Yes | Features |
|
Merges the attributes and/or geometry of one set of features onto another set of features, based on matching key attribute values and expressions. |
| ListBasedFeatureMerger | List Attribute to Single Attribute | No | Yes | Features |
|
Merges the attributes and/or geometry of one set of features onto another set of features, based on matching list attribute values with key attribute values and expressions. |
| InlineQuerier | SQL query | Yes | No | Features |
|
Creates a set of SQLite database tables from incoming features, executes SQL queries against them, and outputs the results as features. |
| SQLCreator | SQL query | Yes | No | External DB |
|
Generates FME features from the results of a SQL query executed once against a database. One FME feature is created for each row of the results of the SQL query. |
| SQLExecutor | SQL query | Yes | No | External DB |
|
Executes SQL queries against a database. One query is issued to the database for each initiating feature that enters the transformer. Both the initiating features and the results of the query may be output as features. |
| DatabaseJoiner | Attributes | No | Yes | External DB and Features |
|
Joins attributes from an external table to features already in a workspace, based on a common key or keys. SQL knowledge not required. Non-blocking transformer. |
| Matcher | Geometry and/or Attributes | No | Yes | Features |
|
Detects features that are matches of each other. Features are declared to match when they have matching geometry, matching attribute values, or both. A list of attributes which must differ between the features may also be specified. If matching on attributes only (not geometry), using the FeatureMerger or another method will give better performance. |
Configuration
Input Ports
Requestor
Features that have a list attribute to join on, and will receive new attributes and/or geometry from the features connected to theSupplier port.
Supplier
The source of new attributes and/or geometry for features that enter through the Requestor port.
Output Ports
Merged
Requestors that have matching Suppliers for all of the keys in the specified Requestor List AttributeOnly fully matched Requestors are output onto this port.
UnmergedRequestor
This port used to be labelled as Incomplete and has retained the semantics of that label. So, this port label is a shorthand for 'NotFullyMerged'. Partially matched Requestors are output onto this port, and all matching Suppliers are merged onto the output features if Incomplete Requestors is set to Merge Supplier Information. If a key in the Requestor List Attribute is null, that key will not match any Suppliers, and as a result, the Requestor feature will be output onto this port. For example, if the Requestor keys are:
list{0}.key = 1
list{1}.key = <null>
list{2}.key = 3
and furthermore there are Suppliers with 'supplierKey' values of 1, <null>, and 3, and Incomplete Requestors is set to Merge Supplier Information, then the Requestor will be output onto the UnmergedRequestor port, with Suppliers merged onto it whose 'supplierKey' is set to 1 or 3.
UsedSupplier
Suppliers that are merged onto at least one Requestor.
UnusedSupplier
Suppliers that do not match any Requestors. If a Supplier Attribute key value is null or missing, that feature is output onto this port.
<Rejected>
If Process Duplicate Suppliers is No, then duplicate suppliers will be rejected.
Rejected Feature Handling: can be set to either terminate the translation or continue running when it encounters a rejected feature. This setting is available both as a default FME option and as a workspace parameter.
Parameters
Transformer
| Group By |
The input features may be partitioned by the Group By parameter. If you choose any Group By attributes, then references between features will only be resolved if they share a common value for the selected attributes. If you do not choose any Group By attributes, all features are processed together. If you have more than one Reader, a typical use is to group by reader_id to ensure that references are resolved within the correct set of features. |
| Group By Mode |
Process At End (Blocking): This is the default behavior. Processing will only occur in this transformer once all input is present. Process When Group Changes (Advanced): This transformer will process input groups in order. Changes of the value of the Group By parameter on the input stream will trigger processing on the currently accumulating group. This may improve overall speed (particularly with multiple, equally-sized groups), but could cause undesired behavior if input groups are not truly ordered. Considerations for Using Group By
There are two typical reasons for using Process When Group Changes (Advanced) . The first is incoming data that is intended to be processed in groups (and is already so ordered). In this case, the structure dictates Group By usage - not performance considerations. The second possible reason is potential performance gains. Performance gains are most likely when the data is already sorted (or read using a SQL ORDER BY statement) since less work is required of FME. If the data needs ordering, it can be sorted in the workspace (though the added processing overhead may negate any gains). Sorting becomes more difficult according to the number of data streams. Multiple streams of data could be almost impossible to sort into the correct order, since all features matching a Group By value need to arrive before any features (of any feature type or dataset) belonging to the next group. In this case, using Group By with Process At End (Blocking) may be the equivalent and simpler approach. Note: Multiple feature types and features from multiple datasets will not generally naturally occur in the correct order. As with many scenarios, testing different approaches in your workspace with your data is the only definitive way to identify performance gains. |
Join Attributes
| Requestor List Attribute | The list attribute on Requestor features whose keys will be matched against the Supplier Attribute on Supplier features. |
| Supplier Attribute | The attribute on Supplier features that will be matched against the keys in the Requestor List Attribute for Requestor features. |
| Comparison Mode | Specifies how to perform the comparison between Requestor and Supplier attribute values. If Automatic or Numeric is specified, an attempt will be made to convert attribute values to numbers before comparing them. If the numeric conversion fails, string comparison will be used. |
Merge Parameters
| Feature Merge Type |
Attributes Only: Supplier attributes will be joined to Requestor features. Attributes will be merged according to the Attribute Accumulation and Generate List parameters Geometry: Supplier geometry will be merged to the Requestors, and will replace any existing geometry. Attributes are not merged. Attributes and Geometry: Both the geometry and attributes from the suppliers are joined to the Requestor features. Any geometry on the Requestor will be overwritten. Attributes will be merged according to the Attribute Accumulation and Generate List parameters. |
| Process Duplicate Suppliers |
If more than one Supplier is found for a given Requestor, and Process Duplicate Suppliers is No, then every Supplier after the first is output via the <Rejected> port and only the first of the Suppliers will be matched with a Requestor. If set to Yes, then the duplicate Suppliers are all matched with the corresponding Requestor merging attributes based on the Attribute Accumulation mode, and optionally output as an attribute list of the Requestor, using the specified List Name. The Supplier geometry is merged using the specified Geometry Merge Type. |
| Geometry Merge Type |
Build Polygons: If the Suppliers consist exclusively of polygon and donut polygon features, any common border segments will be removed. If the Suppliers contain at least one non-donut or non-polygon feature, then the transformer will form polygons and donuts from the Suppliers and will join connected line segments of the Supplier features before setting the geometry of the Requestor feature. In this case, the geometry may be an aggregate if several disjoint geometries were created. Build Aggregates: The transformer will create an aggregate of the geometries of the Supplier features. (If there is only one Supplier feature, then the Requestor geometry will be an aggregate with one element.) Build Lines from Points: The transformer will connect the points of the Supplier features into lines. Note that any non-point features that are referenced will be ignored when building lines. |
| Incomplete Requestors |
This parameter governs what happens to partially matched Requestors. If this parameter is set to Merge Supplier Information, then the Suppliers that were found will be merged onto the Requestor and then output via the UnmergedRequestor port. The Suppliers used will be output via the UsedSuppliers port. If this parameter is set to Do Not Merge Supplier Information, then the Requestor will be output untouched via the UnmergedRequestor port and the Suppliers will be output via the UnusedSupplier port. |
Attribute Accumulation
If attributes on the Supplier and Requestor feature share the same name, but are not geometry attributes that start with fme_, then they are deemed conflicted.
| Accumulation Mode |
Merge Supplier: The Requestor feature will retain all of its own un-conflicted attributes, and will additionally acquire any un-conflicted attributes that the Supplier feature has. This mode will handle conflicted attributes based on the Conflict Resolution parameter. Prefix Supplier: The Requestor feature will retain all of its own attributes. In addition, the Requestor will acquire attributes reflecting the Supplier feature’s attributes, with the name prefixed with the Prefix parameter. Only Use Supplier: The Requestor feature will have all of its attributes removed, except geometry attributes that start with fme_. Then, all of the attributes from one (arbitrary) Supplier feature will be placed onto the Requestor. |
| Conflict Resolution |
Use Requestor: If a conflict occurs, the Requestor values will be maintained. Use Supplier: If a conflict occurs, the values of the Supplier will be transferred onto the Requestor. |
| Ignore Nulls |
No: Treat null attribute values like other attribute values. Yes: Treat null attribute values as less important than other attribute values. Whenever a null value is merged with a non-null value, the non-null value shall prevail, regardless of what Conflict Resolution is set to. Note: Null and other attribute values are always more dominant than missing attribute values. That is, whenever an attribute value is merged with a missing attribute value, the attribute value shall prevail, regardless of the Ignore Nulls and Conflict Resolution settings. |
| Prefix |
To prevent a Supplier attribute from being ignored because the Requestor attribute already exists, you can optionally specify a prefix that will be applied to each Supplier attribute when it is added to the Requestor. When there are multiple Supplier features for a Requestor feature, multiple Supplier attribute values are merged into the same prefixed attribute. When multiple Supplier features have the same attribute, generally the resulting attribute value is taken from the last of these features. However, this process is governed by the Ignore Nulls parameter. |
Generate List
If there are duplicate Suppliers and Generate List is enabled, then any Suppliers that are combined with a Requestor will have their attributes added to the specified list on the Requestor.
Note: List attributes are not accessible from the output schema in Workbench unless they are first processed using a transformer that operates on them, such as ListExploder or ListConcatenator. Alternatively, AttributeExposer can be used.
| List Name | Enter a name for the list attribute. |
| Add To List |
All Attributes: Every attribute from a Supplier that combined with a Requestor will be added to the list specified in List Name. Selected Attributes: Only the attributes specified in the Selected Attributes parameter will be added to the list specified in List Name. |
| Selected Attributes | The attributes to be added to the list when Add To List is Selected Attributes. |
Editing Transformer Parameters
Using a set of menu options, transformer parameters can be assigned by referencing other elements in the workspace. More advanced functions, such as an advanced editor and an arithmetic editor, are also available in some transformers. To access a menu of these options, click 
beside the applicable parameter. For more information, see Transformer Parameter Menu Options.
Defining Values
There are several ways to define a value for use in a Transformer. The simplest is to simply type in a value or string, which can include functions of various types such as attribute references, math and string functions, and workspace parameters. There are a number of tools and shortcuts that can assist in constructing values, generally available from the drop-down context menu adjacent to the value field.
How to Set Parameter Values
Using the Text Editor
The Text Editor provides a convenient way to construct text strings (including regular expressions) from various data sources, such as attributes, parameters, and constants, where the result is used directly inside a parameter.
Using the Arithmetic Editor
The Arithmetic Editor provides a convenient way to construct math expressions from various data sources, such as attributes, parameters, and feature functions, where the result is used directly inside a parameter.
Conditional Values
Set values depending on one or more test conditions that either pass or fail.
Parameter Condition Definition Dialog
Content
Expressions and strings can include a number of functions, characters, parameters, and more.
When setting values - whether entered directly in a parameter or constructed using one of the editors - strings and expressions containing String, Math, Date/Time or FME Feature Functions will have those functions evaluated. Therefore, the names of these functions (in the form @<function_name>) should not be used as literal string values.
Content Types
| These functions manipulate and format strings. | |
|
Special Characters |
A set of control characters is available in the Text Editor. |
| Math functions are available in both editors. | |
| Date/Time Functions | Date and time functions are available in the Text Editor. |
| These operators are available in the Arithmetic Editor. | |
| These return primarily feature-specific values. | |
| FME and workspace-specific parameters may be used. | |
| Creating and Modifying User Parameters | Create your own editable parameters. |
Reference
|
Processing Behavior |
|
|
Feature Holding |
Yes |
| Dependencies | None |
| FME Licensing Level | FME Base Edition and above |
| Aliases | |
| History |
|
| Categories |
FME Community
The FME Community is the place for demos, how-tos, articles, FAQs, and more. Get answers to your questions, learn from other users, and suggest, vote, and comment on new features.
Search for all results about the ListBasedFeatureMerger on the FME Community.
Examples may contain information licensed under the Open Government Licence – Vancouver