SchemaMapper
Converts the existing schema (data model) of features to a new structure, based on mappings defined in an external lookup table.
This technique is very useful when the mappings are potentially complex or when they need to be maintained by someone who is not familiar with FME. Using an external lookup table to define these mappings simplifies the completion of these tasks.
The schema mapping lookup table, used by the SchemaMapper transformer, defines a series of conditions that are to be met (filters), and a series of actions that will be executed when the conditions are met.
The lookup table may come in different formats such as: a comma-separated or plain text file; a spreadsheet (Excel or Google); or a database such as Oracle, PostGRES, Informix, SQL Server, etc.
If an incoming feature matches the rules in any row of the table, the potential actions of the transformer on that feature are listed by type as follows:
Map Feature Types:
Feature types – as defined by fme_feature_type – are mapped from their existing value, to one which defines the new feature type. For example:
Roads → Center Lines
Map Attributes:
One or more attributes on the feature are mapped by renaming their existing attribute name (s) to a new one. For example:
Name_of_Road → RoadName
Set New Attributes:
A new attribute is created whose name and value are defined in a lookup table.
Output Ports
Features that have an action carried out on them are output via the Mapped port. Otherwise, they are output via the Unmapped port.
In addition to the feature type, the geometry of each feature is left untouched.
Features output through the Mapped port will receive a new attribute that specifies which rows from the schema mapping table that were applied to the feature. The rows are specified as a comma-separated string (for example, '3,4,5'). This attribute is specified by the Selected Row Attribute parameter, and can only be changed from within the Navigator pane. Setting this parameter to a blank value will prevent the attribute from being added to features.
SchemaMapper Uses
You can use the SchemaMapper for:
Domain Mapping: where attributes values can be remapped according to a well-defined domain or lookup table. For example:
- Primary Route → US Highway
- Secondary Route → Interstate Highway
- Primary Route → County Road
Dynamic Translations: where workspaces are created to handle any data structure
Automated Schema Mapping: where the manual connections between source and destination schemas are done automatically using an external lookup table.
Note: A schema mapping table (domainSchema.csv) is often derived from a database metadata document such as Esri's XML database schema description, which can be exported from ArcCatalog for any selected geodatabase (Export > XML Workspace Document > Schema Only).
Example: Using the SchemaMapper transformer
The following example describes a typical scenario
Feature Type Mapping
Using an external lookup table (.csv file), you can map feature types to simplify a schema by mapping the old feature type to a new feature type:
Old Feature Type | New Feature Type |
---|---|
River | Water |
Lake | Water |
Canal | Water |
Road | Transportation |
Railway | Transportation |
Airport | Transportation |
Attribute Mapping
Using an external lookup table, you can map attributes to new names:
Old Attribute | New Attribute |
---|---|
River_Name | WaterName |
River_Alt_Name | AltWaterName |
River_Country | WaterCountry |
River_Length | WaterSize |
River_Owner | WaterOwner |
Filters
You can use filters to set your mapping rules. For example, if you want to map an “old pipe type” to a “new pipe type” You can set the rule (filter) according to the size of the pipes:
Old Pipe Type | Pipe Size Attribute | Pipe Size | New Pipe Type |
---|---|---|---|
Gas | PipeSize | 6 | GasSmall |
Gas | PipeSize | 12 | GasMedium |
Gas | PipeSize | 24 | GasLarge |
Water | Diameter | 6 | WaterSmall |
Water | Diameter | 12 | WaterMedium |
Water | Diameter | 24 | WaterLarge |
Sewage | PipeDiam | 6 | SewageSmall |
Sewage | PipeDiam | 12 | SewageMedium |
Sewage | PipeDiam | 24 | SewageLarge |
Blank or Null Attribute Value Handling
When creating a filter, the Blank Attribute Value parameter determines how blank or null attribute values are handled:
Ignore |
Blank or null attribute values will be ignored, and will not be assigned new values by the filter. |
Don't Ignore |
Blank or null attribute values will be handled like any other attribute value. They may be referenced in either the source or destination values as an empty string (""). |
Example Using Feature Type Mapping and Filtering
You can use the SchemaMapper wizard to define the filters and the type of mapping that you need. You accomplish this task by creating actions to be executed on the source schema.
To open the SchemaMapper wizard, click the Properties button of the transformer. The SchemaMapper Parameters dialog opens.
- Specify the format and location of the schema mapping table (lookup table) to be used. You may want to edit format parameters but you can use the defaults that Workbench provides. Click Next.
- If a dataset has more than one table, specify the table to be used. Click Next.
- On the Create Actions pane, click Add and select the action type. For this example, we'll select Feature Type Map.
- Select the Source and Destination Feature Type Fields. You can show/hide the table columns, and refresh the contents.
- Click OK to return to the Create Actions pane, which now shows the Feature Type Map action you just set.
Note: To transform your source schema to a new schema, you can create more than one action depending on the mapping rules that you want to apply to the existing schema.
- To add another action, select from the Add drop-down menu. In this example, we'll select Filter to display the Create Filter dialog:
- Select the table columns in the Attribute Name and Value Fields. By default, Blank Attribute Values are ignored but you can choose to include them.
- Click OK to return to the Create Actions dialog. The Action column now shows two actions: Map Feature Types and Filter Features:
- Click Finish to set the SchemaMapper parameters.
More Information
- For a good basic introduction to the SchemaMapper, see Crouching Schema Hidden Dragon.
- This FME Community article contains descriptions and examples.
Editing Transformer Parameters
Using a set of menu options, transformer parameters can be assigned by referencing other elements in the workspace. More advanced functions, such as an advanced editor and an arithmetic editor, are also available in some transformers. To access a menu of these options, click beside the applicable parameter. For more information, see Transformer Parameter Menu Options.
Defining Values
There are several ways to define a value for use in a Transformer. The simplest is to simply type in a value or string, which can include functions of various types such as attribute references, math and string functions, and workspace parameters. There are a number of tools and shortcuts that can assist in constructing values, generally available from the drop-down context menu adjacent to the value field.
Using the Text Editor
The Text Editor provides a convenient way to construct text strings (including regular expressions) from various data sources, such as attributes, parameters, and constants, where the result is used directly inside a parameter.
Using the Arithmetic Editor
The Arithmetic Editor provides a convenient way to construct math expressions from various data sources, such as attributes, parameters, and feature functions, where the result is used directly inside a parameter.
Conditional Values
Set values depending on one or more test conditions that either pass or fail.
Parameter Condition Definition Dialog
Content
Expressions and strings can include a number of functions, characters, parameters, and more.
When setting values - whether entered directly in a parameter or constructed using one of the editors - strings and expressions containing String, Math, Date/Time or FME Feature Functions will have those functions evaluated. Therefore, the names of these functions (in the form @<function_name>) should not be used as literal string values.
These functions manipulate and format strings. | |
Special Characters |
A set of control characters is available in the Text Editor. |
Math functions are available in both editors. | |
Date/Time Functions | Date and time functions are available in the Text Editor. |
These operators are available in the Arithmetic Editor. | |
These return primarily feature-specific values. | |
FME and workspace-specific parameters may be used. | |
Creating and Modifying User Parameters | Create your own editable parameters. |
Dialog Options - Tables
Transformers with table-style parameters have additional tools for populating and manipulating values.
Row Reordering
|
Enabled once you have clicked on a row item. Choices include:
|
Cut, Copy, and Paste
|
Enabled once you have clicked on a row item. Choices include:
Cut, copy, and paste may be used within a transformer, or between transformers. |
Filter
|
Start typing a string, and the matrix will only display rows matching those characters. Searches all columns. This only affects the display of attributes within the transformer - it does not alter which attributes are output. |
Import
|
Import populates the table with a set of new attributes read from a dataset. Specific application varies between transformers. |
Reset/Refresh
|
Generally resets the table to its initial state, and may provide additional options to remove invalid entries. Behavior varies between transformers. |
Note: Not all tools are available in all transformers.
FME Community
The FME Community is the place for demos, how-tos, articles, FAQs, and more. Get answers to your questions, learn from other users, and suggest, vote, and comment on new features.
Search for samples and information about this transformer on the FME Community.