SchemaMapper

Converts the existing schema (data model) of features to a new structure, based on mappings defined in an external lookup table.

This technique is very useful when the mappings are potentially complex or when they need to be maintained by someone who is not familiar with FME. Using an external lookup table to define these mappings simplifies the completion of these tasks.

The schema mapping lookup table, used by the SchemaMapper transformer, defines a series of conditions that are to be met (filters), and a series of actions that will be executed when the conditions are met.

The lookup table may come in different formats such as: a comma-separated or plain text file; a spreadsheet (Excel or Google); or a database such as Oracle, PostGRES, Informix, SQL Server, etc.

If an incoming feature matches the rules in any row of the table, the potential actions of the transformer on that feature are listed by type as follows:

Map Feature Types:

Feature types – as defined by fme_feature_type – are mapped from their existing value, to one which defines the new feature type. For example:

Roads → Center Lines

Map Attributes:

One or more attributes on the feature are mapped by renaming their existing attribute name (s) to a new one. For example:

Name_of_Road → RoadName

Set New Attributes:

A new attribute is created whose name and value are defined in a lookup table.

Output Ports

Features that have an action carried out on them are output via the Mapped port. Otherwise, they are output via the Unmapped port.

In addition to the feature type, the geometry of each feature is left untouched.

Features output through the Mapped port will receive a new attribute that specifies which rows from the schema mapping table that were applied to the feature. The rows are specified as a comma-separated string (for example, '3,4,5'). This attribute is specified by the Selected Row Attribute parameter, and can only be changed from within the Navigator pane. Setting this parameter to a blank value will prevent the attribute from being added to features.

SchemaMapper Uses

You can use the SchemaMapper for:

Domain Mapping: where attributes values can be remapped according to a well-defined domain or lookup table. For example:

Primary Route → US Highway
Secondary Route → Interstate Highway
Primary Route → County Road

Dynamic Translations: where workspaces are created to handle any data structure

Automated Schema Mapping: where the manual connections between source and destination schemas are done automatically using an external lookup table.

Note A schema mapping table (domainSchema.csv) is often derived from a database metadata document such as Esri's XML database schema description, which can be exported from ArcCatalog for any selected geodatabase (Export > XML Workspace Document > Schema Only).

Example: Using the SchemaMapper transformer

The following example describes a typical scenario

Feature Type Mapping

Using an external lookup table (.csv file), you can map feature types to simplify a schema by mapping the old feature type to a new feature type:

Old Feature Type	New Feature Type
River	Water
Lake	Water
Canal	Water
Road	Transportation
Railway	Transportation
Airport	Transportation

Attribute Mapping

Using an external lookup table, you can map attributes to new names:

Old Attribute	New Attribute
River_Name	WaterName
River_Alt_Name	AltWaterName
River_Country	WaterCountry
River_Length	WaterSize
River_Owner	WaterOwner

Filters

You can use filters to set your mapping rules. For example, if you want to map an “old pipe type” to a “new pipe type” You can set the rule (filter) according to the size of the pipes:

Old Pipe Type	Pipe Size Attribute	Pipe Size	New Pipe Type
Gas	PipeSize	6	GasSmall
Gas	PipeSize	12	GasMedium
Gas	PipeSize	24	GasLarge
Water	Diameter	6	WaterSmall
Water	Diameter	12	WaterMedium
Water	Diameter	24	WaterLarge
Sewage	PipeDiam	6	SewageSmall
Sewage	PipeDiam	12	SewageMedium
Sewage	PipeDiam	24	SewageLarge

Blank or Null Attribute Value Handling

When creating a filter, the Blank Attribute Value parameter determines how blank or null attribute values are handled:

Ignore	Blank or null attribute values will be ignored, and will not be assigned new values by the filter.
Don't Ignore	Blank or null attribute values will be handled like any other attribute value. They may be referenced in either the source or destination values as an empty string ("").

Expressions

Expressions are supported in Filter Features (FilterValue) and Set New Attributes (TargetValue) actions.

These are defined in the external lookup table, and are identified with the keyword prefix fmeexpression=.

Filtering

This example lookup table will filter features based on the value of the attribute attr. If:

less than or equal to 15, or
greater than or equal to 450

the feature is considered mapped and is passed to the next Action (or output via the Mapped port, if there are no further actions).

Adding Attributes

Using the same example lookup table, if the action is Set New Attributes, features will receive two new attributes - attr_scaled10 and attr_scaled100, both with values calculated by expressions.

Note The AttributeManager and various filter transformers perform similar functions, without an external lookup table.

Example Using Feature Type Mapping and Filtering

You can use the SchemaMapper wizard to define the filters and the type of mapping that you need. You accomplish this task by creating actions to be executed on the source schema.

To open the SchemaMapper wizard, click the Properties button of the transformer. The SchemaMapper Parameters dialog opens.

Specify the format and location of the schema mapping table (lookup table) to be used. You may want to edit format parameters but you can use the defaults that FME Workbench provides. Click Next.
If a dataset has more than one table, specify the table to be used. Click Next.
On the Create Actions pane, click Add and select the action type. For this example, we'll select Feature Type Map.

Select the Source and Destination Feature Type Fields. You can show/hide the table columns, and refresh the contents.

Click OK to return to the Create Actions pane, which now shows the Feature Type Map action you just set.

Note To transform your source schema to a new schema, you can create more than one action depending on the mapping rules that you want to apply to the existing schema.

To add another action, select from the Add drop-down menu. In this example, we'll select Filter to display the Create Filter dialog.

Select the table columns in the Attribute Name and Value Fields. By default, Blank Attribute Values are ignored but you can choose to include them.
Click OK to return to the Create Actions dialog. The Action column now shows two actions: Map Feature Types and Filter Features.

Click Finish to set the SchemaMapper parameters.

More Information

For a good basic introduction to the SchemaMapper, see Crouching Schema Hidden Dragon.
This FME Support Center article contains descriptions and examples.

Editing Transformer Parameters

Transformer parameters can be set by directly entering values, using expressions, or referencing other elements in the workspace such as attribute values or user parameters. Various editors and context menus are available to assist. To see what is available, click beside the applicable parameter.

How to Set Parameter Values

Defining Values

There are several ways to define a value for use in a Transformer. The simplest is to simply type in a value or string, which can include functions of various types such as attribute references, math and string functions, and workspace parameters.

Using the Text Editor

The Text Editor provides a convenient way to construct text strings (including regular expressions) from various data sources, such as attributes, parameters, and constants, where the result is used directly inside a parameter.

Text Editor

Using the Arithmetic Editor

The Arithmetic Editor provides a convenient way to construct math expressions from various data sources, such as attributes, parameters, and feature functions, where the result is used directly inside a parameter.

Arithmetic Editor

Conditional Values

Set values depending on one or more test conditions that either pass or fail.

Parameter Condition Definition Dialog

Content

Expressions and strings can include a number of functions, characters, parameters, and more.

When setting values - whether entered directly in a parameter or constructed using one of the editors - strings and expressions containing String, Math, Date/Time or FME Feature Functions will have those functions evaluated. Therefore, the names of these functions (in the form @<function_name>) should not be used as literal string values.

Content Types

String Functions	These functions manipulate and format strings.
Special Characters	A set of control characters is available in the Text Editor.
Math Functions	Math functions are available in both editors.
Date/Time Functions	Date and time functions are available in the Text Editor.
Math Operators	These operators are available in the Arithmetic Editor.
FME Feature Functions	These return primarily feature-specific values.
FME Parameters	FME and workspace-specific parameters may be used.
Creating and Modifying User Parameters	Create your own editable parameters.

Dialog Options - Tables

Table Tools

Transformers with table-style parameters have additional tools for populating and manipulating values.

Row Reordering

Enabled once you have clicked on a row item. Choices include:

Add a row
Remove a row
Move current row up one
Move current row down one
Move current row to top
Move current row to bottom

Cut, Copy, and Paste

Enabled once you have clicked on a row item. Choices include:

Cut a row - delete and copy to clipboard
Copy a row to the clipboard
Paste a row from the clipboard

Cut, copy, and paste may be used within a transformer, or between transformers.

Filter

Start typing a string, and the matrix will only display rows matching those characters. Searches all columns. This only affects the display of attributes within the transformer - it does not alter which attributes are output.

Import

Import populates the table with a set of new attributes read from a dataset. Specific application varies between transformers.

Reset/Refresh

Generally resets the table to its initial state, and may provide additional options to remove invalid entries. Behavior varies between transformers.

Note: Not all tools are available in all transformers.

For more information, see Transformer Parameter Menu Options.

FME Community

The FME Community has a wealth of FME knowledge with over 20,000 active members worldwide. Get help with FME, share knowledge, and connect with users globally.

Search for all results about the SchemaMapper on the FME Community.