ChangeDetector

Categories

Data Quality

Filters and Joins

Detects changes between two sets of input features.

This transformer is often used with multiple readers, to identify changed features in the two files. It can identify all features that two input files have in common, and those which are in one file and not the other, such as the additions and the deletions.

Usage Notes

  • The Matcher provides a more general approach, which may be more convenient for certain applications.
  • Oriented points are supported, however, precision loss when converting between orientation representations will often cause the ChangeDetector and Matcher to identify two oriented points as Updated/NotMatched unless Lenient Geometry Matching is enabled.

Configuration

Input Ports

One set of features enters the transformer via the Original port.

Another set of features enters via the Revised port.

Output Ports

This port is only relevant when Update Detection Key Attributes are specified in the “Transformer” section of the parameters.

When Updated Output > Mode is Insert & Update, a Revised feature is output via the Updated port when it has been prejoined on a unique key, but it differs from the Original feature that shares the same unique key.

When Updated Output > Mode is Insert & Update, a Revised feature with no match in the Original set is output via the Inserted port.

When Updated Output > Mode is Upsert, both Revised features with changes and new Revised features are ouput via the Upserted port.

An Original feature is output via the Deleted port when no match for it can be found in the Revised set.

An Original feature is output via the Unchanged port when it is found to have either matching geometry, matching attribute values, or both, with a feature in the Revised set.

Parameters

Update Detection Key Attributes This allows the features from the Original and Revised ports to be joined before they are compared. If a joined pair is identical, the feature from the Revised port will go out the Unchanged port. If a joined pair differs, the feature from the Revised port will go out the Updated port.
Attribute Matching Strategy

In conjunction with Selected Attributes, this parameter controls which attributes of input features must have the same value before a match is declared.

  • Match Selected Attributes: the attributes specified in the Selected Attributes parameter will be matched.
  • Match All Except Selected Attributes: all attributes will be matched, except those specified in Selected Attributes. Matching is performed on all attributes, including unexposed format attributes.
  • Match All Attributes: matching is performed on all attributes, including unexposed format attributes.
Selected Attributes Applicable when Attribute Matching Strategy is set to Match Selected Attributes or Match All Except Selected Attributes, and specifies which attributes to match, or exclude from the match, depending on the matching strategy chosen.
Differentiate Empty, Missing, and Null Attributes

No: Empty, missing, and null attributes will be treated as equivalent.

Yes: Empty, missing, and null attributes will be considered as pairwise different.
Check Attribute Types and Encodings

No: Attributes with different types and encodings are considered a match if their value is the same

Yes: Attributes must have the same type, encoding, and value to be considered a match.

Match Geometry

Controls whether 2D or 3D geometry must be the same before a match is declared. FULL makes sure 3D, measures, and geometry traits all match.

When comparing raster geometries:

  • 2D matches the properties
  • 3D matches the properties and values
  • FULL matches the properties, sidedness, values and geometry traits.

When comparing surface and solid geometries: 2D behaves the same way as 3D, that is, Z values will also be compared.

When comparing point cloud geometries:

  • 2D and 3D match the properties
  • FULL matches the properties and component values
Differentiate Empty, Missing, and Null Traits

No: Empty, missing, and null traits (and names) will be treated as equivalent.

Yes: Empty, missing, and null traits (and names) will be considered as pairwise different.

Lenient Geometry Matching

If Yes, then the order of points in line and area features will be ignored. Composition differences between paths, lines, and other curves will be ignored. The order of elements in collections, and the order of holes in Donuts will be ignored.

When comparing raster geometries, only the extents are compared.

Check Coordinate Systems

No: The Coordinate Systems are ignored

Yes: The name of the Coordinate System must be the same for both features
Vector Tolerance

Enter a value, in ground units. Allows two geometries to have spatial variation and still be considered a match if they are within the specified tolerance. It only applies to vector based geometries (points, curves, areas).

Default tolerance is zero (0). Unexpectedly high tolerance values can produce unpredictable results.

If more than two features match, the first will be designated a match. The others will then be compared against the remaining unmatched features.

Vector Tolerance is similar to a Fréchet distance: You have a dog leash of length “vector tolerance.” You will walk along the boundary of one of the geometries, your dog will walk along the boundary of the other. If you can both complete your walk without dropping the leash, the two geometries are within tolerance. This transformer has the added constraint for curves that you and your dog must both start at one end of your curve, and end at the other.

Mode

Select a handling method for new and modified features:

Insert & Update

Features will be output via the Inserted and/or Updated output ports as appropriate.

 
Upsert Features will be output via the Upserted port.
Detailed Change List Name

When given, new attributes will be added to the Updated port that indicate why each feature is different from the feature from the Original input port with the same unique key. There are three lists:

ChangeReportPrefix.ADDED{},

ChangeReportPrefix.REMOVED{}, and

ChangeReportPrefix.UPDATED{}.

For attributes, each element in the list will have an entry for “AttributeName,” the value of the attribute on the “Original” (if it was present), and the value on the “Revised” (if it was present). After the modified attributes are listed in the ChangeReportPrefix.UPDATED{} list, the OriginalCoordSys may be listed (if it differed), and a flag for the OriginalGeometry may be added (if it differed).

Original Attributes to Preserve

Only the attributes that exist on the original input port are allowed. The attributes that are selected in this parameter will not be updated, even if the revised feature has attributes with the same name.
Unchanged Output Ports When a feature from the Original port is matched with a feature from the Revised port, a representative feature is output from the Unchanged port. This setting specifies whether the representative should be taken from the Original port, the Revised port, or both
Match ID Attribute If Unchanged Output Ports is set to Output Both, then filling in this field will add a unique match id to each unchanged output pair, so that they may be rematched by a single attribute in the future.

Editing Transformer Parameters

Using a set of menu options, transformer parameters can be assigned by referencing other elements in the workspace. More advanced functions, such as an advanced editor and an arithmetic editor, are also available in some transformers. To access a menu of these options, click beside the applicable parameter. For more information, see Transformer Parameter Menu Options.

Defining Values

There are several ways to define a value for use in a Transformer. The simplest is to simply type in a value or string, which can include functions of various types such as attribute references, math and string functions, and workspace parameters. There are a number of tools and shortcuts that can assist in constructing values, generally available from the drop-down context menu adjacent to the value field.

Using the Text Editor

The Text Editor provides a convenient way to construct text strings (including regular expressions) from various data sources, such as attributes, parameters, and constants, where the result is used directly inside a parameter.

Text Editor

Using the Arithmetic Editor

The Arithmetic Editor provides a convenient way to construct math expressions from various data sources, such as attributes, parameters, and feature functions, where the result is used directly inside a parameter.

Arithmetic Editor

Conditional Values

Set values depending on one or more test conditions that either pass or fail.

Parameter Condition Definition Dialog

Content

Expressions and strings can include a number of functions, characters, parameters, and more.

When setting values - whether entered directly in a parameter or constructed using one of the editors - strings and expressions containing String, Math, Date/Time or FME Feature Functions will have those functions evaluated. Therefore, the names of these functions (in the form @<function_name>) should not be used as literal string values.

String Functions

These functions manipulate and format strings.

Special Characters

 A set of control characters is available in the Text Editor.

Math Functions

Math functions are available in both editors.
Date/Time Functions Date and time functions are available in the Text Editor.

Math Operators

These operators are available in the Arithmetic Editor.

FME Feature Functions

These return primarily feature-specific values.

FME Parameters

FME and workspace-specific parameters may be used.
Creating and Modifying User Parameters Create your own editable parameters.

Dialog Options - Tables

Transformers with table-style parameters have additional tools for populating and manipulating values.

Row Reordering

Enabled once you have clicked on a row item. Choices include:

  • Add a row
  • Remove a row
  • Move current row up one
  • Move current row down one
  • Move current row to top
  • Move current row to bottom

Cut, Copy, and Paste

Enabled once you have clicked on a row item. Choices include:

  • Cut a row - delete and copy to clipboard
  • Copy a row to the clipboard
  • Paste a row from the clipboard

Cut, copy, and paste may be used within a transformer, or between transformers.

Filter

Start typing a string, and the matrix will only display rows matching those characters. Searches all columns. This only affects the display of attributes within the transformer - it does not alter which attributes are output.

Import

Import populates the table with a set of new attributes read from a dataset. Specific application varies between transformers.

Reset/Refresh

Generally resets the table to its initial state, and may provide additional options to remove invalid entries. Behavior varies between transformers.

Note: Not all tools are available in all transformers.

FME Community

The FME Community is the place for demos, how-tos, articles, FAQs, and more. Get answers to your questions, learn from other users, and suggest, vote, and comment on new features.

Search for samples and information about this transformer on the FME Community.