PointOnPointOverlayer

Example: Aggregating information within a point dataset

In this example, we start with a single point dataset of reported crimes, over many years. The points are routed into the Point input port of the PointOnPointOverlayer.

There are two tasks we want to accomplish.

First, we want to count how many crimes were reported at any given location within the dataset. The Overlap Count Attribute will tell us how many other points were found within 2 meters of the location (as defined in the Point Tolerance parameter, with the data in a UTM projection, ground units in meters).

Second, we will build a list attribute that contains the type of each report in the location. We enable Generate List, and add the Selected Attribute “TYPE” to a list named IncidentType.

The point are output with no changes to their geometry, but with new attributes added. The example point selected now has an _overlaps value of 8, indicating that eight points (crime reports) in total are at this location.

The IncidentType list collects the Type attribute from each of the matching points.

Spatial Transformers Comparison Matrix

Transformer	Can Merge Attributes	Alters Geometry	Counts Related Features	Creates List	Supported Types*	Recommended For
SpatialFilter	Yes	No	No	No	Point Text Curve Area	Testing for the existence of spatial relationships between two sets of features, and routing them according to whether they pass or fail the test(s).
SpatialRelator	Yes	No	Yes	Yes	Point Text Curve Area	Identifying the nature of spatial relationships between two sets of features.
AreaOnAreaOverlayer	Yes	Yes	Yes	Yes	Area	Finding polygon overlaps and extracting them into new geometry.
LineOnAreaOverlayer	Yes	Yes	Yes	Yes	Curve and Area	Finding intersections between lines and polygons, and splitting the lines where they intersect with the polygons.
LineOnLineOverlayer	Yes	Yes	Yes	Yes	Curve	Finding intersections between line features, splitting them, and generating new line geometry as well as points representing the intersections.
PointOnAreaOverlayer	Yes	No	Yes	Yes	Point and Area Text and Area	Identifying points that fall within polygons, and merging attributes between them
PointOnLineOverlayer	Yes	Yes	Yes	Yes	Point and Curve Text and Curve	Identifying where points fall on lines, and splitting the lines into new geometry.
PointOnPointOverlayer	Yes	No	Yes	Yes	Point Text	Identifying points in the same location (within a tolerance), and merging attributes between them.
Intersector	Yes	Yes	Yes	Yes	Point Text Curve Area	Finding intersections between all input features, regardless of geometry (optionally including self-intersections), splitting features, and creating new geometry.
Clipper	Yes	Yes	No	No	Point Text Curve Area Solids Raster Point Cloud	Comparing features against a set of Clipper features, and splitting the features at or along the Clipper boundaries. Outputs both new and untouched geometry, identified as either Inside or Outside the Clipper.
NeighborFinder	Yes	In some cases	No	Yes	Point Text Curve Area	Identifying the nearest other feature(s) to each feature being considered, either in another set of features or within the same feature set.
TopologyBuilder	Yes	Yes	No	Yes	Point Text Curve Area	Analyzing spatial relationships between features to compute topology, splitting features and creating new geometry representing topologically significant nodes, edges, and faces, with associated attributes.

* NOTE: Curve includes Lines, Arcs, and Paths. Area includes Polygons, Donuts, and Ellipses.

The Effect of Spatial Transformer Choice on Performance

Spatial analysis can be processing-intensive, particularly when a large number of features are involved. If you would like to tune the performance of your workspace, this is a good place to start.

When there are multiple ways to configure a workspace to reach the same goal, it is often best to choose the transformer most specifically suited to your task. As an example, when comparing address points to building polygons, there are a few ways to approach it.

The first example, using a SpatialFilter to test whether or not points fall inside polygons, produces the correct result. But the SpatialFilter is a fairly complex transformer, able to test for multiple conditions and accept a wide range of geometry types. It isn’t optimized for the specific spatial relationship we are analyzing here.

With a SpatialFilter:

The second example uses a PointOnAreaOverlayer, followed by a Tester. The features output are the same as in the first method, but the transformer is optimized for this specific task. The difference in processing time is substantial - from 54.3 seconds in the first configuration, down to 13.7 seconds in the second one.

With a PointOnAreaOverlayer and a Tester:

If performance is an issue in your workspace, look for alternative methods, guided by geometry.

Point

Point features against which all other input points will be compared.

Point

Point features, with attributes added according to transformer parameter configuration. Geometry is unmodified.

Features with invalid geometries will be rejected and output via this port.

Rejected features will have an fme_rejection_code attribute with one of the following values:

INVALID_POINT_GEOMETRY_TYPE

Rejected Feature Handling: can be set to either terminate the translation or continue running when it encounters a rejected feature. This setting is available both as a default FME option and as a workspace parameter.

Transformer

Group By

The default behavior is to use the entire set of features as the group. This option allows you to select attributes that define which groups to form.

Parallel Processing

Select a level of parallel processing to apply. Default is No Parallelism.

Parallel Processing

Note: How parallel processing works with FME: see About Parallel Processing for detailed information.

This parameter determines whether or not the transformer should perform the work across parallel processes. If it is enabled, a process will be launched for each group specified by the Group By parameter.

Parallel Processing Levels

Parameter	Number of Processes
No Parallelism	1
Minimal	coresThe processor, or CPU, is the physical part of the computer that performs mathematical calculations. It is the most important part of a computer system. Traditional processors have only one core on the processor, meaning that at any given time, only one set of calculations is being performed. If a processor is dual-core, this means the single chip contains hardware for two processors, now called cores to distinguish them from the single chip, running simultaneously, side by side. (Source: http://www.ehow.com/facts_5730257_computer-core-processors_.html) / 2
Moderate	exact number of cores
Aggressive	cores x 1.5
Extreme	cores x 2

For example, on a quad-core machine, minimal parallelism will result in two simultaneous FME processes. Extreme parallelism on an 8-core machine would result in 16 simultaneous processes.

You can experiment with this feature and view the information in the Windows Task Manager and the Workbench Log window.

Input Ordered

No: This is the default behavior. Processing will only occur in this transformer once all input is present.

By Group: This transformer will process input groups in order. Changes of the value of the Group By parameter on the input stream will trigger batch processing on the currently accumulating group. This will improve overall speed if groups are large/complex, but could cause undesired behavior if input groups are not truly ordered. Specifically, on a two input-port transformer, "in order" means that an entire group must reach both ports before the next group reaches either port, for the transformer to work as expected. This may take careful consideration in a workspace, and should not be confused with both port's input streams being ordered individually, but not synchronously.

Considerations for Using Input is Ordered By

Using Ordered input can provide performance gains in some scenarios, however, it is not always preferable, or even possible. Consider the following when using it, with both one- and two-input transformers.

Single Datasets/Feature Types: Are generally the optimal candidates for Ordered processing. If you know that the dataset is correctly ordered by the Group By attribute, using Input is Ordered By can improve performance, depending on the size and complexity of the data.

If the input is coming from a database, using ORDER BY in a SQL statement to have the database pre-order the data can be an extremely effective way to improve performance. Consider using a Database Readers with a SQL statement, or the SQLCreator transformer.

Multiple Datasets/Feature Types: Since all features matching a Group By value need to arrive before any features (of any feature type or dataset) belonging to the next group, using Ordering with multiple feature types is more complicated than processing a single feature type.

Multiple feature types and features from multiple datasets will not generally naturally occur in the correct order.

One approach is to send all features through a Sorter, sorting on the expected Group By attribute. The Sorter is a feature-holding transformer, collecting all input features, performing the sort, and then releasing them all. They can then be sent through an appropriate filter (TestFilter, AttributeFilter, GeometryFilter, or others), which are not feature-holding, and will release the features one at a time to the transformer using Input is Ordered By, now in the expected order.

The processing overhead of sorting and filtering may negate the performance gains you will get from using Input is Ordered By. In this case, using Group By without using Input is Ordered By may be the equivalent and simpler approach.

In all cases when using Input is Ordered By, if you are not sure that the incoming features are properly ordered, they should be sorted (if a single feature type), or sorted and then filtered (for more than one feature or geometry type).

As with many scenarios, testing different approaches in your workspace with your data is the only definitive way to identify performance gains.

Parameters

Overlap Count Attribute

The Overlap Count Attribute added to output point features holds the number of point features to which the point was near.

Point Tolerance

Each input point has the attributes from any other point within the Point Tolerance distance merged onto it.

Aggregate Handling

Choose how aggregate geometries are to be handled.

Deaggregate: Decompose aggregates into their individual components.

Reject: Do not process aggregates and output them via the <Rejected> port.

Attribute Accumulation

If attributes on the incoming and original feature share the same name, but are not geometry attributes that start with fme_, then they are deemed conflicted.

Accumulation Mode

Drop Incoming Attributes: Features retain their original attributes, and do not receive new attributes.

Merge Incoming Attributes: The original feature will retain all of its own un-conflicted attributes, and will additionally acquire any un-conflicted attributes that the incoming feature has. This mode will handle conflicted attributes based on the Conflict Resolution parameter.

Prefix Incoming Attributes: The original feature will retain all of its own attributes. In addition, the original will acquire attributes reflecting the incoming feature’s attributes, with the name prefixed with the Prefix parameter.

Only Use Incoming Attributes: The original feature will have all of its attributes removed, except geometry attributes that start with fme_. Then, all of the attributes from one (arbitrary) incoming feature will be placed onto the original.

Conflict Resolution

Use Original: If a conflict occurs, the original values will be maintained.

Use Incoming: If a conflict occurs, the values of the incoming will be transferred onto the original.

Prefix

If the Accumulation Mode parameter is set to Prefix Incoming, this value will prefix attributes and measures that are being added to the original feature from the incoming feature.

Generate List

When enabled, adds a list attribute to the Point output features, and the attributes of matching points are added to that feature’s list.

List Name

Enter a name for the list attribute.

Note: List attributes are not accessible from the output schema in Workbench unless they are first processed using a transformer that operates on them, such as ListExploder or ListConcatenator. All list attribute transformers are displayed in the Contents pane of the Transformer Help under Lists. Alternatively, AttributeExposer can be used.

Add To List

All Attributes: All attributes will be added to the output Point features.

Selected Attributes: Enables the Selected Attributes parameter, where specific attributes may be chosen for inclusion.

Selected Attributes

Enabled when Add To List is set to Selected Attributes. Specify the attributes you wish to be included.

How to Set Parameter Values

Using the Text Editor

The Text Editor provides a convenient way to construct text strings (including regular expressions) from various data sources, such as attributes, parameters, and constants, where the result is used directly inside a parameter.

Text Editor

Using the Arithmetic Editor

The Arithmetic Editor provides a convenient way to construct math expressions from various data sources, such as attributes, parameters, and feature functions, where the result is used directly inside a parameter.

Arithmetic Editor

Conditional Values

Set values depending on one or more test conditions that either pass or fail.

Parameter Condition Definition Dialog

Content

Expressions and strings can include a number of functions, characters, parameters, and more - whether entered directly in a parameter or constructed using one of the editors.

Related Transformers

String Functions	These functions manipulate and format strings.
Special Characters	A set of control characters is available in the Text Editor.
Math Functions	Math functions are available in both editors.
Math Operators	These operators are available in the Arithmetic Editor.
FME Feature Functions	These return primarily feature-specific values.
FME Parameters	FME and workspace-specific parameters may be used.
Working with User Parameters	Create your own editable parameters.

Processing Behavior	Group-Based
Feature Holding	Yes
Dependencies
FME Licensing Level	FME Professional Edition and above
Aliases
History
Categories	Filters and Joins Spatial Analysis

PointOnPointOverlayer

Typical Uses

How does it work?

Usage Notes

Choosing a Spatial Transformer

Configuration

Input Ports

Output Ports

Parameters

Editing Transformer Parameters

Defining Values

Reference

FME Knowledge Center