Looping with a Blocking Transformer

FME can create processing loops around blocking (or group-based) transformers that would usually block the flow of features. This feature is similar to looping functionality but it operates in a very different manner.

Background

In a standard workspace, the feature processing loop processes one feature at a time. FME's usual workflow is to take one feature from the reader, push it through transformers, and then write the result to a writer before moving on to the next input feature. Transformers can divert features along different paths, but FME will push the single feature as far down the workspace as possible before moving on to the next feature.

Workbench provides a way to develop a custom transformer that can bypass FME's natural process of transformation: the transformer loop enables FME to take a feature from one point of processing within the custom transformer and insert it into another point. This is called Looping. Until now, looping was limited to feature-based transformers only.

Some transformers can block: that is, they might hold on to some features, very often until FME finishes processing. (This will happen when a transformer's "Group By" functionality is used, for instance, because groups cannot complete until every feature has been considered.) Transformers like this are also often called "Group-Based" transformers. When these transformers hold onto a feature, FME returns to the reader to get the next feature for processing. The blocking transformer cannot generate any output until all of its input has been accumulated. Logically, this means that a blocking transformer cannot be included in a transformer loop because the loop occurs at a time when the transformer is not yet prepared to take on new features.

The Enable Blocked Looping feature overcomes this limitation.

Requirements

There is only one requirement when using blocking transformers in loops: custom transformers must be external (saved and stored as an .fmx file) and always LINKED (not embedded).

Steps

1. Create a custom transformer. If it is an embedded custom transformer, export it as a linked custom transformer.



Alternatively, when you add a loop port to a custom transformer that contains a blocking transformer, FME prompts you to export the custom transformer, and displays the following message:



Note: This message does not display if you add a blocking transformer to a custom transformer that contains a loop port. However, you can still export the custom transformer.

2. Open the linked custom transformer (.fmx file) in Workbench. In the Navigator, under Transformer Properties, set "Enable Blocked Looping" parameter to Yes and save the changes.



Note: Only one LOOP is allowed when "Enable Blocked Looping" is set to Yes.

When "Blocked Looping" is enabled in a linked custom transformer, two additional parameters are exposed on the transformer: Maximum number of iterations and Attribute to hold iteration count:



These two parameters control how many times the loop is to be executed and help prevent infinite looping.

3. Insert this linked custom transformer into a workspace that requires looping. When you add the transformer to a workspace, it will appear as "Linked". (If it does not, right-click on the inserted transformer and choose Link.)
4. This inserted linked custom transformer will have one extra output port called INCOMPLETE. You cannot rename this port. It is created automatically to route the features that satisfy the looping condition but cannot be looped back because the loop iteration count has exceeded the value of "Maximum number of iterations".

Note: Any of the INPUT/OUTPUT ports in the linked custom transformer cannot use the name "INCOMPLETE".