Data Caching

Data caching seamlessly integrates the workspace with the FME Data Inspector. When enabled, output ports record feature data in temporary caches, which you can view and inspect after a completing a full or partial translation.

To enable data caching, do any of the following:

From the Menu Bar, select Test > Enable Data Caching.
Click the Data Caching icon on the Toolbar.
Enter Ctrl+F5.
Set the Enable Data Caching workspace parameter.

This example workspace shows the canvas with data caching enabled (mouse-over to view):

To view cached features, click an inspection icon on an output port ().

Note When a feature is selected for viewing, its icon changes to

The feature data output at that port is presented in the Data Preview pane or the FME Data Inspector, depending on your Data Inspection setting in Utilities > FME Options > Workbench.

To show features from multiple output ports, select one or more objects, right-click, and select Inspect cached features.

Limitations of Data Caching

When data caching is enabled, keep in mind the following:

Data caching affects system resources, and because all features are recorded and stored, it can also affect disk space. You may not want to keep all workspaces in this mode.
Data caches do not maintain input feature order, such as when workspaces run from multiple caches, or transformers run in Group Processing mode that specifies Complete Groups: When Group Changes (Advanced). To avoid unexpected results, place a junction before the location in the workspace where you expect a specific order, and use the junction as the basis for starting or stopping a partial translation (see Running and Inspecting Part of a Workspace, below).
To generate separate translation log files for each process of a translation that runs multiple processes, data caching must be off.

Running and Inspecting Part of a Workspace

You can select one or more objects on the canvas, and run only part of a workspace for inspection. Large workspaces, especially those that access large data sets, can take time to run in full. Running a partial workspace can save time and resources when:

You make a change to a workspace, and you want to see quickly how the change affects your data.
You are preparing data for input, and you want to see quickly how it looks.

You can run partial workspaces in the following ways:

Run From This (or Selected) (F6): All parts of the workspace that flow from a selected object (or objects) run.
Run Just This (or Selected): Only a selected object (or objects) runs. If a collapsed bookmark is included in the selection, only the objects within the bookmark run.
Run To This (or Selected) (Shift+F6): All parts of the workspace that flow to a selected object (or objects) run.
Run Between Selected: All parts of the workspace that flow between the selected objects run.

Working with Data Caches - Other Considerations

When Enable Data Caching is enabled, FME caches data at the output ports of a workspace. If any data caches are missing or stale where current ones are required, you are prompted to run all parts of the workspace that require updated data caches. This can happen when:

The caches were not saved with the workspace, and the workspace has not yet been run in Data Caching mode.
You make a change to the workspace that invalidates one or more caches.

When a cache is invalid, its corresponding inspection icon turns yellow (). (You can still click on an invalid icon to inspect features, though they may no longer be valid.)

When a workspace with existing data caches is run, FME includes any up-to-date caches in the run as required, but their corresponding objects do not run as part of the overall translation. To run the entire workspace and regenerate all caches, including up-to-date ones, specify Run > Run Entire Workspace (Shift + F5).

Tip Use Junctions to create caches where data either merges, or diverges in separate streams, as a useful starting point for partial runs.

Partial Caching

To conserve resources, you can prevent FME from caching portions of a workspace, if you know those portions do not require caching for producing accurate runs. Place the parts of the workspace you do not want to be cached in a bookmark, then collapse the bookmark. Any ports inside the collapsed bookmark that do not output features to components outside of the bookmark will not produce caches.

Clearing Caches

To assist in troubleshooting, you can clear data caches before rerunning a workspace in data caching mode. When you clear the cache of a selected reader or transformer, the cache of that item, and all caches downstream of that item, are cleared permanently. When you clear the caches of items in a selected bookmark, the caches of all items within the bookmark, and all caches downstream of those items, are cleared. To clear caches, right-click on a reader, transformer, or bookmark, and select Clear Data Cache.

Warning Clearing data caches is a permanent operation. It cannot be reversed through the Edit History Window.