Data Caching

Data caching seamlessly integrates the workspace with Data Preview or the FME Data Inspector. When enabled, output ports record feature data in temporary caches, which you can view and inspect after completing a full or partial translation, or even mid-translation.

Data caching is enabled by default. If it is disabled, do any of the following to re-enable:

• From the Menu Bar, select Run > Data Caching > Enable Data Caching.
• Click the Caching icon on the Toolbar.
• Enter Ctrl+F5.
• Set the Enable Data Caching workspace parameter.

This example workspace shows the canvas with generated caches after a workspace is run with data caching enabled:

To view cached features, click a cache icon on an output port. Complete caches are available to view after a translation runs successfully. Caches can also be viewed while a translation is still running. The coloring of icons varies depending on the state of the cache, as follows:

Cache Icon	Cache Icon When Clicked for Viewing	Meaning	Meaning When the Icon is Animated
		The translation completed successfully and a complete cache is available to view.	The translation is running and a cache is available to view before completion.
		The translation stopped and a cache that is considered stale is available to view. For more information, see Stale and Incomplete Caches (below).	The translation is running and a cache that is considered stale is available to view before the translation completes. For more information, see Stale and Incomplete Caches (below).
		The translation stopped or failed and an incomplete cache that is considered up-to-date is available to view. For more information, see Stale and Incomplete Caches (below).

The feature data output at the selected port is presented in the Data Preview pane or the FME Data Inspector, depending on your Data Inspection setting in Utilities > FME Options > Workbench.

Tip  During translation, messages may display with useful information to guide you in viewing more updated caches or options on stopping or pausing translation.

Limitations of Data Caching

When data caching is enabled, keep in mind the following:

• Data caching affects system resources, and because all features are recorded and stored, it can also affect disk space. You may not want to keep all workspaces in this mode. Alternatively, consider Running and Inspecting Part of a Workspace (below) or Skip Caching in Collapsed Bookmarks (below).
• Data caches do not maintain input feature order, such as when workspaces run from multiple caches, or transformers run in Group Processing mode that specifies Complete Groups: When Group Changes (Advanced). To avoid unexpected results, place a junction before the location in the workspace where you expect a specific order, and use the junction as the basis for starting or stopping a partial translation (see Running and Inspecting Part of a Workspace, below).
• To generate separate translation log files for each process of a translation that runs multiple processes, data caching must be off.

Running and Inspecting Part of a Workspace

You can select one or more objects on the canvas, and run only part of a workspace for inspection. Large workspaces, especially those that access large data sets, can take time to run in full. Running a partial workspace can save time and resources when:

• You make a change to a workspace, and you want to see quickly how the change affects your data.
• You are preparing data for input, and you want to see quickly how it looks.

You can run partial workspaces in the following ways:

• Run From This (or Selected) (F6): All parts of the workspace that flow from a selected object (or objects) run.
• Run Just This (or Selected): Only a selected object (or objects) runs. If a collapsed bookmark is included in the selection, only the objects within the bookmark run.
• Run To This (or Selected) (Shift+F6): All parts of the workspace that flow to a selected object (or objects) run.
• Run Between Selected: All parts of the workspace that flow between the selected objects run.

Working with Data Caches - Other Considerations

Stale and Incomplete Caches

A data cache is considered stale if you make a change to the workspace, after or during translation, that invalidates one or more caches. If a workspace fails or is manually stopped mid-translation, the incomplete cache that results may be considered stale depending on the cache settings of the workspace (below).

If a workspace is re-run, stale caches always regenerate if required for downstream translation.

A data cache is considered incomplete if the corresponding workspace fails or is manually stopped mid-translation.

If you want FME Workbench to keep incomplete data caches available for viewing, enable the Save Incomplete Cache on Stop/Fail setting, located under Run > Data Caching on the menu bar, or under Caching on the Toolbar. Alternatively, set the Keep Incomplete Cache on Failure or Stop workspace parameter. This setting is enabled by default.

When you opt to keep incomplete caches, use the following additional parameters to instruct FME on how to treat them when you re-run the workspace:

• Treat Incomplete Cache as Stale (default): When the workspace re-runs, the cache is regenerated if it is required for downstream translation.
• Treat Incomplete Cache as Up-to-date: When the workspace re-runs, the cache is considered up-to-date and is not regenerated.

Cache Icons and Their Meanings

The coloring of cache icons indicates whether they are considered complete, stale, or incomplete and up-to-date, as follows:

Skip Caching in Collapsed Bookmarks

To conserve resources, you can prevent FME from caching portions of a workspace, if you know those portions do not require caching for producing accurate runs. Place the parts of the workspace you do not want to be cached in a bookmark, then collapse the bookmark. Any ports inside the collapsed bookmark that do not output features to components outside of the bookmark will not produce caches.

Clearing Caches

When a workspace containing data caches reruns, FME clears the cache at the point where the run originates, and all downstream caches. This clearing occurs at the start of the run.

To assist in troubleshooting, you can clear individual data caches manually before rerunning a workspace in data caching mode. When you clear the cache of a selected reader or transformer, the cache of that item, and all caches downstream of that item, are cleared. When you clear the caches of items in a selected bookmark, the caches of all items within the bookmark, and all caches downstream of those items, are cleared. To clear caches, right-click on a reader, transformer, or bookmark, and select Clear Data Cache.

Viewing and Controlling Caching Mid-Translation

You cannot enable data caching during a translation, or re-enable caching after disabling it mid-translation. If you disable caching mid-translation, caches are removed.