FME Form: 2024.2
One issue that can affect FME performance when you're working with very large datasets is available RAM.
If you run a very large dataset through a workspace, you may see a message box that says “Out of Memory. Please free some memory, then choose retry.” This means that no more memory can be allocated.
Try the following options:
- Refer to the hints in the FME Community, in Performance Tuning FME.
- Setting the Temporary Directory.
- Purging Temporary Files
- Adding Swap Space (Linux only)
Setting the Temporary Directory
When FME runs a large, multi-dataset translation, it often requires a lot of temporary disk space. This is particularly true when running a dataset fanout, because there is no guarantee that the features will arrive at the fanout in a single dataset group. Therefore, FME has to write out all of the datasets to temporary storage, and then fan them out afterwards. So the amount of available disk space is important, but on a performance issue you might be more concerned about the speed of all this disk activity.
Many of the FME temporary files are created when caching data for larger datasets or for display in the FME Data Inspector. Using a faster hard drive can make a significant difference to the FME translation if disk cache I/O speeds are improved. An example would be if you have an SSD (Solid-State Drive) on your computer. These are typically quite a bit faster than traditional HHD drives, so if you can point your FME_TEMP to an SSD, then you'll see a bit of a performance increase when working with larger datasets.
Where possible, set your temporary folder to point to the fastest disk you have available.
Note This FME Community article tells you how to set the FME_TEMP environment variable. (In Windows 7, look under Control Panel > System > Advanced > Environment Variables).
Usage Notes
Don't set your temporary folder on the same disk that the operating system uses; FME might be slowed down by the operating system writing to the same disk at the same time.
Try to set the temporary folder to a disk that has a large amount of free space – it won't improve the speed, but it might prevent a large translation from failing due to a lack of disk space.
Purging Temporary Files
FME creates temporary files during translations. During a normal shutdown, FME automatically purges these files.
If FME does not shut down cleanly, the temporary files will not be purged, and they will occupy unnecessary disk space.
It’s a good idea to periodically purge any remaining files by selecting Tools > Purge Temporary Files.
Adding Swap Space on Linux
On Linux systems, you may see the following warning in the log:
The system is running low on memory. FME is at risk of being terminated by the OS.
On Linux, FME relies on the operating system to use swap space to manage memory usage, which requires that your system have enough swap space allocated for the entire memory footprint of FME and other running processes.
To increase swap space, see the Ubuntu SwapFaq at https://help.ubuntu.com/community/SwapFaq.