You are here: Other Workbench Tasks > FME Administration Tasks > Adjusting Memory Resources

Adjusting Memory Resources

One issue that can affect FME performance when you're working with very large datasets is available RAM.

If you run a very large dataset through a workspace, you may see a message box that says “Out of Memory. Please free some memory, then choose retry.” This means that no more memory can be allocated.

Try the following options:

Refer to the hints in the FME Knowledge Center, in Performance Tuning FME.
Setting the Temporary Directory.
Purging Temporary Files
Adding Swap Space (Linux only)
Using the /3GB switch (32-bit Windows only).

Setting the Temporary Directory

When FME runs a large, multi-dataset translation, it often requires a lot of temporary disk space. This is particularly true when running a dataset fanout, because there is no guarantee that the features will arrive at the fanout in a single dataset group. Therefore, FME has to write out all of the datasets to temporary storage, and then fan them out afterwards. So the amount of available disk space is important, but on a performance issue you might be more concerned about the speed of all this disk activity.

Many of the FME temporary files are created when caching data for larger datasets or for display in the FME Data Inspector. Using a faster hard drive can make a significant difference to the FME translation if disk cache I/O speeds are improved. An example would be if you have an SSD (Solid-State Drive) on your computer. These are typically quite a bit faster than traditional HHD drives, so if you can point your FME_TEMP to an SSD, then you'll see a bit of a performance increase when working with larger datasets.

Where possible, set your temporary folder to point to the fastest disk you have available.

Note: The FME Knowledge Center article setting a different temporary folder tells you how to set the FME_TEMP environment variable. (In Windows 7, look under Control Panel > System > Advanced > Environment Variables).

Usage Notes

Don't set your temporary folder on the same disk that the operating system uses; FME might be slowed down by the operating system writing to the same disk at the same time.

Try to set the temporary folder to a disk that has a large amount of free space – it won't improve the speed, but it might prevent a large translation from failing due to a lack of disk space.

Purging Temporary Files

FME creates temporary files during translations. During a normal shutdown, FME automatically purges these files.

If FME does not shut down cleanly, the temporary files will not be purged, and they will occupy unnecessary disk space.

A "clean" shutdown is if you stop a translation by using the "Stop Translation" button on the toolbar, or when the workspace runs to completion. Temporary files are not purged if, for example, your computer crashes or if you close Workbench in mid-translation.

It’s a good idea to periodically purge any remaining files by selecting Tools > Purge Temporary Files.

Adding Swap Space on Linux

On Linux systems, you may see the following warning in the log:

The system is running low on memory. FME is at risk of being terminated by the OS.

On Linux, FME relies on the operating system to use swap space to manage memory usage, which requires that your system have enough swap space allocated for the entire memory footprint of FME and other running processes.

To increase swap space, see the Ubuntu SwapFaq at https://help.ubuntu.com/community/SwapFaq.

Using the /3GB Switch (32-bit Windows Only)

Note: The /3GB switch is not required for all, or even most, users of FME. The /3GB switch will only benefit those who require translations that will not currently run with 2GB of addressable memory.

Thirty-two-bit versions of the Windows operating system can manage a maximum of 4GB of addressable memory. The 4GB is divided into 2GB for user applications and 2GB for kernel processes. This means that any given application is restricted to 2GB of memory.

FME takes advantage of a /3GB switch which causes the operating system to divide the available 4GB of memory into 3GB for user applications and 1GB for kernel processes.

How do I enable the 3GB Switch for other versions of Windows?

Before FME can use the /3GB switch, you will need to edit your system’s boot.ini file.

You can access this file by doing the following:

Open the System Properties dialog. You can access this dialog by either opening Control Panel and selecting System, or by right-clicking on My Computer and selecting properties.
Select the Advanced tab at the top of the System Properties dialog.
Click the Settings button under Startup and Recovery.
Click the Edit button under System startup. Notepad will open with the boot.ini file.

Under [operating systems], there should be a line that ends with /fastdetect. Add /3GB to the end of this line.

The following is an example of a boot.ini file before the /3GB switch has been added:

[boot loader]
timeout=30
default=multi(0)disk(0)rdisk(0)partition(2)\WINDOWS
[operating systems] 
multi(0)disk(0)rdisk(0)partition(2)\WINDOWS="Microsoft Windows XP Professional" /fastdetect

When the /3GB switch is added, the above boot.ini file should look like the following:

[boot loader]
timeout=30
default=multi(0)disk(0)rdisk(0)partition(2)\WINDOWS
[operating systems] 
multi(0)disk(0)rdisk(0)partition(2)\WINDOWS="Microsoft Windows XP Professional" /fastdetect /3GB

You will now need to restart your computer. When your computer has finished rebooting, your translation should be able to finish without running out of memory.

To disable the 3GB switch for other versions of Windows:

Note: It is recommended that you remove the /3GB switch when you don’t require it.

Simply remove the /3GB switch from your boot.ini file and restart your computer.