Configuration File Layout

The FME Engines configuration file is comprised of several sections, each of which performs a specific set of tasks. The configuration file can contain two components: a global section and zero or more subsections.

The general layout looks like this:

GLOBAL_SECTION

<GlobalDirectives>

SUB_SECTION <name>

<SubsectionDirectives>

Global Section

The Global section defines all global directives that apply to a single FME Engine session. A session is defined as the period of time when an FME Engine is started up, performs some number of translations, and is shut down.

The Global section has the following general form:

GLOBAL_SECTION \

[FME_WORKING_DIR <results-dirpath> \]

[FME_MAPPING_DIR <repository-root-dirpath> \]

[FME_RESULT_LIFETIME <results-lifetime-seconds> \]

[FME_PURGE_INTERVAL <results-deletion-interval-seconds> \]

[AUTO_DIR_PREFIX <dir_prefix> \]

[AUTO_FILE_PREFIX <file_prefix> \]

[RECEIVE_TIMEOUT 0 \

[FME_SHARED_RESOURCE_DIR <shared_dir>

[FME_SERVER_LOG_FILE <server-logfilepath>\]

[MACRO_DEF FME_DATA_REPOSITORY <data-repository-dirpath>\]

[FME_ENGINE_MEMORY_REDLINE <percentage> \]

[SUCCESS_RESPONSE <message> \]

[FAILURE_RESPONSE <message> \]

The global directives are described in the following table. It’s important to remember that the subsection directives override the global directives.

GLOBAL_SECTION Directive

Description

FME_WORKING_DIR <results-dirpath>

Specifies the directory path into which the FME Flow writes all translation results unless the FME_WORKING_DIR is specified in the subsection for a particular job.

Default: Current Working Directory

Note  When changing the directory, you must also change the corresponding directive in the properties files of any web application services you use. For example, if you use the Data Download service, you must update the RESULT_DIR directive in the properties file for that service. The web application properties files are located at <FMEFlowDir>\Utilities\tomcat\webapps\<WebApp>\WEB-INF\conf.
Warning  Take care to ensure that the FME_WORKING_DIR value is set to the appropriate directory. This location is where the FME Engine writes its translation results. This is important – you can set the FME Engine to periodically and recursively delete all files in this directory that are older than a certain age (specified by FME_RESULT_LIFETIME). When the FME_WORKING_DIR is set to the wrong directory, the FME Flow deletes files in this directory instead of writing the translation results here.

FME_MAPPING_DIR <repository-root-dirpath>

Specifies the FME Flow’s repository root directory path, where workspaces run by the FME Engine during translations are contained.

Default: Current Working Directory

FME_RESULT_LIFETIME <results-lifetime-seconds>

Specifies how long, in seconds, translation results are held. The FME Engine periodically checks for, and recursively deletes, all files older than this amount from the FME_WORKING_DIR as a housekeeping measure. The frequency of this automatic deletion check is set by the value of FME_PURGE_INTERVAL, described next. Whenever automatic file deletion is performed, it occurs after a translation.

Default: If value is zero or directive is absent, results are never deleted – lifetime is infinite.

FME_PURGE_INTERVAL <results-deletion-interval-seconds>

Specifies, in seconds, the automatic deletion checking period. The FME Engine checks for files older than FME_RESULT_LIFETIME and deletes any it finds. This check is performed at the FME_PURGE_INTERVAL period. Note that the FME Engine decides whether or not to do the check only after each translation. It does not set a timer to trigger the check. This means that the check can occur at the interval set, but may also occur at longer intervals depending on how frequently translations are made.

Default: If this directive is absent, the default is 0 – checks after each translation if there are translation results to purge.

AUTO_DIR_PREFIX <dir_prefix> and AUTO_FILE_PREFIX <file_prefix>

The FME Engine configuration file uses the two pseudo-variables:S

!FME_AUTO_DIR_NAME!

!FME_AUTO_FILE_NAME!

to represent unique names for translation result directories and files respectively. By default, the names have this form:

FME_<cnnnnnnnnnnnnnn>

For example:

FME_a03508997017636

You can change the default FME_ prefix to a site-specific value. Site-specific prefixes are defined using the AUTO_DIR_PREFIX directive to set the prefix of directory names, and the AUTO_FILE_PREFIX directive to set the prefix for file names. The directives have the following syntax:

AUTO_DIR_PREFIX <prefix>

AUTO_FILE_PREFIX <prefix>

The most useful application of these directives is to set the filename prefix with the AUTO_FILE_PREFIX directive. Doing so allows the names of translation result files to better reflect the originating organization.

When either of these directives is present, the specified <prefix> value is used instead of FME_. When absent, the prefix defaults to the FME_ default value.

For example, if AUTO_FILE_PREFIX Noodle_ was specified, the translation result filenames would take this form:

Noodle_<cnnnnnnnnnnnnnn>

Note that a trailing underscore character in the prefix value is not assumed. If you want your prefix to have one of these characters, you must explicitly include it as part of the prefix value.

You can specify either or both of these directives in the global section and/or in any subsection of the configuration file. When specified in the global section, the prefix is used by all subsections that don't themselves specify one. Subsections that do specify a prefix use that one instead, and do not use any global prefix that was specified.

Default: When the directive is absent, a prefix of FME_ is used.

RECEIVE_TIMEOUT <timeout_period_ms>

This timeout is measured in milliseconds. When the FME Engine doesn’t receive any translation requests within this period of time, the FME Engine shuts itself down. The FME Flow system then starts a new FME Engine instance.

Under most operating conditions this value is set to its recommended default of zero. This setting means that the FME Engine waits indefinitely for requests and never shuts down due to a lack of incoming requests.

However, you can use this directive to specify a finite timeout period in environments where a network monitor shuts down connections that remain inactive beyond a preset period of time. When this is done to the connection on which an FME Engine is listening to receive requests, the FME Engine can no longer respond and enters a hung state.

Setting this directive to a non-zero value lets the FME Engine terminate itself after the specified time. The FME Engine effectively breaks out of the hung state, allowing the FME Flow system to start a new FME Engine instance that can respond, once again, to requests.

FME_SHARED_RESOURCE_DIR <shared_dirs>

Specifies the file paths of one or more root directories. These directories are used by the FME Engine to obtain different types of Resources, such as custom format, transformer, and coordinate system definitions. Use semicolons to separate multiple shared directory file paths.

Default:<FMEFlowDir>\Server\resources\shared

FME_SERVER_LOG_FILE <engine-logfilepath>

Specifies the file path to the FME Engine’s log file. This log file contains a dump of the Engine’s configuration file and other information relating to the overall operation of the FME Engine. Note that this file does not log individual translation request processing, which is logged separately in individual translation log files.

For each FME Engine and, therefore, for each fmeEngineConfig.txt, you need to specify a different fmeEngine.log filename.

You can use the pseudo variable !FME_INSTANCE_NAME! to create a log file named for the instance name like this: <FMEFlowDir>\Logs\fmeEngine_!FME_INSTANCE_NAME!.log

This directive includes an additional parameter that follows the file path. The parameter’s value can be TRUE or FALSE, indicating whether or not the log file should be appended to the time of each FME Engine startup. TRUE means the file is appended to and FALSE means it is overwritten.

Default:<FMEFlowDir>\Logs\fmeEngine.log FALSE

MACRO_DEF FME_DATA_REPOSITORY <data-repository-dirpath>

Specifies the directory used by FME Flow’s Web User Interface for temporarily uploading data. In a distributed environment, this directory must be accessible to the web server and to the FME Engines. Specifying a UNC path is recommended.

FME_ENGINE_MEMORY_
REDLINE <factor>

The automatic resource manager determines the optimal total memory the FME Engine process should use. It also dynamically allocates this total memory optimally to the algorithms within FME requesting it.

The FME_ENGINE_MEMORY_REDLINE directive is a hint to the FME Engine on how aggressive it should be in consuming memory. It takes a value between 0 and 1 (0.5 is the default value). For more aggressive memory usage, a value above 0.5 should be used. For safer memory usage, a value below 0.5 should be used. The risk in being too aggressive is the process running out of memory or the machine thrashing. The risk in being to conservative is that the process will take longer to complete.

Default: 0.5

Response Messages

SUCCESS_RESPONSE

FAILURE_RESPONSE

The SUCCESS_RESPONSE and FAILURE_RESPONSE directives are specified at the global level. They define the message string that’s returned by the FME Engine for successful and failed translations respectively. These directives provide the mechanism by which the FME Engine communicates results back to the client.

The content of these response messages is meaningful only to the client. The FME Engine simply passes the message as defined back to the client, which then processes it in whatever manner it chooses. The message can include the predefined pseudo-variables and several other directives.

Special-Purpose Directives

In addition to the common Global section directives listed in the preceding table, the following special-purpose directive are also available when required. It’s important to remember that the subsection directives override the global directives.

Special Purpose
Directives

SDE30_PERM_CONNECT <host> <instance> <database> <userID> <password>

 

Defines a permanent connection to an ESRI SDE Server. This connection is brought up just before the first translation is performed on the FME Engine. The connection is then held by the Engine so subsequent translations need not establish a new SDE connection.

Take care when using this directive because SDE connections are a valuable resource and you should use them sparingly. In general, it’s good practice to first use the FME Engine without any permanent connection. You may consider a permanent connection later, if you find that connecting and disconnecting to SDE is too expensive.

<host>

The name of the host computer on which the SDE server is running.

Default: Site-specific

<instance>

The SDE instance to which the FME Engine is to connect.

Default:port:5151

<database>

The database on the instance that is to be connected. When the SDE is on databases such as Oracle the value specified is not used. Although you can specify any value, the convention is to specify the value NOTUSED for the database.

Default: Site-specific

<userID>

The user account used to log in to the SDE.

Default: Site-specific

<password>

The user password of the user account.

Default: Site-specific

Subsections

Subsections are optional and when present their directives override global directives. When a subsection is present it’s named with a keyword by which client applications can reference it. You can define multiple subsections for various purposes.

In an FME Flow environment, each service (such as Data Download, Data Streaming, and so forth) has its own subsection.

If a job request — including one from an FME Flow service — specifies a subsection, the FME Engine runs that job using the directives in the specified subsection. The following table shows the FME Flow Web Services and the corresponding subsections used by the FME Engine when running a job request from that service.

Note  Subsection directives take precedence over global directives.

FME Flow Service

FME Engine SUB_SECTION

Data Download Service

FILE_DOWNLOAD_SERVICE

Data Streaming Service

STREAM_DOWNLOAD_SERVICE

FME Flow Console

SERVER_CONSOLE_CLIENT

FMEServerJobSubmitter transformer
(default subsection)

SERVER_JOB_SUBMITTER

Job Submitter Service

JOB_SUBMITTER_SERVICE

REST Service

REST_SERVICE

FME Flow Scheduler

SERVER_SCHEDULER

Each subsection can specify operations that the FME Engine should perform before and after a translation request is processed. The subsection can also define what translation success and failure response messages get returned to the client, and can define FME workspace macro values as well.

In addition to their own directives, subsections can use a number of directives available to the Global section to override the global values for that particular subsection.

By specifying a subsection by name in the translation request, clients can cause the operations defined by the subsection to be performed by the FME Engine for the translation being requested.

Each subsection has the following general form:

SUB_SECTION <keyword> \

[<Global Directives> \]

[FME_TRANSFORMATION_LOG_DIR <logfile-dirpath> \]

[MACRO_DEF <macroName> <macroValue> \]*

[SUCCESS_RESPONSE <message>  \]

[FAILURE_RESPONSE <message>  \]

The subsection directives, which override global directives, are described in the following table.

SUB_SECTION Directives

Description

SUB_SECTION <keyword>

Identifies the subsection that’s being defined. There is no limit to the number of subsections that you can define. Identify each subsection with a unique <keyword> – it cannot begin with FME_.

<Global Directives>

The following global directives are also available to subsections where you can redefine them to have different values:

FME_WORKING_DIR

FME_MAPPING_DIR

AUTO_DIR_PREFIX

AUTO_FILE_PREFIX

SUCCESS_RESPONSE

FAILURE_RESPONSE

Redefined directives override the global value for the particular subsection for which they are defined.

FME_TRANSFORMATION_LOG_DIR <logfile-dirpath>

Specifies the path to the default directory containing the per-translation log files generated by the FME Engine for that subsection.

Default: <FMEFlowDir>\Logs\<subsectionSpecificDir>

MACRO_DEF <macroName> <macroValue>>

Defines a macro value that is supplied to the FME workspace when the translation is performed. The first parameter <macroName> specifies the name of the macro.

All the tokens up to the next directive or the end of the SUB_SECTION definition form the value for <macroValue>.

You can nest macro references.

When defined, macros are available for use in success and failure response strings, using the pseudo-variable (!macroName!) syntax.

Some FME Engine subsections define a specific macro called FME_SERVER_DEST_DIR whose value is set as a path to a unique, system-generated directory. The intention is that this directory is used as an output location for translation result files. Workspace authors can reference this value as a published parameter and use it as a well-defined location for destination datasets.

Response Messages

SUCCESS_RESPONSE

FAILURE_RESPONSE

The SUCCESS_RESPONSE and FAILURE_RESPONSE directives define the message string that’s returned by the FME Engine for successful and failed translations respectively. These directives provide the mechanism by which the FME Engine communicates results back to the client.

The content of these response messages is meaningful only to the client. The FME Engine simply passes the message as defined back to the client, which then processes it in whatever manner it chooses. The message can include the predefined pseudo-variables and several other directives.