About Log Files in FME Flow

FME Flow creates log files for every workspace it runs and for all of its components. Use these logs to test and troubleshoot any issues related to the component that is not producing expected results.

To ensure that log files do not consume too much disk storage over time, FME Flow is configured, by default, to delete old log files. You can manage automatic log file cleanup on the System Cleanup page of the FME Flow Web User Interface.

Log files can be accessed on the Resources page (Logs folder) of the Web User Interface, or from the REST API.

Log files are grouped in subdirectories based on their function, in either a current or old directory. Except for FME job logs, log files are auto-archived from the current to the old directory when either of the following occurs:

• A new log file of the same name is created.
• The size of the file has reached 5000 lines of text.

You can control the way log files are auto-archived, and other properties of log files, by editing the applicable messagelogger.properties file. For more information, see Message Logger Properties.

FME Job Logs

Location:

<LogDir>\engine\current\jobs\<number>\job_<value>.log

Workspaces that are run from FME Flow generate FME logs in the same manner as those run from FME Form. These files log the details of a translation performed by an FME Engine, and can provide important information when testing or troubleshooting jobs run from FME Flow.

FME job logs can also be accessed from the Jobs page of the Web User Interface.

FME job logs always reside in the \engine\current folder and are not auto-archived to the \engine\old folder.

FME job log files are stored and named using the following format:

\<number>\job_<value>.log

Where <value> is the job ID and <number> is the job ID group, which increments by a thousand. Logs for job IDs from 0 - 999 are grouped into a 0 directory, Logs for job IDs from 1000 - 1999 are grouped into a 1000 directory, and so on.

Logs for jobs that run from Streams are stored and named using the following format:

\<number>\job_<value>\<number>\job_<value>_<sequence>.log

Where <sequence> is the sequence of each streaming job for the job ID.

Additionally, logs for jobs from streams split after 5000 lines by default (customizable in Message Logger Properties). FME job logs do not otherwise split. When a streaming job log splits, _<date>_<time> is appended to the filename, where <date> is the date that the streaming job log split, and <time> is the time that the streaming job log split.

Automations Logs

Location:

<LogDir>\automations\current|old

Provides logs on Automations that are run in Start mode. For more information, see Working with Logs of Automations Run in Debug Mode.

FME Core Logs

Publisher Logs

Location:

<LogDir>\core\current|old\publishers

This directory provides logs for each of the Notification Service publisher protocols supported by FME Flow. Each file logs activities of any publishers configured on the respective protocol.

Subscriber Logs

Location:

<LogDir>\core\current|old\subscribers

This directory provides logs for each of the Notification Service subscriber protocols supported by FME Flow. Each file logs activities of any subscribers configured on the respective protocol.

Task Logs

Location:

<LogDir>\core\current|old\tasks

FME Flow configuration Backup & Restore and project export/import operations.

Other Core Logs

Location:

<LogDir>\core\current|old\

fmeserver.log

This log is frequently used in testing and troubleshooting, and is created each time the FME Flow is started. It shows:

• Authentication and authorization information, including connecting to Authentication Services, if applicable.
• Connection parameters relating to the FME Flow Database.
• Messages created by Job Directives and the Repository Manager.
• The job_id of a specific job and its life cycle, including submission to the queue, processing by an FME Engine, and completion.

fmeconfiguration.log

Logs activity when performing a backup and restore.

fmedirectoryserver.log

Logs activity about connecting and authenticating with an LDAP server.

fmeprocessmonitorcore.log

Process monitor logs are frequently used in testing and troubleshooting. A new process monitor log is created each time the FME Flow is started. This log details the FME Flow components that were started and if any components were restarted at any time. This log shows translation (workspace) jobs that are submitted to the queue.

fmescheduler.log

The scheduler log records the startup and shutdown of the Scheduler component, and every workspace run by the Scheduler.

fmesharedresource.log

This log records activity that affects any Resources directories.

FME Engine Logs

 

Location:

<LogDir>\engine\current|old\

fmeprocessmonitorengine.log

This log shows:

• The build number of the engines running the FME Flow.
• If an FME Engine fails to start and an error message indicating why, such as a licensing issue.
• Any debugging information generated by a Python shutdown script.

Services Logs

Location:

<LogDir>\service\current|old\

This directory provides logs for each of the Web Services supported by FME Flow. Login activity through a SAML identity provider is also recorded here.

Installation Logs

Location:

<FMEFlowDir>\resources\logs\installation

These logs contain information about the installation of FME Flow. It is only necessary to read them if you have had a problem installing FME Flow.

CreateDatabase.log/CreateSchema.log/CreateUser.log

These logs contain information about initializing the FME Flow Database.

Web Application Server Logs

Location:

<FMEFlowDir>\Utilities\tomcat\logs

The Web Application Server is a third-party application used to run FME Flow Web Services. The default application server is Apache Tomcat, but others can be configured. Application servers generally do their own logging and these logs can be helpful in troubleshooting problems with services.