Remote Engine Services

Select Engine Management > Remote Engine Services.

Technology Preview

Note: This feature is under active development. Expect changes to the current behavior, and avoid using it in production environments.

Remote Engine Services allow you to use queues to run jobs on separate, specialized installations of FME Flow that may be closer to your data, while bypassing Queue Control rules. You can install Remote Engine Services and connect to them on servers that are part of your network, available outside your network on accessible endpoints, or in the Cloud, such as Azure Functions or Google Cloud Functions. Unlike Adding FME Engines on a Separate Machine, Remote Engine Services may be especially useful if you want to run jobs on servers outside your network, while maintaining your primary FME Flow installation behind a firewall.

Capabilities and Limitations

When running jobs on Remote Engine Services, keep in mind the following:

  • FME Engines of Remote Engine Services can run on any type of license, Standard or Dynamic. Multiple engines of both types are supported, depending on licensing terms.
  • Connecting to Remote Engine Services through a Proxy is supported.
  • FME Flow queues jobs when Remote Engine Services are offline, and submits them for processing once they are online.
  • Remote Engine Services can accept workspaces that are registered to run only the Job Submitter Service. Other FME Flow Web Services are not supported.
  • Remote Engine Services cannot run jobs submitted directly from a Run a Workspace or Run a Dynamic Workspace automations action. Workspaces submitted from Schedules or Workspace Apps are supported.

    • Tip: Workspaces that contain FMEFlowJobSubmitter transformers can submit jobs to Remote queues, even if those workspaces are invoked from Run a Workspace or Run a Dynamic Workspace actions.

  • Remote Engine Services cannot run custom formats or linked transformers.

Getting Started with Remote Engine Services

To target remote engine services to run jobs:

  1. Install remote engine services on your remote server.
  2. Create queues (if necessary).
  3. Create a connection to a Remote Engine Service and associate it with one or more queues.
  4. Run the job on one of the specified queues.

Install Remote Engine Services

Follow the instructions here to install remote engine services on your remote server. To obtain a remote engine service installer, visit the FME downloads page. Remote engine service installers are named beginning with fme-flow-remote-engine-*.

Create Queues (if necessary)

You may already have queues that you use to target jobs to FME Engines based on existing Queue Control rules. When specified explicitly in a Run Workspace directive, those same queues can run jobs on remote engine services, effectively bypassing queue control rules. Alternatively, you can create new queues and use those instead. To create new queues, select the Queues tab, and click New.

Create a Remote Engine Services Connection and Associate it with Queues

  1. On the Remote Engine Services tab of the Engine Management page, click Create and complete the following fields:
    • Name: Specify a name of your choice for the Remote Engine Service connection.
    • URL: The URL of the remote server on which the FME Flow Remote Engine Service was installed.
    • Username: The username of the FME Flow Remote Engine Service account.
    • Password: The password to authenticate the account Username.
    • Queues: The queues to associate with the Remote Engine Services connection. When one of these queues is specified explicitly in a run workspace directive, the job is routed to this connection.
  2. (Optional) If the Remote Engine Service is currently online, click Test to ensure the specified credentials are valid.
  3. Click OK.

The Remote Engine Services connection appears on the Engine Management page. You may need to refresh the page until the STATUS icon shows a green check mark.

Run a Job with the Remote Engine Services Connection

On the Run Workspace or Schedules page (under Advanced), specify the desired Job Queue associated with the Remote Engine Services connection that you want to run the job. The specified queue overrides the queue that would otherwise be assigned based on queue control rules, and instead routes the job to the Remote Engine Services connection.

Administering Remote Engine Services

Administrators of Remote Engine Services installations can access the Jobs, Analytics, Licensing, and Engines pages of the FME Flow Remote Engine Service Web User Interface.

Licensing Remote Engines

Licensing remote engines must be performed from the Licensing page on the Remote Engine Service, rather than from the primary FME Flow installation.

Resetting the Cache

When jobs are submitted to remote engines, FME Flow automatically uploads contents to the remote engine service that are necessary to run the job, including the workspace and its dependencies, such as resources, packages, web connections, web services, and database connections. To clear this cache, select the Remote Engine Service for the cache you want to clear, click Actions, and select Reset Cache.

Removing Remote Engine Services

Select the Remote Engine Service you want to remove, click Actions, and select Remove.