Remote Engine Services

Select Engine Management > Remote Engine Services.

Technology Preview

Note  This feature is under active development. Expect changes to the current behavior, and avoid using it in production environments.

Remote Engine Services allow you to use queues to run jobs on separate, specialized installations of FME Flow that may be closer to your data, while bypassing Queue Control rules. You can install Remote Engine Services and connect to them on servers that are part of your network, available outside your network on accessible endpoints, or in the Cloud, such as Azure Functions or Google Cloud Functions. Unlike Adding FME Engines on a Separate Machine, Remote Engine Services may be especially useful if you want to run jobs on servers outside your network, while maintaining your primary FME Flow installation behind a firewall.

Capabilities and Limitations

When running jobs on Remote Engine Services, keep in mind the following:

  • FME Engines of Remote Engine Services can run on any type of license, Standard or Dynamic. Multiple engines of both types are supported, depending on licensing terms.
  • Connecting to Remote Engine Services through a proxy is supported.
  • FME Flow queues jobs when Remote Engine Services are offline, and submits them for processing once they are online.
  • Remote Engine Services can accept workspaces that are registered to run only the Job Submitter Service. Other FME Flow Web Services are not supported.
  • Remote Engine Services cannot run jobs submitted directly from a Run a Workspace or Run a Dynamic Workspace automations action. Workspaces submitted from Schedules or Workspace Apps are supported.
    • Tip  Workspaces that contain FMEFlowJobSubmitter transformers can submit jobs to Remote queues, even if those workspaces are invoked from Run a Workspace or Run a Dynamic Workspace actions.
  • Remote Engine Services cannot run custom formats or linked transformers.

Getting Started with Remote Engine Services

To target remote engine services to run jobs:

  1. Install remote engine services on your remote server.
  2. Create queues (if necessary).
  3. Create a connection to a Remote Engine Service and associate it with one or more queues.
  4. Run the job on one of the specified queues.

Install Remote Engine Services

Follow the instructions here to install remote engine services on your remote server. To obtain a remote engine service installer, visit the FME downloads page. Remote engine service installers are named beginning with fme-flow-remote-engine-*.

Create Queues (if necessary)

You may already have queues that you use to target jobs to FME Engines based on existing Queue Control rules. When specified explicitly in a Run Workspace directive, those same queues can run jobs on remote engine services, effectively bypassing queue control rules. Alternatively, you can create new queues and use those instead. To create new queues, select the Queues tab, and click New.

Create a Remote Engine Services Connection and Associate it with Queues

  1. On the Remote Engine Services tab of the Engine Management page, click Create and complete the following fields:
    • Name: Specify a name of your choice for the Remote Engine Service connection.
    • URL: The URL of the remote server on which the FME Flow Remote Engine Service was installed.
    • Username: The username of the FME Flow Remote Engine Service account.
    • Password: The password to authenticate the account Username.
    • Queues: The queues to associate with the Remote Engine Services connection. When one of these queues is specified explicitly in a run workspace directive, the job is routed to this connection.
    • Standard Engines: Specify the number of Standard FME Engines you want to start on the Remote Engine Service for running jobs.
      Note  This number is limited to your Licensing terms.
    • CPU Usage Engines: Specify the number of Dynamic (CPU Usage) FME Engines you want to start on the Remote Engine Service for running jobs.
      Note  This number is limited to your Licensing terms.
    • Proxy Enabled: To connect to the Remote Engine Service through a proxy, check this box and complete the following:
      • Host: The proxy server hostname.
      • Port: The port through which communication with the proxy server takes place.
      • No Proxy For: (optional) A list of hostnames and IP addresses that will be accessed through a direct connection, bypassing the proxy. To avoid entering multiple hosts in the same domain, hosts can be prefixed and appended with the wildcard character *.

        • To delimit the list, use commas, semicolons, or press the tab or enter keys. Do not copy-paste a list of entries. To delineate properly in the display, each entry must be added separately.

        Example:

        Note  Specifying localhost does not include, by default, 127.0.0.1 or ::1. To bypass the proxy when the local host is referenced, you must explicitly specify any other expected forms for referencing local traffic.
      • Authentication Required: Click if authentication is required to access the proxy server, and complete the following:
        • Username: The account username for running the proxy server.
        • Password: The password for the proxy server account.
        • Authentication Method: The authentication method used by the proxy server, either Basic, Digest, or NTLM.
  2. (Optional) If the Remote Engine Service is currently online, click Test to ensure the specified credentials are valid.
  3. Click OK.

The Remote Engine Services connection appears on the Engine Management page. You may need to refresh the page until the STATUS icon shows a green check mark.

Run a Job with the Remote Engine Services Connection

On the Run Workspace or Schedules page (under Advanced), specify the desired Job Queue associated with the Remote Engine Services connection that you want to run the job. The specified queue overrides the queue that would otherwise be assigned based on queue control rules, and instead routes the job to the Remote Engine Services connection.

Administering Remote Engine Services

Administrators of Remote Engine Services installations can access the Jobs, Analytics, and Engines pages of the FME Flow Remote Engine Service Web User Interface.

Resetting the Cache

When jobs are submitted to remote engines, FME Flow automatically uploads contents to the remote engine service that are necessary to run the job, including the workspace and its dependencies, such as resources, packages, web connections, web services, and database connections. To clear this cache, select the Remote Engine Service for the cache you want to clear, click Actions, and select Reset Cache.

Removing Remote Engine Services

Select the Remote Engine Service you want to remove, click Actions, and select Remove.