Automations

Automations allow you to design JSON messaging workflows that support a rich set of communication protocols. With a simple set of tools, you can leverage your FME Flow to receive and send messages, and run workspaces on demand. This powerful combination allows you to control complex data flow patterns in your enterprise.

Automations are comprised of three primary components:

  • Triggers: Listen for and receive messages from external clients or from within FME Flow.
  • Supported protocols

  • External Actions: Send messages to external clients or inside FME Flow.
  • Supported protocols

  • Actions: Process messages between triggers and external actions, either in an FME workspace, or through another tool.
  • Supported Actions

Tip  You do not need an external message to run a workspace. You can schedule a workspace to run at specific times in the future with the Schedule trigger, or on demand with the Manual Trigger.

The example below illustrates a simple workflow comprised of a trigger that receives an email, an action to run a workspace, and an array of external actions that sends emails depending on the workspace results. An Email Received (IMAP) trigger sends messages to a Run Workspace action whenever an email from a specified IMAP server host is received. The workspace validates features in a file attached to the email, and reads features that pass validation into a database. The automation proceeds with separate Send an Email external actions based on the results of the validation in the Run Workspace action. One email sends details about features that passed validation and were read into the database, including details on feature type attributes. Another email sends details about features that failed validation. As well, two additional emails are sent depending on whether the workspace completed successfully or not. Finally, metadata about any Email Received triggers that fail to fetch emails is logged.

Mouse-over to view:

Getting Started with Automations

Start a new Automations workflow from the Get Started, Create Automation, or Manage Automations page. (To use an existing automation as a starting point, select it from the Manage Automations page, click Actions > Duplicate, and open the duplicated automation.)

To start, one undefined Trigger and "Next Action" (which can be either an action or external action) are placed on the workflow area. As you continue to define the workflow, additional "Next Action" components are added.

Alternatively, to add additional components, expand . Double-click or click-and-drag from one of the applicable icons.

Tip  You can disable "guided mode" when building a workflow. Select Menu > Hide guides. To enable, select Menu > Show guides.
  1. Define Components
  2. Define triggers, actions, and external actions, as desired.

  3. Save, Name, and Tag the Workflow
  4. Click the Save icon, or Select Menu > Save As. In the Save As dialog, specify a Name and one or more Tags (optional), and click OK.

    Tip  If you have many workflows, use tags to organize them into meaningful categories. When viewing workflows on the Manage Automations page, you can filter on tags.
  5. Start the Workflow
  6. Workflows do not listen for incoming messages or run until you start them. You can start an automation directly from the build page, or from the Manage Automations page.

    To start an automation from the build page, click Start or Debug:

    Note  You cannot edit a workflow unless it is stopped. Click Stop or Stop Debug.

Other Tasks - Building Workflows

To run a workflow on demand

You can create a workflow that does not respond to any incoming message or schedule, but runs on demand, and allows you the option to pass in sample JSON data. This type of workflow uses a Manual Trigger in place of one of the other trigger options. Manual triggers are useful when you want to test a workflow before you create one that receives real, incoming messages.

To enable looping in workflows

When looping is enabled, you can draw connections from downstream output ports back to the input ports of upstream components. Looping is a useful mechanism for creating process manager patterns in your workflows. For more information, see Looping Data between Workspaces. On the canvas, loops appear as dotted connection lines.

To enable looping, select Enable Looping from the canvas menu. This setting applies to the current workflow, as well as subsequent workflows that you open.

Warning  Looping is an advanced feature. Exercise caution to avoid creating infinite loops that may overload your FME Engines. Additionally, looping upstream of a Merge Messages action is not recommended and may lead to unexpected behavior.

To disable looping, select Disable Looping from the canvas menu. Any existing loops remain in place.

To annotate a workflow

Annotate your workflow with comments. Select a component, right-click, and select Add Annotation. Alternatively, expand and drag from the Annotation icon onto the component. To add an annotation that is not connected to a component, unselect all components, and add an annotation onto a blank area of the canvas.

To add bookmarks to a workflow

Group related components together in bookmarks. Right-click on the canvas and select Add Bookmark, or expand and drag from the Bookmark icon onto the canvas. Update the text of the bookmark header, drag the bookmark over the components you want to group, and resize as desired. To change the color of the bookmark, right-click and select Change Color.

To cut, copy, paste, or remove components in a workflow

On the canvas, right-click on the components and select Cut, Copy, Paste, or Delete. Standard browser-based keyboard shortcuts, such as Ctrl + c, may also apply.

Tip  If you are building multiple workflows across instances of the Web User Interface in separate windows or tabs, you can cut/copy and paste between them.

To disable or re-enable components in a workflow

Disabled components do not run as part of a workflow. On the canvas, select one or more components, right-click and select Disable. To re-enable, right-click and select Enable.

To add or remove a tag from a workflow

On the canvas, select Menu > Save As. In the Save As dialog, configure the tags as desired, and click OK.

Alternatively, on the Manage Automations page, select one or more worflows, and select Actions > Edit Tags.

Managing Automations

To start or stop a workflow

Workflows do not listen for incoming messages or run until they are started, and they cannot be edited until they are stopped.

On the Manage Automations page, select a workflow, and click Actions > Start or Debug:

To stop a workflow

On the Manage Automations page, select a workflow, and click Actions > Stop or Stop Debug, depending on which mode the workflow is running. You can select multiple automations that are running in Start mode to stop them. You can only select one automation that is running in Debug mode to stop it.

To view event counts

FME Flow records the number of times a node in your automation runs. Event counts are visible on connection lines. In this example, both the FME Flow Schedule trigger and the Run a Workspace action have run ten times since the automation started:

Event counts automatically refresh every ten seconds. To expedite a refresh, click Refresh.

Once an automation is stopped, the final event counts display. These final counts are removed and reset when any of the following occurs:

  • The automation is restarted.
  • A connection is updated or removed. Only event counts of downstream nodes are affected.
  • The parameters of a node are updated, or a node is removed, disabled, or enabled. Only event counts of downstream nodes are affected.

To open or edit an existing workflow

On the Manage Automations page, click on the workflow you want to open or edit.

Note  You cannot edit a workflow unless it is stopped. Select the workflow and select Actions > Stop. Alternatively, on the Create page of a workflow, click Stop Automation.

To change the name of a workflow

  • On the Manage Automations page, select a workflow, and select Actions > Rename.
  • On the Create page of a workflow:
    • Click on the workflow name (beside Menu) to edit.
    • Select Menu > Rename.

To remove a workflow

On the Manage Automations page, select a workflow, and select Actions > Remove.

To share a workflow

On the Manage Automations page, click the Share with Others icon. Or, on the Create page of a workflow, select Menu > Share. Through sharing, you can grant levels of permissions on an item to other users. You can share an item if you own it, or if you are a user with Manage permission in Security, such as an administrator. For more information about sharing and ownership, see Role-Based and User-Based Access Control.

Sharing an automation may require granting additional permissions to other components in the automation, such as access to a repository to run a workspace. If these permissions are not already granted, FME Flow attempts to grant them. On the Sharing Options dialog, you can view the additional permissions that are required on the automation.

To view jobs triggered by a workflow

  • On the Manage Automations page, select a workflow, and select Actions > View Triggered Jobs.
  • On the Create page of a workflow, select Menu > View Triggered Jobs.

To create an automation app from a workflow

Select an automation that contains a Manual Trigger, click Actions, and select Create Automation App.

Viewing Automations

The Manage Automations page lists all the automations created on your FME Flow.

The columns displayed on this page are:

Note  Some fields do not display by default. To control which columns display, click the Customize Columns icon.
  • Name: The user-defined name of the automation.
  • Tags: The tags assigned to the automation, if any. For more information, see To filter the list of workflows, below.
  • Last Updated: The time when the automation was last updated, using the following syntax:
  • year-month-day hour:minute:second

  • Started: Whether the automation is started or not. Automations do not listen for incoming messages or run until they are explicitly started.
  • Owner: The user who created the automation and has full permissions on it.
  • Status: The current status of the automation.
  • Share: To share an automation, click the Share with Others icon. For more information, see To share a workflow.
  • Id: The system-generated ID of the automation. To copy, click the Copy to Clickboard icon.

To filter the list of workflows

  • To apply a filter: On the Manage Automations page, expand the Filters bar.
  • To add (or remove) tags: Use tags to organize workflows into meaningful categories on which you can filter. On the Manage Automations page, select Manage Tags.

Working with Automation Logs

Automation logs record all activities performed on workflows.

Working with Logs of Automations Run in Start Mode

Opening Start mode Automation logs

On the Manage Automations page, do one of the following:

  • Select a workflow and click Actions > View Log Files.
  • Click an Error or Warning icon under the Status column of a corresponding workflow. The log opens with "Filter errors" or "Filter warnings" set, respectively.

On the Create page of a workflow, select View Log Files from the menu.

Navigating Start mode Automation logs

The Logs > Automations folder in Resources contains a folder for each process that can comprise any automation, including a folder for each type of trigger, action, external action, and other components. These logs contain metadata about the particular component, including its respective automation name, automation ID, event ID (if applicable), and message.

The following table describes some, but not all, of these folders:

Automation Log Folder

Description

fmeautomations.log Contains metadata about assembling an automation, starting and stopping an automation, and running jobs.
action_fmelogaction

Contains metadata about Log a Message actions.

Note  To open this log directly from within the Log a Message action of an automation, open the Log a Message action from the build page of the automation, and click Go to Debug Log.

Configuring how long to keep Start mode Automation logs

Use the Delete_Automation_Production_Logs task to configure how long to keep Start mode automation logs. By default, messages older than one week are removed.

See Also

Working with Logs of Automations Run in Debug Mode

The log of a Debug mode automation is available only until the automation is restarted, regardless of mode.

Note  Debug mode is not recommended for automations that run in a production environment.

Opening Debug mode Automation logs

On the Manage Automations page, do one of the following:

  • Select a workflow and click Actions > View Debug Log.
  • Click the View Debug Log icon under the Started column of a corresponding workflow.
  • Click an Error or Warning icon under the Status column of a corresponding workflow. The log opens with "Filter errors" or "Filter warnings" set, respectively.

On the Create page of a workflow, select View Debug Log from the menu or, if the automation is still running, from the Debug Mode banner.

If the automation contains a Log a Message action, you can open log action_fmelogaction directly from the build page of the automation. Open the Log a Message action, and click Go to Debug Log.

Viewing Debug mode Automation logs

On the log page of an automation, use the toolbar to filter the log on Log a Message action, error, warning, or information messages. You can also:

  • Download the log.
  • Hide or display timestamps.
  • Reverse the chronological order of log messages.
  • Refresh the log display.

Configuring how long to keep Debug mode Automation logs

On the Scheduled Cleanups page, you can configure how long to keep Automation Debug log messages that have not already cleared by a restart of the automation. By default, messages older than two weeks are removed.