Automations

Automations allow you to design JSON messaging workflows that support a rich set of communication protocols. With a simple set of tools, you can leverage your FME Flow to receive and send messages, and run workspaces on demand. This powerful combination allows you to control complex data flow patterns in your enterprise.

Automations are comprised of three primary components:

• Triggers: Listen for and receive messages from external clients or from within FME Flow.

Supported protocols

• External Actions: Send messages to external clients or inside FME Flow.

Supported protocols

• Actions: Process messages between triggers and external actions, either in an FME workspace, or through another tool.

Supported Actions

The example below illustrates a simple workflow comprised of a trigger that receives an email, an action to run a workspace, and an array of external actions that sends emails depending on the workspace results. An Email Received (IMAP) trigger sends messages to a Run Workspace action whenever an email from a specified IMAP server host is received. The workspace validates features in a file attached to the email, and reads features that pass validation into a database. The automation proceeds with separate Send an Email external actions based on the results of the validation in the Run Workspace action. One email sends details about features that passed validation and were read into the database, including details on feature type attributes. Another email sends details about features that failed validation. As well, two additional emails are sent depending on whether the workspace completed successfully or not. Finally, metadata about any Email Received triggers that fail to fetch emails is logged.

Mouse-over to view:

Getting Started with Automations

Start a new Automations workflow from the Get Started, Create Automation, or Manage Automations page. (To use an existing automation as a starting point, select it from the Manage Automations page, click Actions > Duplicate, and open the duplicated automation.)

To start, one undefined Trigger and "Next Action" (which can be either an action or external action) are placed on the workflow area. As you continue to define the workflow, additional "Next Action" components are added.

Alternatively, to add additional components, expand . Double-click or click-and-drag from one of the applicable icons.

1. Define Components

Define triggers, actions, and external actions, as desired.

To Define a Trigger

Double-click the trigger icon you want to define. Under Trigger Details, expand Select a Trigger Event and make a selection. For information about each trigger, including how to configure, see Automations Triggers.

Configure the following:

• Parameters: The settings required for the Trigger to function as desired.
Tip  Where applicable, click Validate to test that your parameters work as expected.
• Output Attributes: The JSON key-value pairs to include in the outgoing message. By default, all attributes are included. Optionally, you can create your own attributes. For more information, see Working with Output Attributes in Automations Workflows.

For more information, see the documentation for the specified trigger under Automations Triggers.

To Define an Action

Double-click the Next Action icon. Under Next Action Details, expand Select an Action and make a selection under Actions. For information about each action, including how to configure, see Automations Actions.

The canvas automatically connects your action to the previous component in the workflow. However, if connections are not specified, click an output port of an upstream component and an input port of the action. Alternatively, click and drag a connection line between ports.

Tip  Create connections to actions before configuring parameters. This way, you'll have access to any incoming parameters, where applicable. For more information, see Working with Parameters in Automations Workflows.

Configure the following, where applicable:

• Parameters: The settings that are required for the action to function as desired.
• Output Attributes: The JSON key-value pairs to include in the outgoing message. By default, all attributes are included. Optionally, you can create your own attributes. For more information, see Working with Output Attributes in Automations Workflows.
• Advanced: Run options for workspaces.
• Retry: Settings that control attempts to run the action again in case of failure to connect to an external resource.

For more information, see the documentation for the specified action under Automations Actions.

To Define an External Action

Double-click the Next Action icon. Under Next Action Details, expand Select an Action and make a selection under External Actions. For information about each external action, including how to configure, see Automations External Actions.

The canvas automatically connects your external action to the previous component in the workflow. However, if connections are not specified, click an output port of an upstream component and an input port of the external action. Alternatively, click and drag a connection line between ports.

Tip  Create connections to external actions before configuring parameters. This way, you'll have access to any incoming parameters, where applicable. For more information, see Working with Parameters in Automations Workflows.

Configure the following:

• Parameters: The settings required for the External Action to function as desired.
Tip  Where applicable, click Validate to test that your parameters work as expected.
• Output Attributes: The JSON key-value pairs to include in the outgoing message. By default, all attributes are included. Optionally, you can create your own attributes. For more information, see Working with Output Attributes in Automations Workflows.
• Retry: Settings that control attempts to run the action again in case of failure to connect to an external resource.

For more information, see the documentation for the specified external action under Automations External Actions.

2. Save, Name, and Tag the Workflow

Click the Save icon, or Select Menu > Save As. In the Save As dialog, specify a Name and one or more Tags (optional), and click OK.

Tip  If you have many workflows, use tags to organize them into meaningful categories. When viewing workflows on the Manage Automations page, you can filter on tags.
3. Start the Workflow

Workflows do not listen for incoming messages or run until you start them. You can start an automation directly from the build page, or from the Manage Automations page.

To start an automation from the build page, click Start or Debug:

• Start (recommended): This mode writes automation log files to the Logs > Automations folder in Resources, which are saved according to System Cleanup settings (see Configuring how long to keep Start mode Automation logs, below).
• Debug: Automation log files write to the FME Flow Database. Debug logs are cleared each time the automation restarts, regardless of mode. This mode is not recommended for running automations in a production environment.

Note  
• An automation runs as the user who started it. In addition to Run permission on the automation, this user must have Run permission on any workspace within the automation, as well as permissions on any other dependencies.
• You cannot edit a workflow unless it is stopped. Click Stop or Stop Debug.

Other Tasks - Building Workflows

To run a workflow on demand

You can create a workflow that does not respond to any incoming message or schedule, but runs on demand, and allows you the option to pass in sample JSON data. This type of workflow uses a Manual Trigger in place of one of the other trigger options. Manual triggers are useful when you want to test a workflow before you create one that receives real, incoming messages.

To enable looping in workflows

When looping is enabled, you can draw connections from downstream output ports back to the input ports of upstream components. Looping is a useful mechanism for creating process manager patterns in your workflows. For more information, see Looping Data between Workspaces. On the canvas, loops appear as dotted connection lines.

To enable looping, select Enable Looping from the canvas menu. This setting applies to the current workflow, as well as subsequent workflows that you open.

To disable looping, select Disable Looping from the canvas menu. Any existing loops remain in place.

To annotate a workflow

Annotate your workflow with comments. Select a component, right-click, and select Add Annotation. Alternatively, expand and drag from the Annotation icon onto the component. To add an annotation that is not connected to a component, unselect all components, and add an annotation onto a blank area of the canvas.

To add bookmarks to a workflow

Group related components together in bookmarks. Right-click on the canvas and select Add Bookmark, or expand and drag from the Bookmark icon onto the canvas. Update the text of the bookmark header, drag the bookmark over the components you want to group, and resize as desired. To change the color of the bookmark, right-click and select Change Color.

To cut, copy, paste, or remove components in a workflow

On the canvas, right-click on the components and select Cut, Copy, Paste, or Delete. Standard browser-based keyboard shortcuts, such as Ctrl + c, may also apply.

To disable or re-enable components in a workflow

Disabled components do not run as part of a workflow. On the canvas, select one or more components, right-click and select Disable. To re-enable, right-click and select Enable.

To add or remove a tag from a workflow

On the canvas, select Menu > Save As. In the Save As dialog, configure the tags as desired, and click OK.

Alternatively, on the Manage Automations page, select one or more worflows, and select Actions > Edit Tags.

Managing Automations

To start or stop a workflow

Workflows do not listen for incoming messages or run until they are started, and they cannot be edited until they are stopped.

On the Manage Automations page, select a workflow, and click Actions > Start or Debug:

• Start (recommended): This mode writes automation log files to the Logs > Automations folder in Resources, which are saved according to System Cleanup settings (see Configuring how long to keep Start mode Automation logs, below). In this mode, you can select multiple automations to start.
• Debug: Automation log files write to the FME Flow Database. Debug logs are cleared each time the automation restarts, regardless of mode. This mode is not recommended for running automations in a production environment. In this mode, you can select only one automation to start.

An automation runs as the user who started it. In addition to Run permission on the automation, this user must have Run permission on any workspace within the automation, as well as permissions on any other dependencies.

To stop a workflow

On the Manage Automations page, select a workflow, and click Actions > Stop or Stop Debug, depending on which mode the workflow is running. You can select multiple automations that are running in Start mode to stop them. You can only select one automation that is running in Debug mode to stop it.

To view event counts

FME Flow records the number of times a node in your automation runs. Event counts are visible on connection lines. In this example, both the FME Flow Schedule trigger and the Run a Workspace action have run ten times since the automation started:

Event counts automatically refresh every ten seconds. To expedite a refresh, click Refresh.

Once an automation is stopped, the final event counts display. These final counts are removed and reset when any of the following occurs:

• The automation is restarted.
• A connection is updated or removed. Only event counts of downstream nodes are affected.
• The parameters of a node are updated, or a node is removed, disabled, or enabled. Only event counts of downstream nodes are affected.

To open or edit an existing workflow

On the Manage Automations page, click on the workflow you want to open or edit.

To change the name of a workflow

• On the Manage Automations page, select a workflow, and select Actions > Rename.
• On the Create page of a workflow:
  • Click on the workflow name (beside Menu) to edit.
  • Select Menu > Rename.

To remove a workflow

On the Manage Automations page, select a workflow, and select Actions > Remove.

To share a workflow

On the Manage Automations page, click the Share with Others icon. Or, on the Create page of a workflow, select Menu > Share. Through sharing, you can grant levels of permissions on an item to other users. You can share an item if you own it, or if you are a user with Manage permission in Security, such as an administrator. For more information about sharing and ownership, see Role-Based and User-Based Access Control.

Sharing an automation may require granting additional permissions to other components in the automation, such as access to a repository to run a workspace. If these permissions are not already granted, FME Flow attempts to grant them. On the Sharing Options dialog, you can view the additional permissions that are required on the automation.

To view jobs triggered by a workflow

• On the Manage Automations page, select a workflow, and select Actions > View Triggered Jobs.
• On the Create page of a workflow, select Menu > View Triggered Jobs.

To create an automation app from a workflow

Select an automation that contains a Manual Trigger, click Actions, and select Create Automation App.

Viewing Automations

The Manage Automations page lists all the automations created on your FME Flow.

The columns displayed on this page are:

• Name: The user-defined name of the automation.
• Tags: The tags assigned to the automation, if any. For more information, see To filter the list of workflows, below.
• Last Updated: The time when the automation was last updated, using the following syntax:

year-month-day hour:minute:second

• Started: Whether the automation is started or not. Automations do not listen for incoming messages or run until they are explicitly started.
• Owner: The user who created the automation and has full permissions on it.
• Status: The current status of the automation.
• Share: To share an automation, click the Share with Others icon. For more information, see To share a workflow.
• Id: The system-generated ID of the automation. To copy, click the Copy to Clickboard icon.

To filter the list of workflows

• To apply a filter: On the Manage Automations page, expand the Filters bar.
• To add (or remove) tags: Use tags to organize workflows into meaningful categories on which you can filter. On the Manage Automations page, select Manage Tags.

Working with Automation Logs

Automation logs record all activities performed on workflows.

Working with Logs of Automations Run in Start Mode

Opening Start mode Automation logs

On the Manage Automations page, do one of the following:

• Select a workflow and click Actions > View Log Files.
• Click an Error or Warning icon under the Status column of a corresponding workflow. The log opens with "Filter errors" or "Filter warnings" set, respectively.

On the Create page of a workflow, select View Log Files from the menu.

Downloading aggregated Start mode Automation logs

You can download a log of an automation run in Start mode that is aggregated by a specified date range.

1. Do one of the following:
   • From the Manage Automations page, select an automation, and click Actions > Download Logs.
   • From the page of the automation that is open, select Download Logs from the Menu.
2. On the Download Logs dialog, specify a Start Date and End Date for the range of logs you want to download.
3. Click Download.

Navigating Start mode Automation logs

The Logs > Automations folder in Resources contains a folder for each process that can comprise any automation, including a folder for each type of trigger, action, external action, and other components. These logs contain metadata about the particular component, including its respective automation name, automation ID, event ID (if applicable), and message.

The following table describes some, but not all, of these folders:

Automation Log Folder	Description
fmeautomations.log	Contains metadata about assembling an automation, starting and stopping an automation, and running jobs.
action_fmelogaction	Contains metadata about Log a Message actions. Note  To open this log directly from within the Log a Message action of an automation, open the Log a Message action from the build page of the automation, and click Go to Debug Log.

Configuring how long to keep Start mode Automation logs

Use the Delete_Automation_Production_Logs task to configure how long to keep Start mode automation logs. By default, messages older than one week are removed.

See Also

• About Log Files in FME Flow

Working with Logs of Automations Run in Debug Mode

The log of a Debug mode automation is available only until the automation is restarted, regardless of mode.

Opening Debug mode Automation logs

On the Manage Automations page, do one of the following:

• Select a workflow and click Actions > View Debug Log.
• Click the View Debug Log icon under the Started column of a corresponding workflow.
• Click an Error or Warning icon under the Status column of a corresponding workflow. The log opens with "Filter errors" or "Filter warnings" set, respectively.

On the Create page of a workflow, select View Debug Log from the menu or, if the automation is still running, from the Debug Mode banner.

If the automation contains a Log a Message action, you can open log action_fmelogaction directly from the build page of the automation. Open the Log a Message action, and click Go to Debug Log.

Viewing Debug mode Automation logs

On the log page of an automation, use the toolbar to filter the log on Log a Message action, error, warning, or information messages. You can also:

• Download the log.
• Hide or display timestamps.
• Reverse the chronological order of log messages.
• Refresh the log display.

Configuring how long to keep Debug mode Automation logs

On the Scheduled Cleanups page, you can configure how long to keep Automation Debug log messages that have not already cleared by a restart of the automation. By default, messages older than two weeks are removed.