Queue Control

Select Engine Management > Queue Control, Job Routing Rules, or Engine Assignment Rules.

Queue control is a mechanism for targeting FME Engines to run specific jobs. The reasons for using queue control include:

  • Controlling the priority of job requests.
  • Sending a job to an FME Engine in close proximity to a data source.
  • Reserving FME Engines for scheduled tasks.
  • Reserving Dynamic Engines for specific jobs.
  • Reserving some FME Engines for quick jobs, and others for high-load jobs.
  • Sending a job to an FME Engine that supports a particular format.

How It Works

Queue control is configured through three mechanisms that work together to ensure the right engines run your jobs.

  • Job Routing Rules: The criteria for a job to meet in order to qualify for a specific queue. Criteria can be based on performance metrics of a workspace, which are reported on the Workspaces page. Or, the criteria can be based simply on the repository to which a workspace belongs.
  • Engine Assignment Rules: The assignment of engines to queues based on properties of engines or queues. Alternatively, engines can be assigned to queues by engine name.
  • Queues: The FME Server mechanism that holds jobs and routes them to engines based on job routing and engine assignment rules.

Example

Things to Consider

When working with queue control, keep in mind the following:

  • You can assign priorities to queues. Depending on timing, jobs that route to higher-priority queues may be submitted before those in lower-priority queues.
  • A job routing rule can apply to only one queue. However, a queue can be assigned to multiple rules. If a job meets the criteria for more than one rule, the rules that takes precedence are listed in order on the Job Routing Rules tab. For more information, see Prioritizing Job Routing Rules (below).
  • When a job is submitted, it may qualify for more than one queue. However, if that same job is resubmitted, it is assigned the same queue from which it ran previously.
  • Jobs that do not route to any user-defined queues route to the Default queue.
  • For a metric-based job routing rule to take effect, the workspace must have run enough times since statistics are initially collected or reset to generate a statistically significant sample. The required minimum number of jobs to run for a significant sample may vary between 30 to 100, depending on variability of data and the degree of precision required by the rule (for example, minutes versus seconds). The more variable the input data, and the higher degree of precision that is required of this data, the larger the sample size that is required. If a job routing rule does not reference a statistically significant sample of jobs run, the next applicable rule takes effect, by priority, or the job routes to the Default queue.
  • Explicitly defining a queue, such as on the Run Workspace page, a Run a Workspace action, in Schedules, or through Job Directives, effectively bypasses job routing rules.
  • You can view the queues to which your FME Engines are assigned on the Engines tab.

Getting Started with Queue Control

To get started with queue control, create your own queues, specify the job routing rules that apply to each queue, and assign engines to run queued jobs.

Managing Queues and Rules

Prioritizing Job Routing Rules

A rule can apply to only one queue. However, a queue can be assigned to multiple rules. If a job meets the criteria for more than one rule, the rules that takes precedence are listed in order on the Job Routing Rules tab. To change the priority of a rule, click the Priority icon beside the rule, and select Move Up or Move Down.

Editing Queues and Rules

Open the Queues, Job Routing Rules, or Engine Assignment Rules tab, depending on what you want to edit. Click on a queue or rule to edit it.

Enabling and Disabling Rules

Open the Job Routing Rules or Engine Assignment Rules tab, depending on what you want to edit. Select one or more rules, click Actions, and select Enable or Disable.

When a job routing rule is disabled, jobs that would otherwise route to a queue based on the rule instead route to the next applicable rule, by priority (see Prioritizing Job Routing Rules, above). If no other rules apply, jobs route to the Default queue.

When an engine assignment rule is disabled, jobs that would otherwise route to engines based on this rule instead route to the Default queue.

Removing Queues and Rules

Note: You cannot remove the Default queue.

Open the Queues, Job Routing Rules, or Engine Assignment Rules tab, depending on what you want to remove. Select one or more queues or rules, click Actions, and select Remove.