FME Flow: 2024.2
An FME Flow system consists of a Client layer and a Server layer, each with a number of components.
Mouse-over to view architecture diagram:
FME Flow Components
A web application server runs the FME Flow Web User Interface, FME Flow Web Services, and any other web clients. Depending on the installation, the web application server may be a version of Apache Tomcat included with FME Flow, or provided separately.
2 Web Clients
The Web User Interface is included with FME Flow and can be run in a browser.
Custom web clients can be developed on top of the FME Flow REST API.
3 FME Flow Web Services
FME Flow provides predefined services to carry out common tasks. Services provided with FME Flow include:
- Transformation Services, which include the Data Download, Job Submitter, Data Streaming, and KML Network Link services.
- Data Upload service.
4 Non-Web Clients
- FME Workbench is the authoring environment for FME Flow but can also be used to submit jobs to FME Flow through a number of transformers.
- FME Flow CLI is a command line interface to FME Flow.
- Custom clients can be developed on top of the FME Flow REST API service.
- The FME Data Express mobile app uses the REST API and FME Flow Web Services over HTTP.
5 REST API
The FME Flow REST Service provides an underlying REST-based communication mechanism for client-service applications.
6 FME Flow API
All requests are sent to FME Flow through the low-level FME Flow API.
7 Security
FME Flow provides Authentication and Access Control using the Java Authentication and Authorization Service (JAAS) framework.
The FME Flow Core manages job requests, scheduling, and repository contents.
9 Process Monitor
The Process Monitor provides fault tolerance functionality, ensuring that the FME Flow Core and FME Engines remain available to process requests. The Process Monitor also provides a mechanism for managing the FME Flow and FME Engine components, including the ability to start, stop, restart, and add components. The Process Monitor uses the FME Flow Database to manage process configurations.
10 FME Engines
FME Engines process job requests by running FME Workspaces. Each FME Engine processes a single request at a time. FME Flow processing can be scaled by adding FME Engines to the same computer or to separate computers within a distributed FME Flow environment.
11 Datasets
Typically, an FME Flow job runs a workspace that reads and/or writes data. FME Flow administrators must ensure that FME Engines have read access to datasets or databases the workspaces read, and write access to any directories or databases the workspaces write to.
12 FME Flow Database
All information related to FME Flow is stored in the FME Flow Database, including jobs, repositories, automations, users, and other data. The database should never be edited directly.
Express installations of FME Flow run the FME Flow Database on a version of the PostgreSQL database server. Distributed installations run the FME Flow Database on a server provided separately. Supported database servers are PostgreSQL (recommended), Oracle, and SQL Server.
Note PostgreSQL is the recommended database server with FME Flow, offering enhanced performance and stability through targeted optimizations.
13 FME Flow System Share
In a Distributed/Fault-Tolerant Installation, the FME Flow System Share files are installed in directories on a remote file system to enable access from different computers. These files include Repositories and Resources.
FME Flow Connections
A
Web Clients use the FME Flow Web Services over HTTP. Communication is defined by the Web Services API, the REST API, and the FME Flow API.
B
Non-web Clients use the REST API over HTTP.
C
Web Clients, FME Flow Web Services and non-web clients use the FME Flow API to communicate with FME Flow over TCP/IP. Requests are sent to the FME Flow Core over port 7071. Messages are returned to clients over a randomly-assigned port created by the FME Flow Core.
D
The FME Flow API communicates over Port 7500 to the Process Monitor.
E
The FME Flow API sends job and repository requests to the FME Flow Core over TCP/IP port 7071.
F
The Process Monitor monitors the FME Engine processes and restarts them if they halt.
G
Once registered (see K below), the FME Engines communicate with the FME Flow Core over TCP/IP ports dynamically determined by the Core.
H
The FME Engines read and write data from shared/mounted drives, databases, web services, and others.
I
The FME Flow Core communicates with the FME Flow Database via JDBC over TCP/IP port 7082.
J
The FME Engines communicate with the FME Flow Database. The port varies depending on the selected database.
K
The FME Engines perform initial registration with the FME Flow Core over TCP/IP port 7070.
L
The FME Flow API can be used to send job scheduling requests to the FME Flow Core over TCP/IP port 7073.
M
The FME Flow API can be used to send notification requests to the FME Flow Core over TCP/IP port 7072.
N
Automations external actions and Notification Service subscribers perform initial registration with the FME Flow Core over TCP/IP port 7074. Subscribers process notifications received by the FME Flow Core.
O
Automations triggers and Notification Service publishers perform initial registration with the FME Flow Core over TCP/IP port 7076. The FME Flow API can be used to relay requests to the FME Flow Core over TCP/IP port 7075. For Email - SMTP trigger and SMTP Publisher ports, see FME Flow Ports.
P
The FME Flow configuration, and backup and restore requests, are communicated to the FME Flow Core over TCP/IP port 7077.
Q
Access to Resources is communicated with the FME Flow Core over TCP/IP Port 7079.
R
Communication with the FME Flow WebSocket Server takes place over port 7078.
S
Automatic (online) Licensing requests take place through HTTP over port 80.
T
The FME Flow Queue manages Queue Control and dispatches jobs as needed. Communication is over port 6379.
U
Database Connections and Web Connections requests are handled over port 7069.
V
System Cleanup requests are handled over port 7081.