FME Packages Specification

This is the external specification for FME Packages (fpkg), version 1.

Package filenames

An FME Package is a ZIP archive. Its filename follows this pattern:

[publisher-uid].[package-uid]-[package_version].fpkg

The filename is composed of the publisher uid, followed by the package uid and the package version. See the package.yml section for details on these values.

Package structure

This is the unpacked structure of an FME Package. Names are case-sensitive. README.md, CHANGES.md, and package.yml are required.

A package may contain any number of formats, transformers, Web Filesystems, Web Services, and Python libraries. However, the client only installs items listed within package.yml.

FME Package folder structure

package.yml
README.md
CHANGES.md
icon.png
formats
    fooformat.fmf, .fms, .db, .md, .fme, .sch, …
barformat.fmf, .fms, .db, .md, .fme, .sch, …
    ...
transformers
    FooTransformer.fmxj, .fms, .md, .fme, …
    BarTransformer.fmx, .fms, .md, .fme, …
    ...
web_filesystems
    Web Filesystem.fme, .png
    *.fme, *.png
web_services
    English Service Name.xml
    *.xml
python
    fooformat-1.2.3-py2-py3-none-any.whl
    third_dependency-1.2.3-py2-py3-none-any.whl
    *.whl
localization
    transformer-localized.ja, .fr, … 
    guiprompts_ja.txt, guiprompts_fr.txt, … 
    FooTransformer_ja.md, FooTransformer_fr.md, … 
    help_fr_ca/, help_ja_ja/, … 
help
    FooTransformer.htm, … 
    package_help.csv

package.yml

Package metadata. Required. Its contents are defined later in this document.

README.md

Public-facing package documentation. Required. Shown on FME Hub.

CHANGES.md

Public-facing package changelog. Required.

icon.png

Package icon. Optional. If provided, the image must have a square ratio and have a minimum resolution of 200x200. Higher resolutions are accepted, but be aware and take in consideration that the icon will be displayed in canvas as small as 32x32.

formats

This folder contains non-code components of one or more FME formats(s).

fooformat.fmf

Format metafile. Required.

Its FORMAT_NAME and SOURCE_READER entry must be uppercase and fully qualified in the form publisher_uid.package_uid.FORMAT_KEYWORD. The uppercase requirement is for compatibility reasons around FME Quick Translator.

fooformat.db

formats.db entry for this format. Required.

Comment lines start with a semicolon.

Its FORMAT_NAME must be uppercase and fully qualified in the form publisher_uid.package_uid.FORMAT_KEYWORD.

fooformat.md

Format documentation. Required.

This is a freeform text file in Markdown format.

fooformat.fms

Message definitions. Safe Software internal use only.

barbaz.sch, ...

Fixed schema file(s), if applicable.

barbaz.fme, ...

GUI dependency file(s), if applicable.

*.*

Other files may be present, but are ignored by FME.

Note

All dependency files, such as .sch and .fme, should be in a subfolder. The metafile(s) makes reference to these files by paths relative to the metafile itself.

transformers

This folder contains non-code components of one or more FME transformer(s).

FooTransformer.fmxj

Transformer definition file. Required. Valid extensions are FMX and FMXJ.

The transformer's name in the file must be fully qualified using the form publisher_uid.package_uid.transformer_name. In FMXJ files, this is the info.name property. In FMX files, this is the TRANSFORMER_NAME property.

More details on the syntax of transformer definition files can be found here:

Note

FMXJ transformer support requires FME 2023. Packages that support older versions of FME must define transformers in the FMX format.

FooTransformer.md

Transformer documentation. Required.

This is a freeform text file in Markdown format. It is displayed in the Quick Add pane when the transformer is selected.

FME 2023.1: It should not start with any heading, because FME Workbench automatically adds one that specifies the transformer name and package. Prior to FME 2023.1, this file is only shown in Quick Add when the package is not installed.

FooTransformer.fms

Message definitions. Safe Software internal use only.

barbaz.fme, ...

GUI dependency file(s), if applicable.

Note

All dependency files should be in a subfolder. Transformers make reference to these files using relative paths.

*.*

Other files may be present, but are ignored by FME.

web_filesystems

This folder contains Web Filesystem definitions. Web Filesystem definitions consist of an .fme GUI file, and an optional PNG icon.

The contents of this folder only comprise a small part of the Web Filesystem integration. Web Filesystems would typically also have a Python implementation and a “-Connector” transformer, both of which would be distributed through the corresponding mechanisms in the FME Package.

English Web Filesystem Name.fme: The GUI definition file for the Web Filesystem. See FME_HOME/plugins/webservices/ for examples.
English Web Filesystem Name.png: Icon for the Web Filesystem. Optional.

web_services

This folder contains Web Service definitions. Web Service definitions are XML files. Every file in this folder has the following naming convention:

[English Web Service Name].xml

This naming convention is the same as the files shipped in FME_HOME/WebServices/. However, the contents must be in the importable format, i.e. the result of using the Export option in the FME Workbench Web Services dialog.

The value of <service_name> in the XML must be consistent with the XML filename. For instance, A Web Service definition with <service_name>foo.bar.baz</service_name> must have a filename of baz.xml.

python

This folder contains Python code, distributed as Python Wheels. Typically, this is the Python code that implements formats and transformers in the same FME Package. It may also include third-party dependencies required by those implementations. Every file in this folder is a Python Wheel.

The option to include third-party dependencies allows FME Packages to be installed entirely offline, instead of needing access to the Python Package Index (PyPI).

Examples of files that may be in this folder:

fme_fooformat-1.0.0-py2.py3-none-any.whl: A wheel containing the implementation of an FME format or transformer.
pypac-0.3.0-py2.py3-none-any.whl: Example of a simple third-party dependency.
beautifulsoup4-4.4.1-py2-none-any.whl, beautifulsoup4-4.4.1-py3-none-any.whl: Example of a third-party dependency that has different wheels based on Python version.
foobar-x.y.z-cpXX-cpXXm-*.whl: Binary libraries have a wheel for combination of supported OS, architecture, and Python version. This can be a lot of files.

Info

Wheel files have a naming convention, and are always ZIP archives ending with .whl. FME Package clients need not be concerned about this naming convention.

Python Wheels are the standard for distributing Python libraries, and is defined by PEP-427. Wheels contain binary artifacts only (when applicable), so a compiler is never needed when installing a wheel. Python code in wheels must be .py source files, not .pyc.

Python implementation version vs FME Package version

The version of the Python package that implements the format/transformer does not need to match the FME Package version, but to reduce confusion, the versions should match.

localization

This folder contains localization/translation information for formats and transformers. The contents of this folder are not listed in package.yml.

guiprompts_*.txt

Global guiprompts file. Wildcard is a two-letter language code, e.g. “fr” for French. Its contents follows existing guiprompt convention.

transformer-localized.*

Transformer localization file. Wildcard is a two-letter language code. Its contents follows existing transformer localization convention.

FooTransformer_*.md, fooformat_*.md

Localized format/transformer readme. Wildcard is a two-letter language code.

help_*/

FME 2021.0

The parent folder wildcard is a two-letter language and country code separated by an underscore, i.e. help_fr_ca. The contents of this folder are a localized version of the package's root-level help. There can be multiple help_*/ folders, one for each localization.

help

FME 2021.0

This optional folder contains the self-contained package help for the formats and transformers included in the package. The contents of this folder are not listed in package.yml.

package_help.csv

A 2-column comma separated file mapping help topic keywords to their corresponding HTML page.

Help topic keyword (transformers)	Description
`fmx_publisherUid_packageUid_TransformerName`	Full documentation for `TransformerName`

For transformers, TransformerName matches the case of the transformer name. The help topic prefix is fmx for both FMX and FMXJ transformers.

Help topic keyword (formats)	Description
`rw_publisherUid_packageUid_formatKeyword_index`	Home page
`rw_publisherUid_packageUid_formatKeyword_feature_rep`	Feature Representation
`param_publisherUid_packageUid_formatKeyword_r`	Reader Parameters
`param_publisherUid_packageUid_formatKeyword_w`	Writer Parameters
`ft_publisherUid_packageUid_formatKeyword_param_r`	Reader Feature Type Parameters
`ft_publisherUid_packageUid_formatKeyword_param_w`	Writer Feature Type Parameters
`ft_publisherUid_packageUid_formatKeyword_user_attr`	User Attributes

For formats, formatKeyword is all lowercase.

Documentation paths are relative to the help folder, but must start with /. For instance, a transformer named example.package.DemoGreeter having documentation at help/DemoGreeter.htm, would have an entry like so:

package_help.csv

fmx_example_package_DemoGreeter,/DemoGreeter.htm

*.htm

HTML documentation for formats and transformers.

It may reference the CSS file included with FME Workbench, located at ../../css/style.css, relative to the help folder. This is an alternative to the package including its own stylesheets.

Documentation for transformers must contain:

<h1 class="fmx">TransformerName</h1>
<p><span class="TransformerSummary">A short description of the transformer.</span></p>

FME Workbench extracts the contents of these tags in order to show a preview in the Quick Add pane.

package.yml specification

This is a YAML file that contains metadata about the FME Package, including details about compatibility, and the specific components it installs.

Example package.yml

fpkg_version: 1
name: Packaging Demo
uid: packaging-demo
publisher_uid: publisher-demo
description:
    This is a description of the package.
version: 0.4.0
minimum_fme_build: 23224
author:
    name: Carson Lam
    email: carson.lam@safe.com
package_content:
    formats:
        - name: fooformat
    transformers:
        - name: FooTransformer
          version: 1
    web_services:
        - name: English Service Name.xml
    web_filesystems:
        - name: English Filesystem Name
    python_packages:
        - name: internal_python_lib
        - name: included_3rd_lib

Required values

name: Human-readable name for the package. This is the title used on FME Hub.
uid: Machine-friendly name for the package. This is used in combination with the publisher_uid and the version fields to uniquely identify the package. The uid must contain only lowercase characters (a-z), numbers (0-9) and dashes. It must begin and end with a letter or number, cannot contain two consecutives dashes and must be between 3 and 32 characters in length. The format can be validated by using the following regular expression: ^[a-z0-9](?!.*--)[a-z0-9-]{1,30}[a-z0-9]$
publisher_uid: Machine-friendly name for the publisher of the package. This is used in combination with the uid and the version fields to uniquely identify the package. In order to upload the package to FME Hub, a corresponding FME Hub Publisher must be created. The publisher_uid must contain only lowercase characters (a-z), numbers (0-9), and dashes. It must begin and end with a letter or number, cannot contain two consecutives dashes and must be between 3 and 32 characters in length. The format can be validated by using the following regular expression: ^[a-z0-9](?!.*--)[a-z0-9-]{1,30}[a-z0-9]$
description: Package description. This is used as the summary text on FME Hub.
version: Package version, following Semantic Versioning. This has no relation to the version of any transformer or Python Wheel within the package.
author: Details about the FME Package author. Contains name and email keys. name is required. email is optional.
minimum_fme_build: Minimum FME build number required to install this FME Package. Integer. The client (FME) must not install the package if it does not satisfy this requirement.
fpkg_version: FME Package specification version. Integer. Intended to signal major and breaking changes to the specification. This is version 1. The client (FME) must not install the package if it does not recognize the version.

package_content

This key contains a dictionary that enumerates the various items included in the package that are to be installed. The client (FME) will only install items enumerated here.

Each subkey's value is a list of dictionaries. This design makes it easy to extend the specification to include more metadata for each package_content item.

`formats`

These are the formats included in the package, represented as a list of dictionaries. The keys are:

name: The name (keyword) of the format. Required. This is the name of an FMF file within the package's formats folder. Entries should be lowercase.

`transformers`

These are the transformers included in the package, represented as a list of dictionaries. The keys are:

name: The name of the transformer. Required. This is the name of a transformer definition file in the transformers folder.
version: The version of the transformer. Required. This must be the latest version defined in the transformer definition file. It is copied here for performance reasons.

`web_services`

These are the Web Service definitions included in the package, represented as a list of dictionaries. The keys are:

name: The filename of the Web Service. Required. This is an XML file in the web_services folder. For consistency, this key is not filename.

`web_filesystems`

These are the Web Filesystems included in the package, represented as a list of dictionaries. The keys are:

name: The name of the Web Filesystem. Required. This is the filename, with extension omitted, of an .fme file in the web_filesystems folder.

`python_packages`

These are the Python Wheels to install with the FME Package, represented as a list of dictionaries. Typically, these are the Python Wheels included with the FME Package. At the minimum, it specifies the wheels that contain the implementation for the formats and/or transformers included in the package. The keys are:

name: The name of a Python package. Required. This is the value passed to pip install.

FME uses pip to install these packages. pip is the Python Package Manager, and is included with the FME Python installation.

FME Package installation must work offline

Python Wheels include information about dependencies. By default, pip will resolve and download dependencies from the Python Package Index (PyPI). However, FME Packages are expected to be self-contained and explicitly include its dependencies. Therefore, when FME installs the Python Wheels included in an FME Package, dependency resolution is disabled by invoking pip using the --no-deps and --no-index options.

Compatibility policies

Backwards compatibility with old versions and parameters

When an FME Package is upgraded, its Python packages are also upgraded. However, in workspaces, it is necessary for users to agree to upgrade transformers or regenerate workspaces to pick up changes to transformer/format definition files.

Therefore, the Python code that implements formats and transformers must be backwards-compatible with the parameters from previous versions. In particular:

When a parameter is added, the code should be prepared for it to not have a value, and maintain prior behavior.
When a parameter is removed, the code should still check for its value and maintain prior behavior.
When a parameter type is changed, the code should handle both the previous and new serialized forms of its value.

Transformer definition files must contain the complete version history of that transformer.

Python version compatibility

FME includes a Python interpreter on Windows and macOS. Package authors should take care to write Python code that is compatible with the version of Python included with FME. For broadest compatibility, code should be compatible with the entire range of supported Python versions in the FME release specified by minimum_fme_build in package.yml.

The Python versions supported by each FME is listed in Build Numbers for FME Versions.