Publishing Items to FME Hub

FME Hub allows you to upload and share custom formats, custom transformers, workspace templates, transformers via Packages, and web connections with the community.

Each item type has a slightly different process for preparation and upload. This guide will walk you through the steps to publish an item to FME Hub.

Creating Content for FME Hub

FME Hub supports the following types of items:

  • Transformer – A custom transformer is a sequence of standard transformers condensed into a single transformer. On FME Hub, this makes it easy to share workflows.
  • Format – A custom format is a workspace that preprocesses data as if it were a data format on its own. Once created, it will show up in the reader formats list and can be used in any FME component as a separate format.
  • Web Service – Web services provide a convenient and secure way to store and reuse connection parameters in workspaces. Each time you need to use the same connection in a workspace, you can use the connection in a workspace without having to re-enter the connection parameters.
  • Workspace Template – Templates allow you to create a workspace using a predesigned format and structure. They are often linked to transformers, custom formats, and web connections to show users how to use the items in real-world scenarios. To create a template, follow the FME Form documentation.
  • Package – The FME Packages SDK Guide and Transformer Designer enable FME users with a developer background to create production-quality transformers using Python. This provides a more powerful alternative to Custom Transformers, allowing for the creation of complex user interfaces and code structures that are not possible or practical with Custom Transformers.
  • FME Flow Project – A Project in FME is a way to export and migrate FME Flow components.

There are several things to consider for all types when uploading to the FME Hub to ensure maximum usability for the community:

  • Use the latest Release Build of FME Form to create content – avoid the beta builds.
  • Use Annotations and Bookmarks – Any FME user who opens your content for inspection should be able to understand the workflow logic without needing to investigate the parameters of every transformer.
  • Do not embed any sensitive data – This includes passwords, hostnames, or keys. Remember that once it’s up there, it’s out there (if you have selected to upload as Public).
  • Versions help keep track of changes and allows for broader backwards compatibility.
  • FME Hub requires valid Categories – These are the same categories (of transformers) that you can see from within FME Workbench.

When creating content for FME Hub, it's important to follow the guidelines specific to each type of content. Below are the guidelines for each type of content you can create for FME Hub.

Custom Transformer/Format

In the context of the FME Hub, Custom Transformers and Formats can be used to share logic with the community. To create a custom transformer, follow the official FME documentation here. To create a custom format, follow the official FME documentation here.

Custom formats are slightly less flexible than custom transformers as you can only use them to read data. You also can't use some of the more advanced custom transformer functions such as looping. To create a custom format you must have one reader in your workspace (most often CSV, Text, XML, JSON) along with a few transformers to restructure the data.

Some best practices include:

  • Using Conditional Visibility of published user parameters – This optional setting is available when creating published user parameters in FME Form, and controls parameter visibility in the workspace runtime parameter prompt. See Working with Conditionally Visible Published User Parameters in FME Form for details.

  • Versioning your content – Custom transformers and Formats can be versioned instead of overwritten when updating them in FME Form. (Note, however, that versioning is tied to the FME Form build version, so if you open a versioned transformer in a newer version of FME and save it, you cannot recover the previous version.)

Note  If you are looking for the location of a custom transformer you have saved locally, by default they are stored in:
  • Windows: C:\Users\<user>\Documents\FME\Transformers
  • macOS: ~/Library/Application Support/FME/Transformers
  • Linux: ~/.fme/Transformers

Web Service

To create a web connection, follow the official FME documentation here.

When you are creating or updating web connections, please consider the following to ensure maximum usability for the community:

  • Create and upload your item using the latest stable release of FME. This is particularly important for web connections, as they are not currently versioned.
  • Ensure you clear the client ID/secret before you upload it to the FME Hub. These are like your password and community users should generate their own.
  • When creating a web connection ensure you set the following:
  • Description (mandatory): The first part of the description should give a short overview of the item as it is shown in the search results and quick type in FME Workbench. In addition to this, the description should also detail how to use the web connection. A good description is shown here . Some potential things to include:
    • Instructions on how to create an application and generate a client ID and client secret.
    • A link to a transformer or template on the FME Hub that details how to use the API for this web connection.
  • Help Links (recommended): These help links show up in the FME Hub pages. You should try and link to the developer documentation that will help the user figure out how to create a client ID/secret and use the API.

Workspace Template

Within the context of FME Hub, templates are often linked to transformers, custom formats, and web connections to show users how to use the items in real-world scenarios.

  • Avoid using Linked Custom Transformers – This is even more important if you do not own the content. If you plan to leverage custom transformers, it’s better to Embed than to Link. Linked custom transformers are vulnerable to version updates, whereas embedded custom transformers remain static.

To create a template, follow the official FME documentation here.

Package

To create a transformer package, follow the FME Packages SDK Guide documentation.

FME Flow Project

To create an FME Flow Project, follow the official FME documentation: FME Flow Web User Interface.

Best Practices

  • Clean your content of any components used for debugging and testing (for example, Logger, Sampler, and Inspector transformers).
  • Resolve any errors with rejected ports in your content (to save users from searching custom transformer definitions to understand issues).

Preparing Items for Upload

The steps below are crucial, to help populate the web page for your item on FME Hub.

Note  We recommend using Markdown to write your Overview and Help sections for proper formatting on the web and FME Workbench help.

Formats and Templates

Before uploading a format or template to FME Hub, you need to edit the Workspace Parameters:

  1. In the Navigator in Workbench, expand the Workspace Parameters and double-click on the Name parameter.

  2. Fill in the following fields:

    • Name (mandatory) – A short descriptive name for your item. This is used in searches and appears as the header on FME Hub.
    • Category (mandatory) – Select a category from the drop-down that matches the categories in FME. The user can filter by category in the FME Hub search.
    • Overview (mandatory) – A summary of your item. This is a brief and helpful description of the functionality.
    • Help (mandatory) – List all parameters (ideally including defaults and optional parameters), and all attributes that might be added to features. Example use cases can go in this section too.
    • History (recommended) – Add a line each time you re-upload an item to FME Hub, detailing what changes were made and who made them.

Transformers

Before uploading a transformer to FME Hub, you need to edit the Transformer Parameters:

  1. In the Navigator in Workbench, expand the Transformer Parameters and double-click on any parameter.

  2. Fill in the following fields:

    • Name (mandatory) – A short descriptive name for your item. This is used in searches and appears as the header on FME Hub.
    • Category (mandatory) – Select a category from the dropdown that matches the categories in FME. The user can filter by category in the FME Hub search.
    • Overview (mandatory) – A summary of your item. This is a brief and helpful description of the functionality. Transformers will be displayed in the Quick Add window. Consider including two headings – one for a high-level description of the custom transformer, and one for basic details about how the custom transformer fits into an FME workspace.
    • Help (mandatory) – List all input and output ports, all parameters (ideally including defaults and optional parameters), and all attributes that might be added to features. Example use cases can go in this section too.
    • History (recommended) – Add a line each time you re-upload an item to FME Hub, detailing what changes were made and who made them.

Testing Workspaces

Test Workspaces are simple workflows designed to test the functionality of content and ensure that it remains compatible with the latest release build of FME (not including betas). These test workspaces will be run automatically on the latest Linux builds of FME and Safe Software will not modify any test workspace uploaded to FME Hub.

The FME Hub testsuite operates on a successful / failed translation logic. If the workspace is successful the test will show green on FME Hub and be marked as success. If the workspace fails, the Hub item will be marked red.

To create a test workspace, add your transformer/format to the workspace and then use a Tester and TestFilter transformers to verify the expected output of a transformer. Connect Terminator transformers to Failed output ports to ensure the workspace will fail accordingly.

Try to use Linked Custom Transformers – This ensures you will not have to always re-upload a new Test Workspace. FME Hub’s Testsuite will always grab the latest version. If you have custom transformers within the custom transformer you are uploading to the FME Hub, ensure these transformers are embedded.

Additional Information

  • Pay attention to the version of FME running on the Testsuite. If you authored your custom transformer in a newer version of FME, your test workspace will fail.
  • Please note that these workspaces are only visible to the creator.
  • Templates (.fmwt) can also be used as a test workspace – this is helpful if you need to include data with your FME Hub content.
  • Do not use WorkspaceRunner or FMEServerJobSubmitter transformers.
  • Only one test workspace can be uploaded per item.
  • Content that requires additional files (for example, Python libraries) is currently not supported.

Uploading a New Item to FME Hub

After you have created your item and populated the metadata, you are ready to upload it to the FME Hub. The steps to upload a new item are:

  1. Sign Up: You need an account to upload items. Sign up and create a free FME Hub account.
  2. Browse Items: Once you are signed in, click the Contribute button in the top right corner. You may need to create a Publisher first.
  3. Publisher Page: On your Publisher, click Upload Item to start uploading a new item.
  4. Upload Item: Drag the item you wish to upload onto the drop panel, or choose the file by clicking the Select File button.
  5. Add Icon: (Optional) Add an icon for the item by dragging an image file onto the drop panel, or choose one by clicking the Browse file system button.
    6. Note  The icon and other fields may already be set based on the item type. For example, fields for Projects are set on Flow itself.
  1. Test Workspace: Test workspaces ensure that custom transformers and other content submitted to FME Hub are tested on a regular basis for compatibility with official releases of FME.

  2. Visibility: Public items will be visible to everyone on FME Hub and in FME Form. Private items are only available to members of your Publisher.

  3. Publish: Once you have uploaded all three parts and your test suite runs successfully, you can click the Publish button to publish your item.

Editing an Existing FME Hub Item

After you have uploaded an item to the FME Hub, you can re-upload the item at any time.

Note  To edit an item that you have uploaded, log in to FME Hub, navigate to the item you wish to update, and click the Edit button. On this page, you can edit the icon, test workspace, whether it is public or private, and also re-upload the item itself. To publish your changes click Publish Changes.

Alternatively, you can edit items by locating the item in the Publisher.

Setting Items to Private or Public

To make a package Private or Public, click the lock/unlock button on the Publisher Page beside the package name:

 

Changing Visibility from Public to Private

Clicking the lock button will open a modal dialog that provides more information on the implication of making a package private. Before the Change to Private button is enabled, you will have to acknowledge that the changes will affect the selected package.

For other item types, like Transformers, the process is similar. Click the Actions button, then select Make Item Private.

Note  
  • Private items will only be visible to members of the publisher.
  • Private items will not be accessible to download directly from Form.

When you change a public item to private, the following will also happen:

  • The items will be removed from the list of anyone who had previously “starred” it.
  • The star count will be reset to 0.

Changing Visibility from Private to Public

Click the Actions button and select Make Item Public.

This opens a modal dialog that provides more information on the implication of making a package public.