Publishing Items to FME Hub

FME Hub allows you to upload and share custom formats, custom transformers, workspace templates, transformers via Packages, and web connections with the community.

Each item type has a slightly different process for preparation and upload. This guide will walk you through the steps to publish an item to FME Hub.

Creating Content for FME Hub

FME Hub supports the following types of items:

  • Custom Transformers: A custom transformer is a sequence of standard transformers condensed into a single transformer. On FME Hub, this makes it easy to share workflows.
  • Transformers via Packages: The FME Packages SDK Guide and Transformer Designer enable FME users with a developer background to create production-quality transformers using Python. This provides a more powerful alternative to Custom Transformers, allowing for the creation of complex user interfaces and code structures that are not possible or practical with Custom Transformers.
  • Custom Formats: Custom formats are workspaces that preprocess data as if it were a data format on its own. Once created, it will show up in the reader formats list and can be used in any FME component as a separate format.
  • Workspace Templates: Templates allow you to create a workspace using a predesigned format and structure. They are often linked to transformers, custom formats, and web connections to show users how to use the items in real-world scenarios. To create a template, follow the FME Form documentation.
  • Web Services: Web services provide a convenient and secure way to establish connection parameters in workspaces. Each time you need to use the same connection in a workspace, you can use the stored connection without having to re-enter the parameters.
  • FME Flow Projects: A Project in FME is a way to export and migrate FME Flow components.

There are several things to consider for all types when uploading to the FME Hub to ensure maximum usability for the community:

  • Use the latest Release Build of FME Form to create content – avoid the beta builds.
  • Use Annotations and Bookmarks – Any FME user who opens your content for inspection should be able to understand the workflow logic without needing to investigate the parameters of every transformer.
  • Do not embed any sensitive data – This includes passwords, hostnames, or keys. Remember that once it’s up there, it’s out there (if you have selected to upload as Public).
  • Versions help keep track of changes and allows for broader backwards compatibility.
  • FME Hub requires valid Categories – These are the same categories (of transformers) that you can see from within FME Workbench.

When creating content for FME Hub, it's important to follow the guidelines specific to each type of content. Below are the guidelines for each type of content you can create for FME Hub.

Custom Transformers/Formats

In the context of the FME Hub, Custom Transformers and Formats can be used to share logic with the community. To create a custom transformer, follow the official FME documentation here. To create a custom format, follow the official FME documentation here.

Custom formats are slightly less flexible than custom transformers as you can only use them to read data. You also can't use some of the more advanced custom transformer functions such as looping. To create a custom format you must have one reader in your workspace (most often CSV, Text, XML, JSON) along with a few transformers to restructure the data.

Some best practises include:

  • Using Conditional Visibility of published user parameters – This optional setting is available when creating published user parameters in FME Form, and controls parameter visibility in the workspace runtime parameter prompt. See Working with Conditionally Visible Published User Parameters in FME Form for details.

  • Versioning your content – Custom transformers and Formats can be versioned instead of overwritten when updating them in FME Form. (Note, however, that versioning is tied to the FME Form build version, so if you open a versioned transformer in a newer version of FME and save it, you cannot recover the previous version.)

Note  If you are looking for the location of a custom transformer you have saved locally, by default they are stored in:
  • Windows: C:\Users\<user>\Documents\FME\Transformers
  • macOS: ~/Library/Application Support/FME/Transformers
  • Linux: ~/.fme/Transformers

Transformers via Packages

To create a transformer via packages, follow the FME Packages SDK Guide documentation.

Workspace Templates

Within the context of FME Hub, templates are often linked to transformers, custom formats, and web connections to show users how to use the items in real-world scenarios.

  • Avoid using Linked Custom Transformers – This is even more important if you do not own the content. If you plan to leverage custom transformers, it’s better to Embed than to Link. Linked custom transformers are vulnerable to version updates, whereas embedded custom transformers remain static.

To create a template, follow the official FME documentation here.

Web Connections

To create a web connection, follow the official FME documentation here.

When you are creating or updating web connections, please consider the following to ensure maximum usability for the community:

  • Create and upload your item using the latest stable release of FME. This is particularly important for web connections, as they are not currently versioned.
  • Ensure you clear the client ID/secret before you upload it to the FME Hub. These are like your password and community users should generate their own.
  • When creating a web connection ensure you set the following:
  • Description (mandatory): The first part of the description should give a short overview of the item as it is shown in the search results and quick type in FME Workbench. In addition to this, the description should also detail how to use the web connection. A good description is shown here . Some potential things to include:
    • Instructions on how to create an application and generate a client ID and client secret.
    • A link to a transformer or template on the FME Hub that details how to use the API for this web connection.
  • Help Links (recommended): These help links show up in the FME Hub pages. You should try and link to the developer documentation that will help the user figure out how to create a client ID/secret and use the API.

FME Flow Projects

To create an FME Flow Project, follow the official FME documentation: FME Flow Web User Interface.

Best Practices

  • Clean your content of any components used for debugging and testing (for example, Logger, Sampler, and Inspector transformers).
  • Resolve any errors with rejected ports in your content (to save users from searching custom transformer definitions to understand issues).

Preparing Items for Upload

The steps below are crucial, to help populate the web page for your item on FME Hub.

Note  We recommend using Markdown to write your Overview and Help sections for proper formatting on the web and FME Workbench help.

Formats and Templates

Before uploading a format or template to FME Hub, you need to edit the Workspace Parameters:

  1. In the Navigator in Workbench, expand the Workspace Parameters and double-click on the Name parameter.

  2. Fill in the following fields:

    • Name (mandatory): A short descriptive name for your item. This is used in searches and appears as the header on FME Hub.
    • Category (mandatory): Select a category from the drop-down that matches the categories in FME. The user can filter by category in the FME Hub search.
    • Overview (mandatory): A summary of your item. This is a brief and helpful description of the functionality.
    • Help (mandatory): List all parameters (ideally including defaults and optional parameters), and all attributes that might be added to features. Example use cases can go in this section too.
    • History (recommended): Add a line each time you re-upload an item to FME Hub, detailing what changes were made and who made them.

Transformers

Before uploading a transformer to FME Hub, you need to edit the Transformer Parameters:

  1. In the Navigator in Workbench, expand the Transformer Parameters and double-click on any parameter.

  2. Fill in the following fields:

    • Name (mandatory): A short descriptive name for your item. This is used in searches and appears as the header on FME Hub.
    • Category (mandatory): Select a category from the dropdown that matches the categories in FME. The user can filter by category in the FME Hub search.
    • Overview (mandatory): A summary of your item. This is a brief and helpful description of the functionality. Transformers will be displayed in the Quick Add window. Consider including two headings – one for a high-level description of the custom transformer, and one for basic details about how the custom transformer fits into an FME workspace.
    • Help (mandatory): List all input and output ports, all parameters (ideally including defaults and optional parameters), and all attributes that might be added to features. Example use cases can go in this section too.
    • History (recommended): Add a line each time you re-upload an item to FME Hub, detailing what changes were made and who made them.

Testing Workspaces

Test Workspaces are simple workflows designed to test the functionality of content and ensure that it remains compatible with the latest release build of FME (not including betas). These test workspaces will be run automatically on the latest Linux builds of FME and Safe Software will not modify any test workspace uploaded to FME Hub.

The FME Hub testsuite operates on a successful / failed translation logic. If the workspace is successful the test will show green on FME Hub and be marked as success. If the workspace fails, the Hub item will be marked red.

To create a test workspace, add your transformer/format to the workspace and then use a Tester and TestFilter transformers to verify the expected output of a transformer. Connect Terminator transformers to Failed output ports to ensure the workspace will fail accordingly.

Try to use Linked Custom Transformers – This ensures you will not have to always re-upload a new Test Workspace. FME Hub’s Testsuite will always grab the latest version. If you have custom transformers within the custom transformer you are uploading to the FME Hub, ensure these transformers are embedded.

Additional Information

  • Pay attention to the version of FME running on the Testsuite. If you authored your custom transformer in a newer version of FME, your test workspace will fail.
  • Please note that these workspaces are only visible to the creator.
  • Templates (.fmwt) can also be used as a test workspace – this is helpful if you need to include data with your FME Hub content.
  • Do not use WorkspaceRunner or FMEServerJobSubmitter transformers.
  • Only one test workspace can be uploaded per item.
  • Content that requires additional files (for example, Python libraries) is currently not supported.

Uploading a New Item to FME Hub

After you have created your item and populated the metadata, you are ready to upload it to the FME Hub. The steps to upload a new item are:

  1. Sign Up: You need an account to upload items. Sign up and create a free FME Hub account.
  2. Launch Upload Page: Once you are signed in, click on the Contribute button in the top right corner. You may need to create a Publisher first.
  3. Upload Item: Drag the item you wish to upload onto the drop panel or choose the file by clicking the Select File button.
  4. Add Icon: (Optional) Add an icon for the item by dragging an image file onto the drop panel or choose one by clicking the Select File button.
  5. Test Workspace: Test workspaces ensure that custom transformers and other content submitted to FME Hub are tested on a regular basis for compatibility with official releases of FME.
  6. Publish: Once you have uploaded all three parts and your test suite runs successfully, you can click the Publish button to publish your item.

Editing an Existing FME Hub Item

After you have uploaded an item to the FME Hub, you can re-upload the item at any time.

Note  To edit an item that you have uploaded, log in to FME Hub, navigate to the item you wish to update, and click the Edit button. On this page, you can edit the icon, test workspace, whether it is public or private, and also re-upload the item itself. To publish your changes click Publish Changes.

Alternatively, you can edit items by locating the item in the Publisher.

Setting Items to Private or Public

To make a package Private or Public, click the lock/unlock button beside the package name:

 

Changing Visibility from Public to Private

Clicking the lock button will open a modal dialog that provides more information on the implication of making a package private. You will have to type the package name in the confirmation field to initiate the change.

For other item types, like Transformers, the process is similar. Click the edit button, then select Make private.

Note  
  • Private items will only be visible to members of the publisher.
  • Private items will not be accessible to download directly from Form.

When you change a public item to private, the following will also happen:

  • The items will be removed from the list of anyone who had previously “starred” it.
  • The star count will be reset to 0.

Changing Visibility from Private to Public

Clicking the unlock button will open a modal dialog that provides more information on the implication of making a package public. You will have to type the package name in the confirmation field to initiate the change.