You are here: Reading and Writing Data > Separating Output Data with Fanout > Setting Dataset Fanout

Setting Dataset Fanout

Dataset fanout writes a different dataset for each division of data. How this actually appears as output depends on the format.  For example:

  • DWG is a file-based format, and creates new drawings.
  • Esri Shapefile is a folder-based format, and creates new folders.

Note: For an introduction to dataset and feature type fanout, see About Fanout.

Steps

  1. In the Workspace Navigator, double click on the Destination dataset of the writer you want to fanout.

    2. An Edit Dataset dialog opens.

  2. In the Destination Fanout Directory field, specify a directory to hold the fanout datasets.
  3. Check Fanout Dataset and provide a Fanout Expression to indicate how you want to fanout the dataset.

    4. Click the ellipsis to use the Text Editor to assemble an expression of constants, parameters, and output attributes.

    For File-Based Datasets, the expression should evaluate to the full filename of the output files, including any prefixes and file extensions (for example, internal\KML\@Value(PathName).gml or @Value(PathName).dwg).

    For example, you want to perform a dataset fanout in Esri Shapefile format, and three features are output: one has RAIL_TYPE attribute equal to passenger, and two have RAIL_TYPE attribute equal to freight. If Destination Fanout Directory is c:\ and Fanout Expression is RAIL_TYPE (or @Value(RAIL_TYPE) in the Text Editor), you will end up with datasets in two folders: c:\passenger\ and c:\freight\. The output Shapefile names will be based on the output feature types specified in the workspace.

  4. Click OK.

Note: If attribute values have the same name, but the filenames are different letter cases (upper-/lowercase letters), then, in a Windows environment, the previously created file will be overwritten by a subsequent writer on a Windows machine.

For example, say there are two input source filenames and the only difference between them is that one is lowercase (3s1w35.dgn) and one is uppercase (3S1W35.dgn). With dataset fanout, you will get two unique values. Because FME creates two different writers during the translation, the second writer will overwrite the file created by the first. This is because on Windows, filenames are case-preserving but also case-insensitive for various operations. To avoid this, add a StringCaseChanger to the workspace to make sure the attribute names are consistent.

Combining Feature Type and Dataset Fanouts

You can combine Feature Type and Dataset Fanouts.

For example, if you are working with a folder-based format and you perform a feature type fanout on attribute A and a dataset fanout on attribute B, then you'll end up with multiple folders of files, with the filenames dependent on the values of attribute A.

Dataset Fanout and Temporary Disk Storage

Note that a Dataset Fanout can have a huge impact on temporary disk storage, since there is no guarantee that features arrive at the fanout in a single dataset group. Therefore, FME has to write all of the datasets to temporary storage then fan them out afterwards.

Feature Type Fanout has no similar issues, so you may get away with it if your destination format is folder-based.