Protocol Buffers (Protobuf) Reader Parameters

Schema Options

This parameter determines whether or not the reader should use a .proto file to decode and map the binary file:

  • Yes (default) – This option forces the reader to use a .proto schema.
  • No – The reader will decode the binary into readable text. This is similar to the --decode_raw option when using the Protobuf Compiler (Protoc).

This parameter is the file path to the .proto schema. Imports are recursively resolved and included in the Feature Type output.

Note  google/protobuf/go_features.proto and google/protobuf/java_features.proto are not currently supported. Use the Allow Incomplete Proto Schemas option to continue, or use an absolute path to these .proto files. All other Protobuf Well-Known Types are supported.

This parameter asks for the message type to read. This is the same as --decode=MESSAGE_TYPE when using the Protobuf Compiler (Protoc).

By default, the .proto schema parser expects a complete proto definition.

This option will suppress failures regarding missing Message definitions. Missing message definitions will be decoded raw into the _Unknown feature type.

Feature Type Options

  • All Messages (default) – With this option, every message becomes its own feature type. Parent and child messages remain distinct but are connected by matching protobuf_id values. No flattening occurs.
  • Specific Messages – This option gives you control over which messages remain separate and which are flattened. Click the browse button to open a new Selected Messages checklist, which lists all available messages.
    • Check the box next to any message you want to keep as an individual feature. Unchecked messages will be automatically flattened and merged with their parents. All selected messages – parent and child - are converted to features and will maintain their original structural relationship via matching protobuf_id.
    • Leave a message unchecked to have it flattened into its parent, merging its data directly into the parent feature.
Note  For optimal performance, limit the number of feature types that are left unchecked for flattening. Flattening too many may slow down processing, especially in large lists.
Note  To use this option, select Specific Messages in the parameter Create Feature Types For.

For optimal performance, limit the number of feature types that are left unchecked for flattening. Flattening too many may slow down processing, especially in large lists.

Settings

This parameter will produce additional feature attributes related to the .proto file.

Attributes prefixed with Protobuf_ are Protobuf File-level options; attributes prefixed with an underscore are Message, Field, or Service-related options.

Note  Custom options are not yet supported.

When this option is checked, attributes are generated for any unknown fields in the Protobuf data. These fields are present in the source data but are not defined in the .proto schema.

This parameter allows for the preservation and inspection of additional data that would otherwise be discarded.

Schema Attributes

Use this parameter to expose Format Attributes in FME Workbench when you create a workspace:

  • In a dynamic scenario, it means these attributes can be passed to the output dataset at runtime.
  • In a non-dynamic scenario, this parameter allows you to expose additional attributes on multiple feature types. Click the browse button to view the available format attributes (which are different for each format) for the reader.