This section explains how the writer builds the .proto schema from FME features, detailing the metadata that drives the output and the options available for configuration. The focus is exclusively on defining the .proto schema structure, not on the subsequent data writing process.
Quick Links
.proto File Creation Process
|
1. |
Determine whether to generate a new .proto file or use an existing schema. |
If Generate Protobuf Schema is set to Yes, the writer collects incoming features and builds the new .proto file. If an existing schema is supplied, .proto file creation is bypassed. |
|
2. |
Features are scanned to determine the schema. |
The writer reads per-feature attributes and metadata (file options, message/field details, enums, extension ranges, services). Some missing details are filled using safe defaults where supported. When metadata options conflict, the value encountered last will take precedence. |
|
3. |
The writer finalizes and outputs the .proto schema file. |
The writer picks the Protobuf version (Proto2, Proto3, or Editions 2023), unless a specific version is selected, ensures any required file-level options are set, and finalizes field numbering and root handling. The final <package>.proto file is written, containing imports, packages, options, messages (with nesting), enums, and extension ranges. Field options and labels will vary based on the final Protobuf version selected (see Version-Specific Behavior). |
Mapping FME Attributes to Protobuf Types
This section explains the two modes the writer uses to convert FME attributes into Protobuf field definitions.
Without Metadata (Default Type Mapping)
If no specific Protobuf metadata is provided for an FME attribute, the writer defaults to mapping the basic FME attribute type to a standard Protobuf scalar type.
|
FME Type |
Protobuf Type |
|---|---|
| string | string |
| bool | bool |
| int8 | int32 |
| uint8 | uint32 |
| int16 | int32 |
| uint16 | uint32 |
| int32 | int32 |
| uint32 | uint32 |
| real32 | float |
| real64 | double |
| real80 | Not Supported |
| encoded_string | string |
| int64 | int64 |
| uint64 | uint64 |
With Explicit Metadata (Custom Type Definition)
If Protobuf metadata attributes are provided (see Required Attributes for Protobuf Feature Types), the writer uses them to build custom Protobuf structures, including enums, message references, maps, cardinality, defaults, and field options.
If mandatory metadata is missing, the writer issues warnings and downgrades behavior as described under Required Attributes for Protobuf Feature Types.
The Protobuf metadata must be consistent with the metadata attributes generated when the reader parameter Create Field Metadata Attributes is enabled.
Required Attributes for Protobuf Feature Types
If an item in the table is listed as required, it is required only when the attribute is necessary to successfully create that specific Protobuf structure. The writer will otherwise use a simpler default type (for example, a scalar type) rather than fail.
|
Protobuf Structure |
Required FME Attributes |
Details |
|---|---|---|
|
Message Fields |
_[fieldName].type=message _[fieldName].submessage_type_name |
If _[fieldName].type is missing, the default type is string. Example: _MyField.type = message _MyField.submessage_type_name = MyPackage.MyMessage Generated Result: Message MyParent { MyMessage MyField = 1; } |
|
Enum Fields |
_[fieldName].type=enum _[fieldName].enum_value{N} Optional: _[fieldName].enum_type_name |
If _[fieldName].enum_type_name is used by different messages, the definition will be hoisted to the file level to prevent repeated nested definitions. Definition may be hoisted to the file level if multiple parent messages share the same enum. Matching enums are determined by matching enum_type_name and all enum_values{} must be the same. Example: _MyField.type = enum _MyField.enum_type_name = MyEnum _MyField.enum_value{0} = First _MyField.enum_value{1} = Second Generated Result: Message MyParent { MyEnum{ First = 1; Second = 2; } MyEnum MyField = 1; } |
|
Map Fields |
_[fieldName].type=message _[fieldName].is_map=true _[fieldName].submessage_type_name |
Example: Feature containing the map field: _MyField.type = message _MyField.is_map = true _MyField.submessage_type_name = MyMapEntry The following feature called MyMapEntry, containing key/value attributes must exist. See section on typing with and without metadata for how key/value types are resolved. Note the word Entry is stripped from the submessage_type_name. Generated Result: Message MyParent { MyMap<keyType, valueType> MyField = 1; } |
|
Extension Ranges |
_[FeatureType].extension_range{N}.start _[FeatureType].extension_range{N}.end |
Example: _MyParent.extension_range{0}.start = 1000 _MyParent.extension_range{0}.end = 1999 _MyParent.extension_range{1}.start = 3999 _MyParent.extension_range{1}.end = 4999 Generated Result: Message MyParent { extensions 1000 to 1999; extensions 3999 to 4999; } Note
|
|
Field Numbering |
_[fieldName].number |
Example: _MyField.number = 5 Generated Result: Message MyParent { String MyField = 5; } If no field number is present, the writer will auto-assign one. If field numbers within a message conflict, the first-to-use will gain priority, and subsequent conflicts will be assigned the next available field number. Cascading field number conflicts will not occur |
|
Cardinality |
_[fieldName].cardinality |
Options: Required or Optional IMPLICIT is not yet supported. Example: _MyField.cardinality = OPTIONAL _MyField2.cardinality c= REQUIRED Generated Result: Message MyParent { Optional String MyField = 1; Required String MyField2 = 2; } Supported Cardinality Cheatsheet:
|
|
Defaults |
_[fieldName].default |
Example: _MyField.default = “Hello” Generated Result: Message MyParent { String MyField = 1; [Default = “Hello”] } |
Supported Protobuf Metadata Attributes
This section lists the metadata attributes that can be set on FME features to control the generated Protobuf schema.
File / package / imports
|
Used to write the file header (syntax or edition) and package line |
|
|
Language-specific options |
See Protobuf Documentation for usage examples. |
|
Boolean options |
|
|
Optimize option |
Protobuf_optimize_for (SPEED, CODE_SIZE, or LITE_RUNTIME). Note All files that import a file using LITE_RUNTIME must also use LITE_RUNTIME. This will not be resolved automatically by the writer.
|
|
Editions FeatureSet options |
|
Well‑Known Types (WKT)
When a field references a google.protobuf.* type (for example, Timestamp, wrapper types), the writer keeps the fully-qualified name and adds the correct import google/protobuf/… automatically. These are standard imports; they are not redefined in the generated file.
Packages
Use Protobuf_package to set the protobuf file package.
Creating a unique package name enables the following:
- Nested Messages
- Message Grouping by Package
- A .proto file will be created for each package, and all messages belonging to that package will be grouped.
Messages and Nesting
- Messages are created for each feature type and can be nested by interpreting dotted feature type names as parent/child relationships. Nested messages are emitted inside their parents. Protobuf_package must be populated to correctly achieve nested messages.
- Enums first defined inside a message can be “hoisted” to file scope if the same enum type is needed in more than one message (to avoid duplication).
Fields
Name derivation and “list” detection: attribute names are sanitized to valid Protobuf identifiers.
List attributes are recognized and marked as repeated.
Field names must be alphanumeric, with the exception for underscores.
Field names cannot start with an underscore or a number.
Field options supported via metadata include:
- ctype
- jstype
- packed
- deprecated
- debug_redact
- retention
- lazy
- unverified_lazy
- weak
Version-Specific Behavior
How the Version is Chosen
If you set Target Proto Version in the UI, the writer enforces that version when emitting the .proto.
Otherwise in Auto, the writer tries to infer a version from the file-level edition/syntax fields and falls back to Editions 2023 by default.
Editions 2023
Auto-enable presence: if any field requests OPTIONAL cardinality and the writer is targeting Editions 2023, it ensures features.field_presence=EXPLICIT.
Output File and Structure
Filename
- <package>.proto
Header Order
- syntax/edition → imports → package → file options → file-level enums → messages (with nested enums/messages) → per-message extension ranges
Root Handling
If metadata is present, the writer will walk the dependency tree to find a root. A root will be calculated for each Protobuf_package name, in the case of multiple packages (except for Well-Known Type imports) a generated root will always be created:
- If a singular root is found, then it will be used.
- If multiple roots are found, the writer will create a wrapping root (FMEGeneratedRoot) that repeats each top-level root message.
- If no root could be found, the writer will create a wrapping root (FMEGeneratedRoot) that repeats all top-level messages.
If no metadata is present, the writer will create a wrapping root (FMEGeneratedRoot), that repeats all top-level messages, which in this case will be all feature types.