FME Transformers: 2024.1
Extracts portions of JSON (JavaScript Object Notation) formatted text into new FME features.
JSON Queries
A JSON query is a simple mechanism to refer to values within a JSON document. A query is made up of one or more expressions, which are separated by a + operator. There are three types of expressions: JSON structure expressions, JSON property expressions and string literal expressions.
JSON Structure Expressions
A JSON structure expression can refer to values in a JSON document. The outermost JSON element, which must be an array or an object, is always referred to by the term json, and this term must appear at the beginning of every JSON structure expression. The child elements can be referred to using JavaScript-like square bracket index operators. For example, if the outermost element is an array, the first element of the array can be referred to by the expression json[0], the second element can be referred to by the expression json[1], and so on. Likewise, if the outermost JSON element is an object, with keys "name" and "address", then the values of these keys can be referred to by the expressions json["name"] and json["address"] respectively.
These index operators can be cascaded. For example, it the outermost JSON element is an object with a key and "address" whose value is an object containing keys "city" and "province", then these values can be referred to by the expressions json["address"]["city"] and json["address"]["province"].
In order to refer to all of the values in an array or object, a wildcard index * can be used. For example, if the outermost JSON element is an array, then the expression json[*] refers to every element in the array.
JSON Property Expressions
A property expression is a structure expression as described above, followed by a . (dot) operator and a property name. Currently, the only supported properties are type and size. The type property returns the type of the JSON value referred to by the JSON structure expression. For example, if the outermost JSON element is an array, and the first element of the array is a string, then the expression json[0].type would have a value of string. The size property, which can only be applied to an array, returns the number of elements in the array.
String Literal Expressions
A string literal expression is simply a quoted string value, such as "this is a string literal expression".
Configuration
Output Ports
Fragments
For each newly created feature, the attribute identified by the Result Attribute parameter will contain JSON text referred to by the JSON query.
The json_type attribute will contain the JSON type of the text (object, array, string, etc) and the json_index parameter will contain the object key or array index of the JSON value.
If the JSON query cannot be fully evaluated, a message will be logged, and the translation will continue.
Rejected
If Reject Features which Produce No Fragments is set to Yes, features that produce no fragments will be output through this port.
Features that could not be successfully processed will also be output through this port. Typically this happens when the attribute specified in the JSON Attribute parameter does not have a value, or has a value that is not valid JSON.
Parameters
Source
|
Input Source |
Allows for selecting between one of two options for JSON input: JSON Document or File/URL |
|
JSON Document |
If selected, The JSON document can come from any string expression, including attribute values and workspace parameters. |
|
File/URL |
If selected, the JSON source can be read from a local file at a specified file path, or from the internet at a specified URL. |
General
|
JSON Query |
The JSON Query parameter is the JSON query referring to the JSON values which will become new FME features. This parameter can be typed directly in the text box or, if reading JSON from a file or URL, by clicking the browse button that opens a dialog containing the structure of the JSON dataset in a tree form. Selecting a value from the tree will populate the parameter with the appropriate query string to turn the selected values into features. A JSON query is a mechanism to refer to values inside JSON text. The outermost JSON value, which must be an object or an array, is always referred to by the term "json". Contained values can be referred to using JavaScript-like square bracket index operators. A value in an array can be referred to using its zero-based position in the array (for example, json[2] for the third element). A value in an object can be referred to using its object key name. For example: json["key"] All of the values in an array or object can be referred to collectively using a wildcard index. For example: json[*] The query used by this transformer can have multiple expressions, separated by a '+' operator. This allows the transformer to refer to values in different areas of the JSON text. For example: json["resultSet_1"][*] + json["resultSet_2"][*] See below for more information on JSON queries. |
|
Fragment as Format |
If the JSON text is in GeoJSON or EsriJSON format, you can select Fragment as Format to GEOJSON or ESRIJSON. This will inform JSONQueryFactory that GeoJSON or ESRIJSON features need to be automatically recognized and extracted from the query results. The default format, JSON, parses the JSON text as plain JSON. If no GeoJSON or ESRIJSON features were constructed, a warning will be issued and the text will be treated as plain JSON. |
|
Reject Features which Produce No Fragments |
If this parameter is Yes, features which produce no fragments will be output through the <Rejected> port. |
|
Result Attribute |
If the JSONFragmenter is set to read JSON from a feature attribute, then the output is automatically written to the input attribute. If, however, the JSONFragmenter is set to read JSON from a file or URL, then an output attribute must be selected. This is done in the Result Attribute field which is activated when Input Source is set to File/URL. The output attribute may be selected from existing input attributes, user-defined parameters, or a new attribute may also be created. |
Flattening
|
Flatten Query Result into Attributes |
If the JSON query produces a JSON Object, the keys may be used to produce additional attributes for the new feature(s) by setting to Yes. If the JSON query produces a JSON array, and a value is provided to the Prefix New Attributes With parameter, a list attribute will be created, using the prefix as the list name, and the JSON array values as the list elements. Note List attributes are not accessible from the output schema in FME Workbench unless they are first processed using a transformer that operates on them, such as ListExploder or ListConcatenator. Alternatively, AttributeExposer can be used.
|
|
Recursively Flatten Objects/Arrays |
When the Flatten Query Result into Attributes parameter is set to Yes, this parameter controls the attributes which are created. If it is set to No, each JSON object key will become an attribute and the corresponding JSON value will become the attribute value. If this parameter is set to Yes, only object values which are simple types (integers, strings, etc) will become attribute values, while nested objects and arrays will be recursively flattened. |
|
Prefix New Attributes With |
If the Flatten Query Result into Attributes parameter is set to Yes, the Prefix New Attributes With parameter may be used to provide a string with which to prefix the newly-created attributes. If the selected format is GeoJSON or ESRIJSON, the format-specific keys will not be added as attributes, but will instead be handled by the appropriate format parser. |
|
Attributes to Expose |
Exposes attributes so they can be used elsewhere in the workspace. Attribute names may be entered directly or provided in the Enter Values for Attributes to Expose dialog accessed via the ellipsis button, where data type may also be specified. For more information on exposed and unexposed attributes, see Understanding Feature Types and Attributes. |
Editing Transformer Parameters
Transformer parameters can be set by directly entering values, using expressions, or referencing other elements in the workspace such as attribute values or user parameters. Various editors and context menus are available to assist. To see what is available, click 
beside the applicable parameter.
How to Set Parameter Values
Defining Values
There are several ways to define a value for use in a Transformer. The simplest is to simply type in a value or string, which can include functions of various types such as attribute references, math and string functions, and workspace parameters.
Using the Text Editor
The Text Editor provides a convenient way to construct text strings (including regular expressions) from various data sources, such as attributes, parameters, and constants, where the result is used directly inside a parameter.
Using the Arithmetic Editor
The Arithmetic Editor provides a convenient way to construct math expressions from various data sources, such as attributes, parameters, and feature functions, where the result is used directly inside a parameter.
Conditional Values
Set values depending on one or more test conditions that either pass or fail.
Parameter Condition Definition Dialog
Content
Expressions and strings can include a number of functions, characters, parameters, and more.
When setting values - whether entered directly in a parameter or constructed using one of the editors - strings and expressions containing String, Math, Date/Time or FME Feature Functions will have those functions evaluated. Therefore, the names of these functions (in the form @<function_name>) should not be used as literal string values.
Content Types
| These functions manipulate and format strings. | |
|
Special Characters |
A set of control characters is available in the Text Editor. |
| Math functions are available in both editors. | |
| Date/Time Functions | Date and time functions are available in the Text Editor. |
| These operators are available in the Arithmetic Editor. | |
| These return primarily feature-specific values. | |
| FME and workspace-specific parameters may be used. | |
| Creating and Modifying User Parameters | Create your own editable parameters. |
Dialog Options - Tables
Table Tools
Transformers with table-style parameters have additional tools for populating and manipulating values.
|
Row Reordering
|
Enabled once you have clicked on a row item. Choices include:
|
|
Cut, Copy, and Paste
|
Enabled once you have clicked on a row item. Choices include:
Cut, copy, and paste may be used within a transformer, or between transformers. |
|
Filter
|
Start typing a string, and the matrix will only display rows matching those characters. Searches all columns. This only affects the display of attributes within the transformer - it does not alter which attributes are output. |
|
Import
|
Import populates the table with a set of new attributes read from a dataset. Specific application varies between transformers. |
|
Reset/Refresh
|
Generally resets the table to its initial state, and may provide additional options to remove invalid entries. Behavior varies between transformers. |
Note: Not all tools are available in all transformers.
For more information, see Transformer Parameter Menu Options.
FME Community
The FME Community is the place for demos, how-tos, articles, FAQs, and more. Get answers to your questions, learn from other users, and suggest, vote, and comment on new features.
Search for all results about the JSONFragmenter on the FME Community.
Keywords: JSONExploder