List of Transformers

You are here: Web > JSONUpdater

JSONUpdater

Modifies JSON documents by performing one or more actions:

  • Inserting a new value into a JSON object or array
  • Replacing an existing value in a JSON object or array
  • Merging the contents of two JSON objects or arrays
  • Removing a value from a JSON object or array
  • Appending a value to a JSON array
  • Renaming a key in a JSON object.

For each document update, a JSON path expression is used to identify the location of the JSON object or array which is to be modified. A second parameter names the specific object key or array index which is to be changed. Note that the path may refer to more than one item. If this is the case, all the items will be modified.

The JSON path expressions used in this transformer are the same as those used in the JSONExtractor and JSONFragmenter transformers, and in the JSON reader.

For modifications that replace existing items or insert new items, two parameters are given which specify the value to be added to the document. This value may be given as a simple text value, or it may be an XQuery expression, a JSON object or a JSON array.

The Update features provide the attribute values used in the document update operations. The Document feature is only used to supply the JSON document which is to be updated.

For each feature that enters the Update port, all of the update operations listed in the transformer will be applied to each feature that entered the transformer through the Document port. Therefore, no changes will be made to the JSON document if no features enter the transformer through the Update port.

Note that if the document is entered directly into the transformer parameters, it may be processed in two different ways:

  • If no Document features are given to the transformer, a new feature will be created to store the document. Each of the updates will be applied to the document, and it will then be output through the Document port.
  • If Document features are given to the transformer, each of the updates will be applied to the document, and these features will be output from the transformer.

Input Ports

ClosedDocument

Input features contain the JSON documents to be modified. The documents can be contained in an attribute, entered directly into the transformer, or contained in a file referenced by an attribute value.

ClosedUpdate

For each feature which enters this input port, each of the updates listed in the transformer will be executed on each document which entered via the Document input port. These features will contain the attributes values which are used in the document updates.

Output Ports

ClosedDocument

The Document input features will be output through this port. These features will contain the updated document, which will be stored in the attribute named in the Result Attribute parameter.

Parameters

Transformer

ClosedGroup Update Features By

This parameter is used to specify which update features will modify a document. When this parameter is set to a list of attributes, Document features will only be modified by Update features that share the same values for these attributes. The document will be modified by all of the update features with the same attribute values. Note that a document will not be updated if no Update features with the same values for the “group by” attributes are provided to the transformer.

ClosedParallel Processing

Note: How parallel processing works with FME: see About Parallel Processing for detailed information.

This parameter determines whether or not the transformer should perform the work across parallel processes. If it is enabled, a process will be launched for each group specified by the Group By parameter.

Parallel Processing Levels

Parameter Number of Processes
No Parallelism 1
Minimal coresThe processor, or CPU, is the physical part of the computer that performs mathematical calculations. It is the most important part of a computer system. Traditional processors have only one core on the processor, meaning that at any given time, only one set of calculations is being performed. If a processor is dual-core, this means the single chip contains hardware for two processors, now called cores to distinguish them from the single chip, running simultaneously, side by side. (Source: http://www.ehow.com/facts_5730257_computer-core-processors_.html) / 2
Moderate exact number of cores
Aggressive cores x 1.5
Extreme cores x 2

For example, on a quad-core machine, minimal parallelism will result in two simultaneous FME processes. Extreme parallelism on an 8-core machine would result in 16 simultaneous processes.

You can experiment with this feature and view the information in the Windows Task Manager and the Workbench Log window.

ClosedInput Ordered

No: This is the default behavior. Processing will only occur in this transformer once all input is present.

By Group: This transformer will process input groups in order. Changes of the value of the Group By parameter on the input stream will trigger batch processing on the currently accumulating group. This will improve overall speed if groups are large/complex, but could cause undesired behavior if input groups are not truly ordered.

ClosedConsiderations for Using Input is Ordered By

Using Ordered input can provide performance gains in some scenarios, however, it is not always preferable, or even possible. Consider the following when using it, with both one- and two-input transformers.

Single Datasets/Feature Types: Are generally the optimal candidates for Ordered processing. If you know that the dataset is correctly ordered by the Group By attribute, using Input is Ordered By can improve performance, depending on the size and complexity of the data.

If the input is coming from a database, using ORDER BY in a SQL statement to have the database pre-order the data can be an extremely effective way to improve performance. Consider using a Database Readers with a SQL statement, or the SQLCreator transformer.

Multiple Datasets/Feature Types: Since all features matching a Group By value need to arrive before any features (of any feature type or dataset) belonging to the next group, using Ordering with multiple feature types is more complicated than processing a single feature type.

Multiple feature types and features from multiple datasets will not generally naturally occur in the correct order.

One approach is to send all features through a Sorter, sorting on the expected Group By attribute. The Sorter is a feature-holding transformer, collecting all input features, performing the sort, and then releasing them all. They can then be sent through an appropriate filter (TestFilter, AttributeFilter, GeometryFilter, or others), which are not feature-holding, and will release the features one at a time to the transformer using Input is Ordered By, now in the expected order.

The processing overhead of sorting and filtering may negate the performance gains you will get from using Input is Ordered By. In this case, using Group By without using Input is Ordered By may be the equivalent and simpler approach.

In all cases when using Input is Ordered By, if you are not sure that the incoming features are properly ordered, they should be sorted (if a single feature type), or sorted and then filtered (for more than one feature or geometry type).

As with many scenarios, testing different approaches in your workspace with your data is the only definitive way to identify performance gains.

JSON Document

ClosedJSON Input

This parameter is used to specify the type of JSON document which is to be modified. The JSON may come from the input features (Parameter Value: Text or Attribute), or it may come from an external file. (Parameter Value: JSON File)

ClosedJSON Text

When the JSON Input parameter is set to ‘Text or Attribute’, this parameter is used to specify the JSON text. The parameter may be set to an attribute value, or the JSON document may be entered into the parameter directly.

ClosedJSON File

When the JSON Input parameter is set to ‘JSON File’, this parameter is used to specify the location of the file. The file path may be entered directly into the parameter, or the path may come from an attribute value.

Updates

Each row in the Updates table specifies a single update operation.

ClosedUpdate Type

This parameter specifies the type of modification to apply to the XML document. The possible updates are:

Update Type Description

Insert Into Object

Inserts a new key / value pair into an object. The name of the key is taken from the ‘Object/Array Index’ column. The inserted value is taken from the Value parameter.

Merge Objects

Copies all the key / value pairs from one object into another. If the Value parameter is not a JSON object, or an XQuery expression which evaluates to a object, the update will fail.

Insert Into Array

Inserts a new value into an array, at a specified position. The array position (which starts counting at 0) is taken from the ‘Object/Array Index’ column, and should be an integer. The new value is taken from the Value column.

Append to Array

Appends a new value to the end of an array.

Merge Arrays

Copies all values from one array into another. If the Value parameter is not a JSON array, or an XQuery expression which evaluates to an array, the update will fail.

Replace Object Value

Replaces the value of an object key. The key whose value is to be replaced is taken from the ‘Object/Array Index’ parameter.

Replace Array Value

Replaces the value at a specified position within an array. The array index of the value to replace is taken from the ‘Object/Array Index’ column. The index should be an integer. Array indices being counting at 0.

Rename Object Key

Changes an object key name. The key to rename is taken from the ‘Object/Array Index’ parameter.

Delete from Object

Removes a key / value pair from an object. The key which is to be removed is taken from the ‘Object/Array Index’ parameter.

Delete from Array

Removes a value from an array. The array index of the value to remove is taken from the ‘Object/Array Index’ column. The index should be an integer. Array indices being counting at 0.
ClosedJSON Path

This parameter is used to identify the JSON object or array which is to be updated. For updates which modify a specific object key or array index, the key / index is identified using the ‘Object/Array Index’ parameter.

JSON paths use a simple text syntax to trace a route through a JSON document, from the top level object or array, down to the object or array which is being modified.

All JSON documents must have a single object or array as the root value. In a JSON path, this root value is identified by the ‘json’ keyword. Thus, every JSON path begins with this keyword. Square bracket syntax is used to identify each level of the path.

An integer can be used to identify a position within an array:

  • json[0] – If the root value is an array, this path refers to the first value in the array. If the root value is an object, this path will not resolve to any value.
  • json[4] – If the root value is an array, this path refers to the fifth value in the array.

A quoted string can be used to refer to the value of an object key:

  • json["location"] – If the root value is an object, this path refers to the value of the ‘location’ key in the object. If the root value is an array, this path will not resolve to any value.
  • json["features"] – Refers to the value of the ‘features’ key in the root object.

These indices may be repeated, to have the path go deeper into the JSON document. It is important to know the structure of the document, so that the types of indices (object or array) are correct.

  • json[0]["location"] – This path refers to the ‘location’ object key in the first element in the root array.
  • json["features"][0][ "location"] – This path refers to the value of the ‘location key in the first item in the ‘features’ array of the root object. For example, this could apply to the following (incomplete) document:

    {

    "features" : [

    { "location” : { ... } }

    ]

    }

A wildcard may be used to refer to all elements of an array, or all values in an object. A * character is used to denote a wildcard.

  • json[*] – If the root value is an array, this path refers to all values in the array. If the root value is an object, this path refers to the values of all keys in the object.
  • json[*]["location"] – This path refers to the ‘location’ key of all child values of the root object or array.
  • json["features"][*] – This path refers to all the values contained in the value of the ‘features’ key in the root object.

In order to use an attribute value in a JSON path, the @Value FME function should be used. If the attribute value is an object key, the @Value function should be enclosed in quotes.

  • json[@Value(index)] – This path refers to the item in the root array at the position indicated by the ‘index’ attribute.
  • json["@Value(key)"] – This path refers to the object value whose key is named in the ‘key’ attribute.

Note: The @Value() functions used in the path parameter will apply to the Update feature being processed, not the Document feature that is being updated

ClosedObject/Array Index

This parameter is used to specify the specific object key / array index whose value is to be inserted/removed/modified.

For object updates, this value of this parameter should be a string, which will be interpreted as an object key. Quotes are NOT required around the string in this parameter.

For array updates, the value of this parameter should be an integer. Note that array indices start at 0.

ClosedValue Type

This parameter is used to indicate how the value of the Value parameter should be interpreted. The effects of the different values of this parameter are explained in the description of the Value parameter. In general, this parameter should be set to Plain Text if the transformer is intended to insert text content into the document, while it should be set to JSON/XQuery if the transformer is intended to insert JSON objects or arrays into the document, or if the value is being produced through XQuery function calls.

ClosedValue

This parameter is used to specify the new values that should be inserted into the document. If the Value Type parameter is set to Plain Text, the contents of this parameter will be encoded for safe use in an XML document. That is, double quotes will be escaped as \", backslashes will be escaped as \\, and so on.

If the Value Type parameter is set to JSON/XQuery, the contents of this parameter will be interpreted as an XQuery expression. This allows the insertion of JSON values elements into the document. Also, this allows the use of the FME XQuery functions. For example, a JSON Template (for example, from the JSONTemplater transformer) could be entered as the parameter value. It would then be evaluated before being inserted into the document.

Note: Any FME XQuery functions used in the new value will access the Update feature, not the Document feature.

Parameters

ClosedValidate Attribute Names

This parameter specifies whether or not Workbench should try to validate the attribute names which are passed to any XQuery functions used in the Value column.

ClosedResult Attribute

This parameter specifies the attribute to which the updated JSON document will be written. The default is _result.

Editing Transformer Parameters

Using a set of menu options, transformer parameters can be assigned by referencing other elements in the workspace. More advanced functions, such as an advanced editor and an arithmetic editor, are also available in some transformers. To access a menu of these options, click beside the applicable parameter. For more information, see Transformer Parameter Menu Options.

Transformer Categories

Format Specific

Web

Search FME Knowledge Center

Search for samples and information about this transformer on the FME Knowledge Center.