This section describes the keywords the SDE writer module recognizes. Each of the keywords is prefixed by the current <WriterKeyword>_ when they are placed in a mapping file. By default, the <WriterKeyword> for writing vector data is SDE30, the default for writing rastermap data is SDERASTERMAP, and the default for writing raster catalog data is SDERASTERCATALOG.
RECONCILE_AND_POST
Required/Optional: Optional
Note: Valid only for vector features. Not supported for raster tables.
This directive determines which changes to reconcile between the child version (i.e., the version specified by the connection-related directive VERSION_NAME) and its parent version. Conflicts must be resolved manually using Esri ArcGIS. Valid values are INSERTS, UPDATES, DELETES, and ALL. More than one value can be specified as long as each value is separated by a space. A post of the child version to its parent will be automatically performed when ALL is specified, or when INSERTS, UPDATES, and DELETES are all specified. Upon successfully posting the child version to its parent, whether or not the child version is deleted is dependent upon the value of the DELETE_CHILD_AFTER_RECONCILE_AND_POST directive. The directive is also used to determine whether the child version is deleted when it is identical to its parent version, in which case no reconciliation or posting is needed.
The post will be performed automatically when ALL or INSERTS, UPDATES, and DELETES is specified. If an error occurs during the post phase, then all changes made during both the reconcile and post phases will be rolled back.
Value: Any combination of INSERTS, UPDATES, DELETES, and ALL. Each value must be separated by a space.
- INSERTS – features inserted in the parent version
- UPDATES – features updated in the parent version
- DELETES – features deleted in the parent version
- ALL – encompasses INSERTS, UPDATES, and DELETES.
Example:
In the example below, all the updates and deletes made to the child version will be reconciled with the parent version. If a conflict occurs, then none of the reconciled changes will be saved. If INSERTS was also specified, then a post back to the parent version would also occur, but since neither it nor ALL is specified, only a reconciliation will be performed.
SDE30_RECONCILE_AND_POST UPDATES DELETES
Workbench Parameter: Reconcile and Post
TABLES_TO_RECONCILE
Required/Optional: Optional
Note: Note: Valid only for vector features. Not supported for raster tables.
This optional statement specifies a list of tables (separated by spaces) which should be reconciled. Using this directive, it is possible to use an ArcSDE writer just to reconcile and post changes (i.e., not to write features). It is also possible to specify additional tables to reconcile that were not written to during the current translation. This directive only gets used if the directive RECONCILE_AND_POST is specified. If no tables are specified (and RECONCILE_AND_POST is specified), then only the tables written to during the translation will be reconciled. If no tables were written to during the translation, then no tables will be reconciled.
Value: <[table name]*> separated by spaces. If a table is owned by a different user, then the table name must be prefixed by the owner.
Example:
SDE30_TABLES_TO_RECONCILE countries rivers cities
Workbench Parameter: Tables to Reconcile
DELETE_CHILD_AFTER_RECONCILE_AND_POST
Required/Optional: Required
Note: Valid only for vector features. Not supported for raster tables.
This directive determines whether to delete the child version following a reconcile and post, including the case where the child and parent version are identical. A value of ‘YES’ will delete the child version, while a value of ‘NO’ will leave it intact. The default value is ‘YES’.
Value: YES or NO.
Example:
In the example below, the child version will not be deleted after the reconcile and post operation completes.
SDE30_DELETE_CHILD_AFTER_RECONCILE_AND_POST NO
Workbench Parameter: Delete child state following reconcile and post
TRANSACTION
Required/Optional: Optional
This statement instructs the SDE writer module to use transactions when loading data into the SDE. The SDE writer does not write any features to the SDE until the feature is reached that belongs to <last successful transaction> + 1. Specifying a value of 0 causes the SDE writer to use transactions and to write every feature to the SDE. Normally, the value specified is zero – a non-zero value is only specified when a data load operation is being rerun.
If the SDE30_TRANSACTION statement is not specified, then transactions are not used during the data load operation.
Value: <last successful transaction>
The transaction number of the last successful transaction. When loading data for the first time, set this value to 0.
Example:
SDE30_Transaction 0
Workbench Parameter: Last Successful Transaction
TRANSACTION_INTERVAL
Required/Optional: Optional
This statement informs FME about the number of features to be placed in each transaction before a transaction is committed to the database. When set to VARIABLE the SDE writer checks each feature for the fme_db_transaction attribute, for which there are 4 valid values:
- COMMIT_BEFORE - The current transaction is committed before writing the feature.
- COMMIT_AFTER - The current transaction is committed immediately after writing the feature.
- ROLLBACK_AFTER - The current transaction is rolled back immediately after writing the feature.
- IGNORE - The feature is written and no transaction handling occurs.
When the attribute is not found on the feature, then a value of IGNORE is assumed.
If the SDE30_TRANSACTION_INTERVAL statement is not specified, then a value of 100 is used as the transaction interval.
Value: <transaction_interval>
Either the number of features in a single transaction, or the value VARIABLE.
Default value: 100
Note: WARNING: If the SDE30_TRANSACTION statement is not specified, then transactions are not used during the data load operation, even if the SDE30_TRANSACTION_INTERVAL is specified.
Example:
SDE30_Transaction_INTERVAL 50
Workbench Parameter: Features to Write Per Transaction
STRICT_LOAD
Required/Optional: Optional
Note: Valid only for vector features. Not supported for raster tables.
This statement instructs FME to be very strict when loading spatial data from a feature into the SDE. When FME encounters a feature whose geometry cannot be converted into an SDE shape allowed by the layer (vector spatial column) for which it is destined, FME terminates the data load, logs the feature, and aborts the current transaction. In comparison to the CONTINUE_TRANSLATION_BAD_DATA directive, STRICT_LOAD is very limited in the type of errors that it can ignore. Use CONTINUE_TRANSLATION_BAD_DATA when it is desirable to ignore the majority of errors that may occur during a data load.
Value: YES | NO
Default Value: NO
Example:
SDE30_STRICT_LOAD YES
Workbench Parameter: Strict SDE Load
FORCE_IN_AGGREGATES
Required/Optional: Optional
Note: Valid only for vector features. Not supported for raster tables.
This statement instructs FME to make an extra effort to store multi-part polygon shapes (aggregates of polygons) into the SDE. When placed in this mode, the FME breaks apart aggregates that, according to the SDE, contain both polygons and lines, and attempts to store them as two feature aggregates. This is designed to assist with the loading of multi-part polygonal data in which some of the polygons are non-compliant with SDE’s definition of a polygon.
To use this mode effectively, you must ensure that any polygonal layer (vector spatial column) for which this is applicable also allows for the storage of multi-part lines.
Value: YES | NO
Default Value: NO
Workbench Parameter: Force In Aggregates
DEFAULT_Z_VALUE
Required/Optional: Optional
Note: Note: Valid only for vector features. Not supported for raster tables.
The value to be used for the z coordinate when a 2D feature is forced to become 3D because the layer (vector spatial column) is defined as being 3D. The z value specified for this directive must be larger than the z origin.
Value: any real number
Default Value: 0
Example:
SDE30_DEFAULT_Z_VALUE 52.3
Workbench Parameter: Default Z Value
LEAVE_LAYER_EXTENTS
Required/Optional: Optional
Note: Note: Valid only for vector features. Not supported for raster tables.
By default, FME updates the layer (vector spatial column) extents when loading data into the SDE. This directive tells the FME not to perform this processing, thereby leaving the layer extent untouched.
Value: YES | NO
Default Value: NO
Example:
SDE30_LEAVE_LAYER_EXTENTS NO
Workbench Parameter: Leave Layer Extents
SPLIT_DONUTS
Required/Optional: Optional
Note: Note: Valid only for vector features. Not supported for raster tables.
This directive is used when donut polygons are not to be stored as donuts, but rather simple polygons. When set to YES donut polygons are not stored in the SDE.
Value: YES | NO
Default Value: NO
Example:
SDE30_SPLIT_DONUTS NO
Workbench Parameter: Store Donuts as Polygons
CONTINUE_TRANSLATION_BAD_DATA
Required/Optional: Optional
This statement instructs the SDE writer to continue a translation even when an error occurs while attempting to load the data (the error may or may not be related to the data). A warning message will be output for each feature that could not be written to the SDE. The warning message will explain what went wrong. This directive is useful when trying to load bad data. Many more errors can be ignored using this directive than by using STRICT_LOAD. The STRICT_LOAD directive pertains only to converting geometry (from a feature) into an SDE shape to be written to a layer (vector spatial column).
When set to ROLLBACK_THEN_CONTINUE, if a feature fails to be written then the current transaction will be rolled back rather than committed when it comes time to commit the transaction. This means that none of the features in the rolled back transaction will be written to SDE. The translation will continue regardless of the error encountered. Transactions must be used when this value is specified. When used in conjunction with the REJECTED_PIPELINE_DIRECTORY keyword, if a feature returned from a pipeline fails to be written then the current transaction will be rolled back instead of committed.
If set to YES and transactions are being used, all transactions will be committed; however, failed features will not be written to SDE.
Value: YES | NO | ROLLBACK_THEN_CONTINUE
Default Value: NO
Example:
SDE30_CONTINUE_TRANSLATION_BAD_DATA ROLLBACK_THEN_CONTINUE
Workbench Parameter: Skip Bad Data
REJECTED_PIPELINE_DIRECTORY
Required/Optional: Optional
This statement instructs the FME where to find the pipeline file(s) to be used. A pipeline is used when a failure occurs in writing a feature. When this statement is specified and there was an error in writing a feature, the writer first attempts to open a pipeline specific to the current table. The writer looks for a file called <tableName>_pipeline.fmi in the folder specified by this statement. If the file is not found, then the writer looks for a default pipeline called default_pipeline.fmi in the same folder. If neither of these files are found, then the translation is stopped.
If a pipeline file is found, then an FME pipeline is created using all the factories from the file. The pipeline can do almost anything a regular FME pipeline can do. However, only the first feature from the pipeline is retrieved. If the pipeline does not return any feature, then the writer does not insert into SDE the row that corresponds to the feature. At the present time, the feature is recorded as written in the statistics portion of the FME log, whether or not it was actually inserted into SDE.
If this directive is used with the CONTINUE_TRANSLATION_BAD_DATA directive set to YES or ROLLBACK_THEN_CONTINUE, and a feature is returned from the pipeline, then if the returned feature causes an error while being written to the SDE it will not cause the translation to stop. Rather a warning message, explaining why the feature couldn’t be written, will be logged and the translation will continue. If ROLLBACK_THEN_CONTINUE was specified, the current transaction will be rolled back instead of committed when it comes time to commit the transaction.
If this statement is not specified, then no pipeline will be created by the writer for features rejected by SDE. A pipeline is only created if this statement is specified and a failure occurs in writing a feature
Value: The absolute or relative path of a folder containing pipeline files. If a relative path and the command line FME are used, then the path is relative to the location where FME is called from. If a relative path and the Universal Translator are used, then the path is relative to the location of the mapping file. If the path contains spaces in it, the path should be double-quoted.
Example:
SDE30_REJECTED_PIPELINE_DIRECTORY c:\sde\pipelines
Workbench Parameter: Rejected Pipeline Folder
WRITER_MODE
Required/Optional: Optional
Note: For more information on this directive, see the chapter Database Writer Mode.
This statement instructs the FME as to the type of mode in which it is to operate. If the value specified is INSERT, then the features being written to the database are not checked to see if duplicate key values are in the database. This is useful when loading new data into the SDE.
When the WRITER_MODE directive is set to UPDATE or DELETE, the writer will check to see if the format attribute fme_db_operation exists on the feature and, then, if it does the specified operation will be used. If the attribute is set to INSERT, the feature will be inserted; if the value is UPDATE, the feature will be updated if present, otherwise it will be inserted; and if the value is DELETE, the feature will be deleted. If the fme_db_operation is not found then the operation performed will be based on the value of WRITER_MODE.
The writer needs some way to identify which rows to update or delete. It will first look on the feature for a WHERE clause in the format attribute fme_where. If the fme_where attribute is not present, the columns given in SDE_UPDATE_FIELDS (Update Key Columns) configuration parameter will be used to make up a key. If neither the fme_where attribute nor the SDE_UPDATE_FIELDS are present, the writer will use ArcSDE-maintained (Object Id) column as the key, provided the table is registered or is spatially enabled. To avoid updating or deleting multiple features all at once, the user is responsible for ensuring that the specified key uniquely identifies a single feature within the SDE database. It is strongly recommended for performance reasons that the columns referenced in fme_where or in SDE_UPDATE_FIELDS are properly indexed using RDBMS indices.
If the WRITER_MODE statement is not specified, then INSERT mode is used.
Writing a Raster Map
When writing a raster map to a table in SDE, the writer mode functions in a slightly different manner. INSERT mode inserts the raster into the table, overwriting any data that already pre-existed. UPDATE mode specifies that the writer is to mosaic the raster data to the pre-existing data, thereby updating the single row in the table. Raster catalogs may be inserted to, updated, and deleted from in a similar manner as vector data.
In order to perform a successful update, several conditions must be met by all the raster data that is to be mosaicked: the coordinate systems must be the same, the pixel depth must be the same, and the raster data itself must be either palette colored or continuous (they cannot be mixed). There is also a requirement for cell size and alignment to be the same, but the SDE writer will correct for these automatically, so they need not be altered. There is also no need to alter the writer mode when mosaicking, since the writer will automatically detect and correct this based on whether or not the data is pre-existing, in order not to overwrite it. The only way to overwrite existing raster data in a raster map is to set either the SDE_DROP_TABLE or the SDE_TRUNCATE_TABLE flag to YES.
Value: INSERT | UPDATE | DELETE
Default Value: INSERT
Example:
SDE30_WRITER_MODE INSERT
Workbench Parameter: Writer Mode
BUFFERED_WRITES
Required/Optional: Optional
When specified, the buffered writing of the SDE is used which dramatically decreases the load time of data into the SDE.
Value: YES | NO
Default Value: NO
Example:
SDE30_BUFFERED_WRITES YES
Workbench Parameter: Use Buffered Writes
MAX_OPEN_TABLES
Required/Optional: Optional
Specifies the maximum number of streams that can be open simultaneously. Each stream writes to a particular table and so this directive determines the maximum number of tables that can be open and written to simultaneously. If this directive is not specified, or is given the value 0, then the SDE writer will set the maximum number of streams open simultaneously to 4 less than the number specified by MAXSTREAMS in giomgr.defs.
Value: The maximum number of tables that can be open simultaneously
Example:
SDE30_MAX_OPEN_TABLES 30
Workbench Parameter: Maximum Num of Open Tables
ADD_LAYERS_TO_EXISTING_TABLES
Required/Optional: Optional
Specifies whether and existing business table within ArcSDE should have a layer (vector spatial column) added to it. To be eligible for this schema modification, the first feature written to the table must contain vector geometry.
Value: YES | NO
Default Value: YES
Example:
SDE30_ADD_LAYERS_TO_EXISTING_TABLES NO
Workbench Parameter: Add Layers to Existing Tables
INTEGER_OVERRIDE_DEFINITION
Required/Optional: Optional
Specifies a definition to use for all integer column types when creating new tables. Any of the allowed FME attribute types for the ArcSDE Writer can be used as values for this directive. Additionally, the following can also be used: number(<width>) or number(<width>, <decimal>). By default, this directive is not set, and so integer columns are stored using the C language long integer data type.
Values:
- any allowed FME attribute types for the ArcSDE writer
- number(<width>) or number(<width>, <decimal>)
Example:
In the example here, the ArcSDE database will use char(30) instead of integer as the type for all integer columns.
SDE30_INTEGER_OVERRIDE_DEFINITION char(30)
Workbench Parameter: Integer Definition