Data Flow
One central problem with many ESF workspaces is that it they are not structured according to the way data flow works in FME. In FME, splitting data flow is the same as copying, and combining is the same as appending.
If you want records to actually merge attributes or features together, then you need to use combining transformers such as Aggregators, FeatureMergers, and Joiners.
Establishing Relationships with Common Keys
A very common problem is the lack of inter-referencing using the appropriate IDs. For example, every opening needs a unique ESF opening_id and every opening definition is a child on an opening and thus must contain both an ESF_opening_id and an ESF_opening_definition_id. If an opening definition cut block polygon is sent to the ESF writer without a reference to a parent opening that shares an ESF_opening_id with the opening_definition, then the opening_definition polygon will be rejected by the ESF validator.
It is possible that some records will have a repeating key. For example, several cut blocks may be part of the same opening, in which case the composite key might be cut block and cutting permit. In the workspace, you will need place an Aggregator transformer so that these multiple cut blocks can become grouped into one aggregate feature associated with one opening definition and opening.
Lack of Compliance with Feature Specification
Often there are features rejected by the ESF writer because they don't comply with the associated feature representation documents (see Schema tables below). You may need to use FME's attribute processing transformers to parse data in such a way as to make it compliant. For example, you might use a StringReplacer transformer to get rid of the spaces in a telephone number since the ESF results feature representation specification states that the telephone number must be decimal(10,0).
Viewing/Testing Output
Try opening the XML output file with the Inspection application (FME Universal Viewer or FME Data Inspector) using the ESF Results reader. Ensure that you check Validate Dataset in the Reader Parameters. This applies the same schema check that the MOF ESF submission website applies to an online submission.
Trial and Error Approach
You may need to do some parsing and reformatting to get the metadata into a form that ESF will accept. Note that this is often a trial and error process.
Run the workspace, generate the XML. Again, try opening the XML output file with the Inspection application (FME Universal Viewer or FME Data Inspector) using the ESF Results reader. Ensure that you check Validate Dataset in the Reader Parameter.
If it fails, you can do the following:
- Note the row number and error message.
- Open the XML file in a text editor and note the problem field identified by the reader.
- Reference the schema tables in the document 2esf_results_fme_feature_representation.doc inside the ESF Templates.zip file, look up that field and feature type, and see what the requirements are. This is how you might find out that the date and phone number values were incorrectly formatted, of the wrong type, or empty when a value is required.
Example:
XML Parser error: 'Error in input
dataset:'file:///C:/RT/31827/results/RESULTS_OpeningDefinition.xml'
line:53 column:18 message:Datatype error:
Type:InvalidDatatypeValueException, Message:Value is not in enumeration .'
If we look in the source file, we find:
Line 53 shows:
<rst:actionCode/>
This must mean that there is a record where actionCode is blank.
To inspect your output and see why it might be failing, you may also need to try loading it with the Inspection application (FME Universal Viewer or FME Data Inspector) using the ESF Results reader. Ensure that you select Validate Dataset in the Reader Parameters.
Extracting and Managing Attributes
It can be handy to use a Joiner to extract field values from a spreadsheet or table. The trick is to find a common key that relates your geometry features and table records so you can retrieve OpeningDefinition attributes.
Use an AttributeRenamer transformer to rename the attributes to the exact names that ESF is expecting. This way you don't need to manually connect them at the output.
Use NullAttributeMapper transformers to handle those records with null values that would otherwise cause the translation to fail. These nulls may exist because there are records in the geometry file which can’t find a match in the spreadsheet. These will ultimately need to be matched and populated to avoid loss of these records.
Grouping Features
For Results submissions, you typically you need to group the features by opening_id using the Aggregator transformer. You could then branch your data flow and use a GeometryRemover to send the attribute only data to the Opening feature type, and the other dataflow branch could pass the aggregated polygons with attributes to the OpeningDefinition feature type.
Additional Resources
For additional diagnostic ideas, try going through: 7ESF_PROBLEMS_R2.doc which is included with the tutorial, Getting Started with BC MoF ESF.
If you are still experiencing a problem, please visit our Support page at https://www.safe.com/support/.