The directives that are processed by the BigQuery reader are listed below. The suffixes shown are prefixed by the current <ReaderKeyword>_ in a mapping file. By default, the <ReaderKeyword> for the BigQuery reader is COM.SAFE.FME.GOOGLEBIGQUERY_IN.
REFRESH_TOKEN
A refresh token issued by a Google account is required to access BigQuery.
A refresh token should be protected in the same manner as a password, as it will grant access until revoked by the issuing Google account.
Required/Optional
Required
Mapping File Syntax
COM.SAFE.FME.GOOGLEBIGQUERY_IN_REFRESH TOKEN 1/asdkeivnasdfieinadfnasdfiadfh
Workbench Parameter
Refresh Token
BILLING_PROJECT
The project that will be billed for BigQuery operations.
All BigQuery access must be billed to a specific project. If no billing project is specified, the project that owns the table being read will be billed, so long as the account that issued the refresh token has permission to bill to it.
Required/Optional
Optional
Mapping File Syntax
COM.SAFE.FME.GOOGLEBIGQUERY_IN_BILLING_PROJECT mydomain.com:my-project
Workbench Parameter
Billing Project
IDs
This optional specification is used to limit the available and defined database tables that will be read. The syntax of the IDs directive is:
COM.SAFE.FME.GOOGLEBIGQUERY_IN_DEF <tableName> \ [googlebigquery_sql_where_clause <where clause>] \ [googlebigquery_use_cached_results (yes|no)] \ [googlebigquery_allow_large_results (yes|no)] \ [googlebigquery_destination_table <table name>] \ [googlebigquery_write_preference (WRITE_EMPTY|WRITE_APPEND|WRITE_TRUNCATE)] \ [<fieldName> <fieldType>]*
ThThe following example selects rows from the table ROADS, placing the resulting data into FME features with a feature type of ROADS:
COM.SAFE.FME.GOOGLEBIGQUERY_IN_DEF ROADS
Required/Optional
Optional
Configuration Parameters
The configuration parameters present on the definition line are described in the following table:
| Parameter | Contents |
|---|---|
| tableName | The name of the table to be read. |
| googlebigquery_sql_where_clause | The SQL WHERE clause to restrict what data is read. |
| googlebigquery_use_cached_results |
This specifies whether the BigQuery cache from previous queries will be used, if possible. When a query is satisfied by the cache, you will not be billed for any usage. Caching cannot be enabled when a destination table has been selected. Default: Yes |
| googlebigquery_destination_table | In addition to being output as features, the results will stored in this table. |
| googlebigquery_allow_large_results |
Only applies if a destination table has been chosen. Causes the normal restrictions on large volumes of data to be ignored. Default: No |
| googlebigquery_write_preference |
This controls whether existing data in the destination table will result in appending, overwriting, or failure. Valid values are WRITE_EMPTY WRITE_APPEND WRITE_TRUNCATE |
| fieldName |
The name of the field to be read. Only specified fields will be read. If no fields are specified, all fields will be read. |
| fieldType |
The type of a column in a table. This value is ignored. |
IDs
This optional specification is used to limit the tables that will be read. The syntax of the IDs directive is:
COM.SAFE.FME.GOOGLEBIGQUERY_IN_IDs <featureType1> \ <featureType2> \ <featureTypeN>
The feature types must match those used in DEF lines.
The example below selects only the ROADS table for input during a translation:
COM.SAFE.FME.GOOGLEBIGQUERY_IN_IDs ROADS
Required/Optional
Optional
RETRIEVE_ALL_TABLE_NAMES
This parameter is only applicable when generating a mapping file, generating a workspace or when retrieving schemas in an FME Objects application.
When set to yes, and if RETRIEVE_ALL_SCHEMAS is set to no (or missing), names for all of the tables and feature classes in the database are returned. When set to no (or missing), and if RETRIEVE_ALL_SCHEMAS is similarly set to no (or missing), the schemas requested by the IDs directive are returned.
Note: If RETRIEVE_ALL_SCHEMAS is also set to Yes, then RETRIEVE_ALL_SCHEMAS takes precedence.
Required/Optional
Optional
Values
YES | NO (default)
Mapping File Syntax
Not applicable.
FME Objects applications would include RETRIEVE_ALL_TABLE_NAMES followed by “YES” in the parameters array passed to IFMEUniversalReader::open().
Workbench Parameter
Not applicable (used when you browse a Table List)
EXPOSED_ATTRS
This directive allows the selection of format attributes to be explicitly added to the reader feature type.
This is similar to exposing format attributes on a reader feature type once it has been generated; however, it is even more powerful because it enables schema-driven applications other than Workbench to access and leverage these attributes as if they were explicitly on the schema as user attributes.
The result of picking a list of attributes is a comma-separated list of attribute names and types that will be added to the schema features. Currently all reader feature types will receive the same set of additional schema attributes for a given instance of the reader.
Required/Optional
Optional
Mapping File Syntax
Not applicable.
While it is possible for FME Objects applications to invoke this directive, the required format is not documented.
This directive is intended for use in our GUI applications (for example, Workbench) only.
Workbench Parameter
Additional Attributes to Expose