HTTPFetcher

Depending on the HTTP transformer, this parameter is either URL that will be accessed by the transformer, or the URL that identifies where data will be uploaded.

This parameter can be used to provide custom headers for the HTTP Request. Each header should be entered as plain text, on a single line. For example:

User-Agent: FME/Workbench (http://www.safe.com)

Accept-Language: en-US

Expect: 100-continue

If this parameter is set to Yes, any saved cookies that correspond to the URL being accessed will be sent with the request. In addition, any cookies contained in the response will be saved for use with subsequent HTTP requests.

Saved cookies will persist for the duration of the translation – they cannot be used in subsequent translations.

Saving cookies is useful when a web service keeps a "session". Then the following sequence of operations might be used:

1. An HTTPFetcher or HTTPUploader is used to log in to the web service and a session cookie is saved.
2. Another HTTP transformer is used to access the web service, utilizing the saved session cookie.
3. Another HTTPFetcher or HTTPUploader is used to log out of the web service, which invalidates the saved cookie.

This parameter names the attribute in which the HTTP response body will be saved. The default attribute name is _url_contents.

If this parameter is set to Yes, then the HTTP response body will be output to a temporary file and the Target Attribute will be populated with the file name (instead of the contents of the file).

Note: If you are downloading a large file with the HTTPFetcher, it is recommended that you set this parameter to Yes. This avoids creating a large attribute on the feature.

This parameter is used to indicate the character encoding of the target attribute. If an encoding name is selected, the target attribute will be tagged with that encoding. If “Auto Detect from HTTP Headers” is selected, the HTTP response headers will be examined for encoding information. If the headers contain an encoding, it will be used; otherwise the fme-binary encoding will be used. If no value is given for this parameter, the target attribute will be tagged with the system default encoding. When fetching binary data, the parameter should be set to “Binary”, or “Auto Detect from HTTP Headers”.

This parameter is optional. If it is set to a list attribute name, then HTTP headers returned by the server will be stored in the list attribute. The headers will be split into names and values. For example:

`_headers{0}.name' has value `X-XSS-Protection'
`_headers{0}.value' has value `1; mode=block'
`_headers{1}.name' has value `Date'
`_headers{1}.value' has value `Wed, 25 May 2011 17:20:20 GMT'
`_headers{2}.name' has value `Transfer-Encoding'
`_headers{2}.value' has value `chunked'
`_headers{3}.name' has value `Content-Type'
`_headers{3}.value' has value `text/html; charset=ISO-8859-1'
`_headers{4}.name' has value `Cache-Control'
`_headers{4}.value' has value `private, max-age=0'

This parameter is optional. If it is set to an attribute name, then the HTTP response code returned by the server will be stored in the attribute.

The optional HTTP Authentication Username, HTTP Authentication Password and HTTP Authentication Method parameters may be set for accessing a password-protected HTTP server. Basic, Digest, and NTLM access authentication methods are supported:

• Basic (HTTP Basic authentication): This is the default choice, and the only method in widespread use and supported virtually everywhere. This sends the username and password over the network in plain text, easily captured by others.
• Digest (HTTP Digest authentication): Digest authentication is more secure over public networks than the Basic method.
• NTLM (HTTP NTLM authentication): NTLM is a proprietary protocol invented and used by Microsoft. It uses a challenge-response and hash concept similar to Digest to prevent the password from being eavesdropped.

 Note that although the HTTP Basic access authentication is a mechanism designed to allow a client to provide credentials to a server on the assumption that the connection between them is trusted and secure, be aware that any credentials passed from client to server can be easily intercepted through an insecure connection.  

You can also use this transformer to access the contents of an FTP URL. If required, an FTP username and password can be specified in the HTTP Authentication Username and HTTP Authentication Password parameters. The contents of the file (or the filename if the Save File parameter is set to Yes) will be stored in the specified target attribute. When accessing an FTP URL, the _http_status_code attribute will not be populated.

The optional Proxy URL, Proxy Port, Proxy Username, Proxy Password, and Proxy Authentication Method parameters may be set for organizations that require Internet access via an HTTP proxy server.

Basic, Digest, and NTLM access authentication methods are supported:

• Basic (HTTP Basic authentication): This is the default choice, and the only method in widespread use and supported virtually everywhere. This sends the username and password over the network in plain text, easily captured by others.
• Digest (HTTP Digest authentication): Digest authentication is more secure over public networks than the Basic method.
• NTLM (HTTP NTLM authentication): NTLM is a proprietary protocol invented and used by Microsoft. It uses a challenge-response and hash concept similar to Digest to prevent the password from being eavesdropped.