Connects to a Twitter™ stream and outputs a new feature for each tweet. The transformer allows users to set up a filter, so that the output features will only contain tweets containing certain keywords, tweets that are geotagged with a particular location, and/or tweets which are written in particular languages.
A twitter stream is an HTTP connection that never closes, so except for error cases, any workspace containing this transformer will not terminate on its own. The transformer will continue to output features until the workspace is terminated by the user.
At least one feature must enter the transformer in order to start the stream. Once all the input features have arrived, the transformer will open a connection to the Twitter streaming API, and begin outputting features. Depending on the filters set up in the transformer parameters, the output from the transformer may not be continuous. There may be periods in which no tweets match the filters. This is particularly true when filtering by language and geometry.
If no keywords or geometry filters are provided, the transformer will connect to a sample stream. This stream provides a random sample of tweets as they are created. It is possible to filter this stream by language.
Note that a limited number of streaming connections is permitted per user. If too many simultaneous connections are attempted with the same user credentials, one or more of the connections will be terminated.
Output Ports
Tweets
The transformer will output a new feature for each tweet received from the twitter streaming API. Each output feature will contain several attributes:
_tweet_id: The identification number of the tweet.
_tweet_status: The text of the tweet.
_tweet_username: The username of the Twitter user who wrote the tweet.
_tweet_created_at: The date and time that the tweet was created. This will be formatted in the standard FME date/time format: YYYYMMDDHHMMSS
_tweet_search_result: The JSON text of the tweet. Further information about the tweet can be extracted from this value using the JSONFragmenter and JSONExtractor transformers.
Other
The Twitter streaming API will periodically return messages which are not tweets. For example, these messages could contain information on rate limiting or deleted messages. More information can be found here: http://dev.twitter.com/docs/streaming-apis/messages
One feature will be output through this port for each non-tweet message returned from the Twitter streaming API. Each of these features will have a _tweet_search_result attribute, which will contain the JSON object representation of the message. This object can be examined in Workbench using the JSON transformers.
Parameters
Stream Filter Parameters
Keywords
This parameter is used to filter the stream for tweets which contain the given keywords. For each input feature, the value should contain a space separated list of keywords. If a tweet contains all of the keywords from any single feature, it will be returned by the stream.
For example, if one feature provides the keywords “FME Workbench” while a second provides “safe”, the following tweets could be matched:
- FME Workbench
- Safe Software
The following tweets would NOT be matched:
- Twitter Workbench
- FME
Keywords are matched case insensitively. Furthermore, the keywords will be matched if they are contained in the Twitter username, a linked URL or Twitter hashtags. For more details, look at http://dev.twitter.com/docs/streaming-apis/parameters#track
Languages
This parameter can be used to restrict tweets to the selected languages. If no languages are selected, the streamed tweets could be in any language.
Filter by Feature Geometry
If this parameter is set to Yes the bounding box of each input features will be calculated, and the tweet stream will be restricted to tweets which are geotagged, and whose coordinates fall within one of the bounding boxes.
Buffer for Point Features
This parameter only takes effect if the stream is being filtered by geometry. Since the stream requires a bounding box, this parameter determines the size of bounding box which will be used for point geometries.
Authentication Parameters
Note: For access outside of your network using a proxy server, use the Network tools in FME Options. From the Workbench menu, select Tools > FME Options > Network. For more information, see "Network" in the FME Workbench Help.
Twitter Username and Password
A valid Twitter username and password are required in order to access the Twitter streaming API.
Related Transformers
Editing Transformer Parameters
Using a set of menu options, transformer parameters can be assigned by referencing other elements in the workspace. More advanced functions, such as an advanced editor and an arithmetic editor, are also available in some transformers. To access a menu of these options, click 
beside the applicable parameter. For more information, see Transformer Parameter Menu Options.
Transformer Categories
FME Licensing Level
FME Professional edition and above
FME Community
The FME Community is the place for demos, how-tos, articles, FAQs, and more. Get answers to your questions, learn from other users, and suggest, vote, and comment on new features.
Search for samples and information about this transformer on the FME Community.