Articles in this section

Work with Twitter

  • Updated
Follow

Twitter.png

What can you do with Twitter in Etlworks

Create a Twitter app

By creating a Twitter app, you are enabling access to the Twitter API. A Twitter app is not an actual application. It is just a set of credentials and access rights linked to your Twitter account.

Read more

Create a Twitter Connection

You can pull data from Twitter's Graph API or post to Twitter. You can select either the Twitter Feed or the Twitter Update Connection.

Read more

Pull data from Twitter

Create a Twitter Feed Connection, which will be used as the source (FROM) and then create a destination (TO) Connection and (if needed) a Format.

Read more

Stream data from Twitter

The Twitter API platform offers two options for streaming real-time Tweets: follow a Twitter user or track keywords. Etlworks supports both options.

Read more

Post data to Twitter

Create a Twitter Update Connection. Start creating a Twitter Connection by opening the Connections window, clicking +, and typing twitter into the search box. Select the Update Twitter Flow type.

Read more

Delete data from Twitter

Create a Twitter Update Connection. When configuring the Connection, select one of the Delete... actions and selecting the Delete Files Flow type.

Read more

 

Related resources

Automate a status update in multiple social networks

Create Update Connections for multiple networks, for example, a Twitter Update Connection and a Facebook Update Connection.

Read more

Send a content of the file as a payload for an HTTP endpoint

Use the following technique to send a file as a payload for an HTTP endpoint.

Read more

Twitter Streaming connector

Create a Twitter Stream Feed Connection to stream Twitts in real-time. To stream data from Twitter, you first need to create a Twitter app.

Read more

 

 

Video

How to extract data from business application or well-known API and load into database

  1. Create a source Connection for a well-known API.
  2. Create a destination Connection
  3. Create a new Well-known API to database Flow.
  4. Add a new transformation.
  5. Click MAPPING 
  6. Test it

Read the full article

 

Connect to Twitter

Create Twitter app

Create a Twitter Connection

Create a Twitter app

By creating a Twitter app, you are enabling access to the Twitter API. A Twitter app is not an actual application. It is just a set of credentials and access rights linked to your Twitter account.

We never store the actual Twitter credentials (screen name and password) in the Etlworks's database. Instead, we store keys and tokens, which have been assigned to the Twitter app.

In most cases, you will need a new app for each Twitter account you will be accessing, using Etlworks's Twitter connector.

To create a Twitter app:

Step 1. Log out from your personal Twitter account.

Step 2. Go to https://apps.twitter.com/app/new and log in with the account you want to create a Connection to.

Step 3. Enter the required parameters, accept the Developer Agreement, and click Create your Twitter application.

Use a fully qualified URL, the one you would use to log in to Etlworks, for the Web Site field. For example, if app is hosted by Etlworks, enter https://app.etlworks.com or https://your_company_name.etlworks.com.

twitter-app.png

Step 4. Under the Keys and Access Tokens (in some cases, called Application Management), go to Token Actions and click Create my access token.

twitter-create-token.png

Step 5. Go to the Permissions tab and configure the application's permissions. Click Update Settings. If requested, you can use https://etlworks.com/docs/privacy-policy/ as your privacy policy URL.

twitter-permissions.png

Step 6. Go back to the Keys and Access Tokens tab, then copy and paste the Consumer Key API (API Key), Consumer Secret (API Secret), Access Token, and Access Token Secret. You will be using them to create a Twitter Connection in Etlworks.

twitter-tokens.png

You are done. You can always access your application by opening https://apps.twitter.com, logging in using the designated Twitter account, and selecting the app from a list.

twitter-apps.png

Create a Twitter Connection

Step 1. Start creating a Twitter Connection by opening the Connections window, clicking +, and typing twitter into the search box:

twitter-connections.png

Step 2. Depending on whether you will be pulling data from Twitter's Graph API or posting to Twitter, select either the Twitter Feed or the Twitter Update Connection.

Step 3. Enter the Authentication parameters. Then, click Test Connection to verify that the credentials are working.

twitter-authentication-parameters.png

Step 4. Continue by entering the Connection parameters, which are unique for each type of Connection.

Twitter Feed Connection

A Twitter Feed Connection is used to pull data from the Twitter API.

Common Parameters

  • Feed Type: each Twitter feed type is linked to a specific Twitter API endpoint. Pay attention to the hint in the feed type name. For example, Search tweets by keywords implies that the Keyword(s) are a required parameter, which is entered as Query Parameters. Other feed types require an ID or other parameters.
  • Query parameters: this field is mostly used to Search... feed types. A query parameter is required for some endpoints and ignored by others.

Filter

  • Screen Name(s): a comma-separated list of Twitter screen names for the requests that require a screen name. Use valid screen name syntax, for example @etlworks.
  • ID: some clearly marked feed types accept an optional ID, such as a List ID, Location ID (WOEID) or Tweet ID.
  • Location: some clearly marked feed types accept optional location information. The location must be in the Format Latitude,Longitude. For example: 40.4327086,-79.92486689999998.
  • Category: some clearly marked feed types accept an optional category.

  • twitter-feed-connection.png

Twitter Update Connection

A Twitter Update Connection is used to post to Twitter and delete data from Twitter.

  • Action: a type of update. Each action is linked to a specific Twitter POST or DELETE API endpoint.

Post parameters

  • Message: a message to post. It can include direct links in the Format https://link.
  • Picture: the URL of a picture to post with the message.

Additional parameters

  • Screen Name(s): a Twitter screen name in a valid format, for example @etlworks.
  • ID: some clearly marked actions accept an optional ID, such as a Tweet ID or a Direct Message ID.
  • Location: optional location information for the post. The location must be in the format Latitude,Longitude. For example: 40.4327086,-79.92486689999998.
  • Possibly Sensitive Post: if it is selected, the post will be marked as possibly sensitive.

  • twitter-update-connection.png

Step 5. Optionally, continue by entering the auto-retry parameters:

  • Retries: the number of retries allowed for failed execution attempts.
  • Delay between retries: the delay between retries, in minutes.

Step 6. Click Save to save the Connection.

Pull data from Twitter

Step 1. Create a Twitter Feed Connection, which will be used as the source (FROM).

Step 2. Create a destination (TO) Connection and (if needed) a Format.

Step 3. Start creating a Twitter Connection by opening the Connections window, clicking +, and typing well-known API to into the search box:

Step 4. Select Flow type based on what the destination is going to be. For example, Extract data from well-known API, create file.

Step 5. Select the Connection created in Step 1, as the source (FROM) and the Connection/Format created in Step 2, as the destination (TO). Set the FROM and TO fields.

twitter-to-file.png

Step 6. Continue by setting the Parameters and defining the Mapping.

Stream data from Twitter

The Twitter API platform offers two options for streaming real-time Tweets: follow a Twitter user or track keywords. Etlworks supports both options.

Step 1. Create a Twitter Streaming Connection.

Step 2. Create a destination Connection, for example, a Connection to the relational database.

Step 3. Start creating a Flow by selecting Flow type Well-known API to database.

Step 4. Add new source-to-destination transformation.

  • FROM Connection: Connection created in step 1
  • FROM: comma-separated keywords (hashtags) or numerical Twitter user IDs
  • TO Connection: Connection created in step 2.

twitter-streaming-trans.png

Step 5. Modify Mapping and Parameters if needed.

Step 6. Schedule Flow to run in real-time.

Post to Twitter

Step 1. Create a Twitter Update Connection.

Step 2. Start creating a Twitter Connection by opening the Connections window, clicking +, and typing twitter into the search box. Select the Update Twitter Flow type.

update-twitter-flow.png

Step 3. Continue by selecting the Connection created in Step 1 as a Twitter Update Connection.

twitter-update-flow-params.png

Step 4. Optionally, define the message text in the Parameters tab. The message can contain {tokens}, which will be substituted at runtime.

twitter-message.png

Delete data from Twitter

Step 1. Create a Twitter Update Connection. When configuring the Connection, select one of the Delete... actions. Enter an ID. Most of the Delete... actions require an ID: User ID, Tweet ID, etc.

Step 2. Start creating a Twitter Connection by opening the Connections window, clicking +, and typing delete files into the search box. Select the Delete Files Flow type.

delete-api-endpoint.png

Step 3. Add a transformation and select the Connection created in Step 1 as the source Connection.

Related articles

Comments

0 comments

Please sign in to leave a comment.