4.18 X-Lab: Twitter’s REST API

We’ll now turn to a different type of Twitter data – static data, either recent tweets or user-level information. This type of data can be retrieved with Twitter’s REST API. We will use the tweetscores package here – a package that Pablo Barberá created to facilitate the collection and analysis of Twitter data.

4.18.1 Searching recent tweets

It is possible to download recent tweets, but only up those less than 7 days old, and in some cases not all of them.

What are the most popular hashtags?

You can check the documentation about the options for string search here.

4.18.2 Extracting users’ profile information

This is how you would extract information from user profiles:

Which of these has the most followers?

Download up to 3,200 recent tweets from a Twitter account:

What are the most common hashtags?

4.18.3 Building friend and follower networks

Download friends and followers:

What are the most common words that friends of the JayBadran account use to describe themselves on Twitter?

4.18.4 Estimating ideology based on Twitter networks

The tweetscores package also includes functions to replicate the method developed in the Political Analysis paper Birds of a Feather Tweet Together. Bayesian Ideal Point Estimation Using Twitter Data. For an application of this method, see also this Monkey Cage blog post.

4.18.5 Other types of data

The REST API offers also a long list of other endpoints that could be of use at some point, depending on your research interests.

  1. You can search users related to specific keywords:

  2. If you know the ID of the tweets, you can download it directly from the API. This is useful because tweets cannot be redistributed as part of the replication materials of a published paper, but the list of tweet IDs can be shared:

  3. Lists of Twitter users, compiled by other users, are also accessible through the API.

This is also useful if e.g. you’re interested in compiling lists of journalists, because media outlets offer these lists in their profiles.

  1. List of users who retweeted a particular tweet – unfortunately, it’s limited to only 100 most recent retweets.

  2. And one final function to convert dates in their internal Twitter format to another format we could work with in R: