Is there any limitation of twitter data fetching using R.
I am trying to get 2000 tweets but twitter API returns only 261
Warning messages: 1: In if (nchar(searchString) > 1000) { : the
condition has length > 1 and only the first element will be used 2: In
doRppAPICall(“search/tweets”, n, params = params, retryOnRateLimit =
retryOnRateLimit, : 2000 tweets were requested but the API can only
return 261
2
Answers
You will definitely not be getting as many tweets as exist. The way Twitter limits how far back you can go (and therefore how many tweets are available) is with a minimum since_id parameter passed to the GET search/tweets call to the Twitter API. In Tweepy, the API.search function interfaces with the Twitter API. Twitter’s GET search/tweets documentation has a lot of good info:
In practical terms, Tweepy’s API.search should not take long to get all the available tweets. Note that not all tweets are available per the Twitter API, but I’ve never had a search take up more than 10 minutes.
To avoid Twitter limitations use :
filterStream opens a connection to Twitter’s Streaming API that will return public statuses that match one or more filter predicates. Tweets can be filtered by keywords, users, language, and location. The output can be saved as an object in memory or written to a text file.
filterStream(file.name = NULL, track = NULL,
follow = NULL, locations = NULL, language = NULL, timeout = 0, tweets = NULL, oauth = NULL, verbose = TRUE)
NOTE: This function gets tweets on real time and avoids limitations.