skip to Main Content

Python code used to gather tweets and send it to a csv file

keeps returning error

tweepy.error.RateLimitError: [{‘code’: 88, ‘message’: ‘Rate limit exceeded’}]

trying to get the most recent timeline and send all these tweets to the csv file

thanks for any help

def get_all_tweets(screen_name):

    #authorize twitter, initialize tweepy
    auth = tweepy.OAuthHandler(consumer_token, consumer_secret)
    auth.set_access_token(access_token, access_secret)
    api = tweepy.API(auth)

    #initialize a list to hold all the tweepy Tweets
    alltweets = []    


    new_tweets = api.home_timeline (screen_name = screen_name,count=20)

    #save most recent tweets
    alltweets.extend(new_tweets)

    #save the id of the oldest tweet less one
    oldest = alltweets[-1].id - 1

    while len(new_tweets) > 0:
        print ("getting tweets before %s" % (oldest))

        #all subsiquent requests use the max_id param to prevent duplicates
            new_tweets = api.home_timeline(screen_name =      screen_name,count=20,max_id=oldest)

        #save most recent tweets
        alltweets.extend(new_tweets)

        #update the id of the oldest tweet less one
        oldest = alltweets[-1].id - 1

        print ("...%s tweets downloaded so far" % (len(alltweets)))

    #transform the tweepy tweets into a 2D array that will populate the csv    
    outtweets = [[tweet.id_str, tweet.created_at, tweet.text.encode("utf-8")] for tweet in alltweets]

    #write the csv    
    with open('%s_tweetsBQ.csv' % screen_name, 'w') as f:
        writer = csv.writer(f)
        writer.writerow(["id","created_at","text"])
        writer.writerows(outtweets)

    pass


    if __name__ == '__main__':
    #pass in the username of the account you want to download
    get_all_tweets("BQ")

2

Answers


  1. Did you tried to add this option on tweepy initialization, so it will wait instead of failing, when rate limit is reached :

    api = tweepy.API(auth, wait_on_rate_limit=True)
    
    Login or Signup to reply.
  2. Your code is okay, you just reached the Twitter Streaming API limit. It takes about one hour to let you extract tweets again.

    You should add the wait_on_rate_limit=True option when initializing tweetpy :

    api = tweepy.API(auth, wait_on_rate_limit=True)
    

    I strongly suggest that you take a look at : https://dev.twitter.com/rest/public/rate-limiting for more details.

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search