Question 81339 in Docker Question posted in
The official documentation can be found here.

Not able to read from Tweets from twitter – Docker

I am trying to read tweets having specific keywords using docker. I have taken reference from
Github link .

I have made some minor changes. While I’m trying to execute I am facing issues with a number of arguments through all the details in place. It would be great if anybody can guide me where I’m doing wrong

### twitter
import tweepy
from tweepy.auth import OAuthHandler
from tweepy import Stream
#from tweepy.streaming import StreamListener
import json
import logging 


### logging 
FORMAT = "%(asctime)s | %(name)s - %(levelname)s - %(message)s"
LOG_FILEPATH = "C:\docker-kafka\log\testing.log"
logging.basicConfig(
    filename=LOG_FILEPATH,
    level=logging.INFO,
    filemode='w',
    format=FORMAT)

### Authenticate to Twitter
with open('C:\docker-kafka\credential.json','r') as f:
    credential = json.load(f)

CONSUMER_KEY = credential['twitter_api_key']
CONSUMER_SECRET = credential['twitter_api_secret_key']
ACCESS_TOKEN = credential['twitter_access_token']
ACCESS_TOKEN_SECRET = credential['twitter_access_token_secret']
BEARER_TOKEN = credential['bearer_token']



#from tweepy.streaming import StreamListener
from tweepy import OAuthHandler
from tweepy import Stream
from kafka import KafkaProducer
producer = KafkaProducer(bootstrap_servers='localhost:9092',
    value_serializer=lambda v: v.encode('utf-8')) #Same port as your Kafka server


topic_name = "docker-twitter"


class twitterAuth():
    """SET UP TWITTER AUTHENTICATION"""

    def authenticateTwitterApp(self):
        auth = OAuthHandler(consumer_key=CONSUMER_KEY, consumer_secret=CONSUMER_SECRET)
        auth.set_access_token(ACCESS_TOKEN, ACCESS_TOKEN_SECRET)

        return auth



class TwitterStreamer():

    """SET UP STREAMER"""
    def __init__(self):
        self.twitterAuth = twitterAuth()

    def stream_tweets(self):
        while True:
            listener = ListenerTS() 
            auth = self.twitterAuth.authenticateTwitterApp()
            stream = Stream(auth, listener)
            stream.filter(track=["Starbucks"], stall_warnings=True, languages= ["en"])


class ListenerTS(tweepy.Stream):

    def on_status(self, status):
        tweet = json.dumps({
            'id': status.id, 
            'text': status.text, 
            'created_at': status.created_at.strftime("%Y-%m-%d %H:%M:%S")
        }, default=str)  

        producer.send(topic_name, tweet)
        return True


if __name__ == "__main__":
    TS = TwitterStreamer()
    TS.stream_tweets()

enter image description here

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    I was able to resolve the issue by adding the secrets but got a different error.

    def stream_tweets(self):
    while True:
        listener = ListenerTS(CONSUMER_KEY,CONSUMER_SECRET,ACCESS_TOKEN,ACCESS_TOKEN_SECRET) 
        listener.filter(track=["Starbucks"], stall_warnings=True, languages= ["en"])

    enter image description here


  2. 0

    Answer reference:

    Not able to read from Tweets from twitter

    As far as i understand, the class tweepy.Stream needs to be initialized, even if inheriting it. So, instead of:

    try

    
class ListenerTS(tweepy.Stream):
    def __init__(self):
        tweepy.Stream.__init__(self, CONSUMER_KEY, CONSUMER_SECRET, ACCESS_TOKEN, ACCESS_TOKEN_SECRET)

    Also check this docs:
    https://docs.tweepy.org/en/stable/streaming.html

    And maybe this link:
    https://improveandrepeat.com/2022/04/python-friday-117-streaming-search-results-with-tweepy/

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search