I need to stream live tweets from twitter API and then analyse them. I should use kafka to get tweets or spark streaming directly or both ?
Question posted in Twitter API
The official Twitter API documentation can be found here.
The official Twitter API documentation can be found here.
2
Answers
Depending on your language of choice I would use one of the libraries listed here: https://developer.twitter.com/en/docs/developer-utilities/twitter-libraries. Which ever you choose, you will be using
statuses/filter
in the Twitter API, so get familiar with the doc here: https://developer.twitter.com/en/docs/tweets/filter-realtime/api-reference/post-statuses-filter.htmlYou can use Kafka Connect to ingest tweets, and then Kafka Streams or KSQL to analyse them. Check out this article here which describes exactly this.