Twitter API Question posted in
The official Twitter API documentation can be found here.

Recursive awaited promise does nothing after resolving – Twitter API

I’m making some requests to the Twitter API, and in order to retrieve tweets I need to perform some recursive method calls as each request will only return a maximum of 100 tweets.

So the process is pretty simple.

  1. Call the function and await it
  2. Perform a http request, await that
  3. If the metadata of the response contains a next_token, cache the results and perform another http request using the next token.
  4. Repeat until the next_token is undefined, at which point resolve the promise with the list of all tweets.

However this isn’t working as expected, the recursive http works, however when the else block of the recursive function satisfied and the promise is resolved, nothing happens. Execution doesn’t go back to the first function. Everything just seems to spin and do nothing. I’ve added in breakpoints on every line but nothing causes a breakpoint to trigger either.

Where am I going wrong here?

public async getTweetList(ticker: string): Promise<string[]>{
        let tweets: string[] = [];

        tweets = await this.getAllTweetsRecursively(ticker, null, tweets);
        return tweets;
    }

    public async getAllTweetsRecursively(ticker: string, nextToken: string, tweetList: string[]): Promise<string[]>{
        return new Promise(async (resolve, reject) => {
            let query = `?query=(${ticker})`
            query += this.extraQuery;

            if(nextToken){
                query += this.nextTokenQuery + nextToken
            }

            let res = await axios.default.get(this.url + query, {
                headers: this.headers
            })

            let newNextToken = res.data.meta.next_token;
            if(res.data.data.length > 0 && newNextToken){
                res.data.data.forEach(tweet => {
                    tweetList.push(tweet.text);
                })
                this.getAllTweetsRecursively(ticker, newNextToken, tweetList);
            }
            else {
                resolve(cleanedTweets)
            }
        })
    }

Alternative implementation – same issue

public async getTweetList(ticker: string): Promise<string[]>{
        return new Promise(async (resolve) => {
            let tweets: string[] = [];

            tweets = await this.getAllTweetsRecursively(ticker, null, tweets);
            resolve(tweets);
        })

    }

    public async getAllTweetsRecursively(ticker: string, nextToken: string, tweetList: string[]): Promise<string[]>{
        return new Promise(async (resolve, reject) => {
            let query = `?query=(${ticker})`
            query += this.extraQuery;

            if(nextToken){
                query += this.nextTokenQuery + nextToken
            }

            let res = await axios.default.get(this.url + query, {
                headers: this.headers
            })

            let newNextToken = res.data.meta.next_token;
            if(res.data.data.length > 0 && newNextToken){
                res.data.data.forEach(tweet => {
                    tweetList.push(tweet.text);
                })
                await this.getAllTweetsRecursively(ticker, newNextToken, tweetList);
            }
            else {
                let cleanedTweets: string[] = [];
                tweetList.forEach(tweet => {
                    if(tweet.startsWith("RT")){
                        return;
                    }
                    if(!tweet.toLowerCase().includes("$" + ticker)){
                        return;
                    }
                    cleanedTweets.push(tweet);
                });
                resolve(cleanedTweets)
            }
        })
    }
Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    The issue is that the inner promise was not resolving in the case where it recursively called itself.

    Adding resolve(await this.getAllTweetsRecursively(ticker, newNextToken, tweetList)); fixed the problem.

    public async getTweetList(ticker: string): Promise<string[]> {
  let tweets: string[] = [];
  return await this.getAllTweetsRecursively(ticker, null, tweets);
}

public async getAllTweetsRecursively(ticker: string, nextToken: string, tweetList: string[]): Promise<string[]>{
  return new Promise(async (resolve, reject) => {
    let query = `?query=(${ticker})`
    query += this.extraQuery;

    if(nextToken){
      query += this.nextTokenQuery + nextToken
    }

    let res = await axios.default.get(this.url + query, {
      headers: this.headers
    })

    let newNextToken = res.data.meta.next_token;
    if(res.data.data.length > 0 && newNextToken){
      res.data.data.forEach(tweet => {
        tweetList.push(tweet.text);
      })
      resolve(await this.getAllTweetsRecursively(ticker, newNextToken, tweetList));
    }
    else {
      res.data.data.forEach(tweet => {
        tweetList.push(tweet.text);
      })
      let cleanedTweets: string[] = [];
      tweetList.forEach(tweet => {
        if(tweet.startsWith("RT")){
          return;
        }
        if(!tweet.toLowerCase().includes("$" + ticker)){
          return;
        }
        cleanedTweets.push(tweet);
      });
      resolve(cleanedTweets)
    }
  })
}

  2. 0

    Remember that async function() {} already defines a functions that return a Promise, because that’s what the async keyword does for you. It makes your function body asynchronous by wrapping it in a Promise without you have to write the promise code yourself. And then for convenience, the await keyword is equivalent to Promise.then() without having to write the extraction code yourself.

    As such, the following two functions are identical:

    function() {
  return new Promise(resolve, reject => {
    resolve(4);
  });
}

async function() {
  return 4;
}

    Both return a Promise, both can be awaited inside another async function, and both can be .then() chained, because they are literally the same thing. So in the case of your code, if you want to use async/await you should not also build Promises:

    public async getTweetList(ticker: string): Promise<string[]> {
  return await this.getAllTweetsRecursively(ticker, null, []);
}

public async getAllTweetsRecursively(ticker: string, nextToken: string, tweetList: string[]): Promise<string[]>{
    let query = `?query=(${ticker})${this.extraQuery}`;

    if(nextToken){
      query = `${query}${this.nextTokenQuery}${nextToken}`;
    }

    let res = await axios.default.get(this.url + query, {
      headers: this.headers
    })

    let newNextToken = res.data.meta.next_token;
    if(res.data.data.length > 0 && newNextToken){
      res.data.data.forEach(tweet => tweetList.push(tweet.text))
      return await this.getAllTweetsRecursively(ticker, newNextToken, tweetList));
    }

    // no need for "else": the "if" already returns.

    res.data.data.forEach(tweet => tweetList.push(tweet.text))
    let cleanedTweets: string[] = [];
    tweetList.forEach(tweet => {
      if(tweet.startsWith("RT")) return;
      if(!tweet.toLowerCase().includes("$" + ticker)) return;
      cleanedTweets.push(tweet);
    });

    return cleanedTweets
  })
}

    Your code contains async getAllTweetsRecursively(...) { return new Promise(async (resolve, reject) => { ... })) which will now return THREE nested promises, because those async keywords make this:

    getAllTweetsRecursively(...) {
  return new Promise(resolve, reject => {
    return new Promise(async (resolve, reject) => {
      ...
    }))
  });
}

    which because of that async further unrolls to:

    getAllTweetsRecursively(...) {
  return new Promise(resolve, reject => {
    return new Promise((resolve, reject) => {
      return new Promise(resolve, reject => {
        resolve(...)
      })
    }))
  });
}

    this way lies madness =)

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search