skip to Main Content

I have two services, service A (node) and service B (flask)
there is an endpoint on serivce A which calls endpoint of service B like this:

const response = await axios.get(
              endpoint_url,
              {
                params: {
                  library_id: req.query.library_id,
                },
              }
            );

On service B this goes to a library update route which updates all the books for that library in a for loop which takes some time. I want service B to respond immediately and do the computation in the background. There were some libraries which were suggested like asyncio, threading, celery but I don’t know which is the most suitable for me and I went with threading because that seemed the simplest. I implemented the below code however the "testing" is not printed immediately after the thread is started and the entire process function is executed before the response is sent back to node app. What am I doing wrong here?

Service B endpoint:

import threading
from src.Library import Library

@bp_csv.route('update-library', methods=['GET'])
def get_library():
        library_id = request.args.get('library_id')

        if not library_id:
            raise ValueError("Missing query parameter 'library_id' ")
        

        LIB = Library(library_id)
        thread = threading.Thread(target=LIB.process()).start()
        print("testing")
        
        status_code = 202
        return_message = "Updating library"
        response = make_response(
            jsonify({"message": str(return_message)}), status_code)
        response.headers["Content-Type"] = "application/json"
        return response

2

Answers


  1. There are many ways to solve this problem. But I think the easiest way is to save that ID in a cache-like system as an array (For example Redis) And return the response.
    In another function, you get the values from that array (like a FIFO in Redis) and process them.
    in this case, you can avoid using the same function for doing multiple works, and also you can scale it later if you see fit. also using Redis is very easy, a 30-minute crash course can get you up and running.

    Login or Signup to reply.
  2. When you fork a thread to run function, It cannot run on time or run after some request that come from client. If you want to run that function after response, you can use decorate after_request. Example:

    @bp_csv.after_request
    def callback_library(response):
        if request.endpoint in ["bp_csv.get_library"]:
            library_id = request.args.get('library_id')
            LIB = Library(library_id)
            LIB.process()
        return response
    
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search