Question 109430 in Json Question posted in
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

Json – Run parallel instances of same program on Pycharm

I have a program named Scanner.py which loads parameters from a configuration file called Config.json- is there a way in which I can run two instances of this same file parallely on Pycharm with different sets of configuration passed through Config.json.

Example: this is the Json file which is read into the Python file of Pycharm. I want various combinations of this to be run without one after the other, rather in parallel

"Traded Instrument": "STOCK",
"Start Date": "2019/01/01",
"End Date": "2022/12/20",
"Trade Start": "09:16",
"Trade End": "15:29",
"Trade End Scalp": "09:16",

I do not have clue how to get started with this, other than just using different machines, but I would need more than 3/4 instances at the same time by running Scanner.py

Tags:

2

Answers


  1. 0

    I would suggest using multiprocessing. You can create a function that accepts a configuration dict and run it with different configs.

    from multiprocessing import Pool

def runner(config: dict):
    # your logic
    ...

if __name__ == '__main__':
    configs = [{...}, {...}, {...}]
    with Pool(len(configs)) as p:
        p.map(runner, configs)
    Login or Signup to reply.
  2. 0

    I would say in your case it’s better to make a multi-thread wrapper, simply because threading uses the same memory space, and no awaiting or such is required, whereas multi-processing uses a different memory space.

    from threading import Thread

def async(func: object):
    """ This is a decorator for async functions """
    def asyncFunction(*args, **kwargs):
        """ This is the wrapper function that will be returned """
        thread: Thread = Thread(target=func, args=args, kwargs=kwargs)
        thread.start()
        return thread
    return asyncFunction

@async
def runner(config: dict):
    ...
    
if __name__ == "__main__":
    configs = [{...}, {...}, {...}]
    for i in range(len(configs)):
        runner(configs[i])

    NOTE: there is a drawback to using threading, where it is not memory safe (mostly), and you cannot interrupt the program when it is doing the processing, unless you make a sort of check inside the function.

    The program will look like this if you want it to work with keyboard interrupts:

    from threading import Thread

KeyboardInterruptDetected = False # This is a global variable that will be used to stop the program

def async(func: object):
    """ This is a decorator for async functions """
    def asyncFunction(*args, **kwargs):
        """ This is the wrapper function that will be returned """
        thread: Thread = Thread(target=func, args=args, kwargs=kwargs)
        thread.start()
        return thread
    return asyncFunction

@async
def runner(config: dict):
    if KeyboardInterruptDetected:
        return
    ... # Do something.

if __name__ == "__main__":
    configs = [{...}, {...}, {...}] # This is a list of the config dictionaries
    try:
        for config in configs:
            runner(config)
    except KeyboardInterrupt:
        KeyboardInterruptDetected = True
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search