skip to Main Content

I’m trying to run a python script inside a Docker container every 2 minutes. To achieve this, I am using a cron job. When I run the Python script on the host machine, it executes perfectly, but inside the docker container I keep getting

ModuleNotFoundError: No module named 'requests'

The container builds perfectly fine, and the requirements are installed correctly according to the build log. My Dockerfile and cron job are as follows:

Dockerfile


    FROM python:latest 
    WORKDIR /root 
    COPY *.py . 
    COPY requirements.txt . 
    COPY cronjob.txt . 
    RUN pip install -r requirements.txt 
    RUN apt update 
    RUN apt install cron -y 
    RUN crontab -l | { cat; cat cronjob.txt; } | crontab -
    ENV PYTHONPATH "$PYTHONPATH:/usr/local/lib/python3.11/site-packages"
    CMD cron -f

cronjob.txt
*/2 * * * * python3 runSync.py > /proc/1/fd/1 2>/proc/1/fd/2

runSync.py

import requests


def run_sync(network, ips, port, endpoint):
    for ip in ips:
        out = ""
        try:
            url = "http://" + network.format(ip=ip, port=port) + endpoint
            print("Resolving " + url + "...")
            res = requests.get(url)
            if res.status_code != 200:
                raise ValueError("Unexpected status code: {code}".format(code=res.status_code))
            print(res)
            print("Success!")
        # TODO: Better error handling
        except requests.exceptions.Timeout:
            out = "Connection to server timed out."
            break
        except requests.exceptions.ConnectionError:
            out = "Unable to connect to server, please check internet connection."
            break
        except requests.exceptions.InvalidURL:
            out = "Malformed URL ", url, " please check and try again."
            break
        except KeyboardInterrupt:
            out = "Stopping..."
            break
        except ValueError as e:
            out = "Encountered error: "{error}"".format(error=e.args[0])
            break
    if out != "":
        return out
    else:
        return "Success"


if __name__ == '__main__':
    defaultNetwork = "192.168.33.{ip:n}:{port:n}"
    defaultPort = 8080
    defaultEndpoint = "/api/system/sync"
    defaultIPs = [29, 28, 12]
    out = run_sync(defaultNetwork, defaultIPs, defaultPort, defaultEndpoint)
    if out != "Success":
        print(out)
        exit(1)
    else:
        exit(0)

requirements.txt

requests==2.28.1

I’ve tried manually setting the PYTHONPATH (as seen in Dockerfile), as well as changing the application root to a couple of different options. I should be seeing the output of runSync.py in the docker logs, but instead I just get ModuleNotFoundError: No module named 'requests'.

3

Answers


  1. The python:latest image comes with both Python 3.9 (because Debian) and 3.11, and cron environment seems to use the former. Specifying the full path to the python executable in the cronjob.txt should fix it:

    */2 * * * * /usr/local/bin/python3 runSync.py > /proc/1/fd/1 2>/proc/1/fd/2
    

    or

    */2 * * * * /usr/local/bin/python3.11 runSync.py > /proc/1/fd/1 2>/proc/1/fd/2
    

    You don’t need to mess with the PYTHONPATH environment in the Dockerfile after this change.

    Login or Signup to reply.
  2. There’s no need for using cron

    Try apscheduler, it’s a python module for shceduling tasks:

    https://pypi.org/project/APScheduler/

    short example:

    from apscheduler.schedulers.blocking import BlockingScheduler
    
    def some_job():
        print("Doing something...")
    
    scheduler = BlockingScheduler()
    scheduler.add_job(some_job,"interval",minutes=2)
    
    try:
        scheduler.start()
    except (KeyboardInterrupt, SystemExit):
        pass
    
    Login or Signup to reply.
  3. If you look at /etc/crontab, it has :

    PATH=/usr/local/bin:/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin
    

    You can put above setting in your cronjob.txt, then you don’t have to use absolute path for python3.

    Also, above setting make sure the PATH is the same on the command line and in crontab, this way it avoids problems for other commands as well.

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search