Question 264658 in Docker Question posted in
The official documentation can be found here.

Docker – Run another python script within a bash script

I am trying to run another Python script(service_scheduler.py) along with the one which runs the app server( main.py) which is actually a scheduler. It will schedule a job and it will be called at a regular time interval. The problem is when I add this to my bash script neither job gets scheduled nor the app server runs.

Here is my bash script(svc_invoke.sh):

#!/bin/bash
cd /opt
/opt/env/bin/python3.11 main.py &
/opt/env/bin/python3.11 /api_topics/services/service_scheduler.py &
status=$?
if [ $status -ne 0 ]; then
    echo "Failed to start python process: $status"
    exit $status
else
    echo "Started python $status"
fi

This one works, app server goes up and running:

#!/bin/bash

cd /opt
/opt/env/bin/python3.11 main.py
status=$?
if [ $status -ne 0 ]; then
    echo "Failed to start python process: $status"
    exit $status
else
    echo "Started python $status"
fi

And here is my Dockerfile:

FROM python:3.11-slim

RUN apt-get update && 
    apt-get install -y gcc && 
    apt-get clean;

# Add files
COPY topic_modelling/ /opt
COPY svc_invoke.sh /svc_invoke.sh

RUN chmod -R 755 /svc_invoke.sh

RUN cd /opt
RUN python3.11 -m venv /opt/env
RUN /opt/env/bin/python3.11 -m pip install --upgrade pip
RUN /opt/env/bin/python3.11 -m pip install wheel
RUN /opt/env/bin/python3.11 -m pip install -r /opt/requirements.txt

ENV PYTHONUNBUFFERED=1

# Expose Port for the continuous learning
EXPOSE 8000

# Run the service
CMD ./svc_invoke.sh

I am new to Docker and shell scripting and I am stuck here. Any help will be highly appreciated.

Thanks

Tags:

2

Answers


  1. 0

    Try this:

    #!/bin/bash
cd /opt
/opt/env/bin/python3.11 main.py & mainpid=$!
/opt/env/bin/python3.11 /api_topics/services/service_scheduler.py & schedpid=$!

status=0
wait $mainpid ; (( status |= $? ))
wait $schedpid ; (( status |= $? ))

if [ $status -ne 0 ]; then
    echo "Failed to start python process: $status"
    exit $status
else
    echo "Started python $status"
fi

    When you move the python scripts into the background with & the main script is allowed to exit before they finish, which in turn would allow Docker to exit. If you want them both to have an opportunity to finish running then you need to wait for them.

    Login or Signup to reply.
  2. 0

    Normally a Docker container only runs a single process. If you need to run multiple processes, usually the easiest path is to launch multiple containers from the same image. It is important that you launch the main container process as a foreground process: in your first script you launch two background processes and then the script completes, and when it completes the container exits.

    I’d recommend setting the image’s CMD to the most common thing you expect the container to do. You do not need a wrapper script to report its exit code (this can also interfere with some things like docker stop).

    CMD /opt/env/bin/python3.11 main.py

    # run the main script
docker run -d --name main your-image

# run the scheduler
docker run -d --name scheduler your-image 
  /opt/env/bin/python3.11 /api_topics/services/service_scheduler.py

    (Based only on the Dockerfile you’ve shown, the latter container will probably fail because there is no /api_topics/ directory in the image. Note that this won’t affect the main container running, and you can debug and restart the scheduler container separately.)

    There are two more changes you can make that will make this a little easier to run. You shouldn’t normally need to explicitly mention the python interpreter on the command line. Your Python scripts should start with a "shebang" line, usually

    #!/usr/bin/env python3

    that tells the system where to find the interpreter. If you also make the script(s) executable chmod +x main.py (commit the permission change to source control) then you can just run ./main.py and it will use "the default" Python.

    The other change is to make "the default" Python have your dependencies. In your setup you can make the virtual environment be part of the command search path, so you’ll find the python3 in the virtual environment

    ENV PATH=/opt/env/bin:$PATH
CMD ./main.py

    However, a Docker image is already an isolated environment and there’s not risk of the "system" Python interfering with other applications. In a Docker context it’s common to not use a virtual environment at all and just install your dependencies in the "system" Python, but still isolated in this image.

    # using the "system" pip/python3
RUN pip install wheel
RUN pip install -r requirements.txt
CMD ./main.py
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search