Question 317288 in Docker Question posted in
The official documentation can be found here.

Docker not properly setting environment variables via GitHub actions

In my python application I am trying to access/print out an environment variable but it is printing out None.

logger.info(f"OIDC SECRET: {os.getenv('OIDC_CLIENT_SECRET')}")

This is my step/job in github actions

name: Build and push image to harbor
    run: |
      IMAGE_VERSION=$GITHUB_RUN_NUMBER
      IMAGE_WITH_TAG="$IMAGE_NAME:develop.$IMAGE_VERSION"
      docker build 
        -t $IMAGE_WITH_TAG --file "$DOCKERFILE_PATH" 
        --build-arg OIDC_CLIENT_SECRET=${{ secrets.OIDC_CLIENT_SECRET }} 
        --build-arg DB_PWD=${{ secrets.DB_PWD }} 
        --build-arg APP_ENV="dev" .
      docker login TEST.com -u "$DEV_DEPLOY_USENAME" -p "$DEV_DEPLOY_PWD"
      docker push $IMAGE_WITH_TAG
      echo "IMAGE_VERSION=$IMAGE_VERSION" >> $GITHUB_ENV
      echo "PUSHED_IMAGE=$IMAGE_WITH_TAG" >> $GITHUB_ENV
      echo "develop branch"
      echo "NAMESPACE=$PROJECT-dev" >> $GITHUB_ENV
    env:
      DEV_DEPLOY_USENAME: ${{ secrets.DEV_DEPLOY_USENAME }}
      DEV_DEPLOY_PWD: ${{ secrets.DEV_DEPLOY_PWD }}
      PUSHED_IMAGE: ${{ env.PUSHED_IMAGE }}
      BRANCH: ${{ env.BRANCH }}

And below is my docker file.

FROM TEST.com/library/python:3.10-slim as builder

RUN pip install poetry==1.6.1

ENV POETRY_NO_INTERACTION=1 
    POETRY_VIRTUALENVS_IN_PROJECT=1 
    POETRY_VIRTUALENVS_CREATE=1 
    POETRY_CACHE_DIR=/tmp/poetry_cache

WORKDIR /usr/src

ARG OIDC_CLIENT_SECRET
ARG DB_PWD
ARG APP_ENV

ENV OIDC_CLIENT_SECRET=$OIDC_CLIENT_SECRET
ENV DB_PWD=$DB_PWD
ENV APP_ENV=$APP_ENV

RUN echo $OIDC_CLIENT_SECRET
RUN echo $DB_PWD
RUN echo $APP_ENV

COPY pyproject.toml ./

RUN poetry install --without dev --no-root && rm -rf $POETRY_CACHE_DIR

# The runtime image, used to just run the code provided its virtual environment
FROM TEST.com/library/python:3.10-slim as runtime

ARG WORKER_COUNT=1
ENV WORKER_COUNT=${WORKER_COUNT}

RUN mkdir -p /usr/src
WORKDIR /usr/src

ENV VIRTUAL_ENV=/usr/src/.venv 
    PATH="/usr/src:/usr/src/.venv/bin:$PATH"

ENV TZ=America/Chicago
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

COPY --from=builder ${VIRTUAL_ENV} ${VIRTUAL_ENV}

COPY ./src/ /usr/src/
RUN ls -R
EXPOSE 5000

CMD uvicorn --workers $WORKER_COUNT --host 0.0.0.0 --port 5000 main:app

My project deploys successfully, however the environment variables of OIDC_CLIENT_SECRET and DB_PWD are being set to None which is confirmed by my log statement. I double checked names, values all throughout my files including the secrets in github. I even have print statements in my docker which are outputting *** which I assume is still correct, its probably hiding the value.

Is there something I am doing wrong to where my environment variables are not making it to my project?

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    So I finally figured it out. The issue was in the dockerfile. The environment variables and arguments needed to be declared after the runtime like below.

    FROM TEST.com/library/python:3.10-slim as builder

RUN pip install poetry==1.6.1

ENV POETRY_NO_INTERACTION=1 
    POETRY_VIRTUALENVS_IN_PROJECT=1 
    POETRY_VIRTUALENVS_CREATE=1 
    POETRY_CACHE_DIR=/tmp/poetry_cache

WORKDIR /usr/src

ARG OIDC_CLIENT_SECRET
ARG DB_PWD
ARG APP_ENV

ENV OIDC_CLIENT_SECRET=$OIDC_CLIENT_SECRET
ENV DB_PWD=$DB_PWD
ENV APP_ENV=$APP_ENV

RUN echo $OIDC_CLIENT_SECRET
RUN echo $DB_PWD
RUN echo $APP_ENV

COPY pyproject.toml ./

RUN poetry install --without dev --no-root && rm -rf $POETRY_CACHE_DIR

# The runtime image, used to just run the code provided its virtual environment
FROM TEST.com/library/python:3.10-slim as runtime

ARG WORKER_COUNT=1
ENV WORKER_COUNT=${WORKER_COUNT}

ARG OIDC_CLIENT_SECRET
ARG DB_PWD
ARG APP_ENV

ENV OIDC_CLIENT_SECRET=$OIDC_CLIENT_SECRET
ENV DB_PWD=$DB_PWD
ENV APP_ENV=$APP_ENV

RUN mkdir -p /usr/src
WORKDIR /usr/src

ENV VIRTUAL_ENV=/usr/src/.venv 
    PATH="/usr/src:/usr/src/.venv/bin:$PATH"

ENV TZ=America/Chicago
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

COPY --from=builder ${VIRTUAL_ENV} ${VIRTUAL_ENV}

COPY ./src/ /usr/src/
RUN ls -R
EXPOSE 5000

CMD uvicorn --workers $WORKER_COUNT --host 0.0.0.0 --port 5000 main:app

  2. 0

    I’ll do my best to address the issue of environment variables not being accessible in your Python application within the Docker container. Here’s a comprehensive response, incorporating insights from expert feedback and addressing potential causes:

    Understanding the Workflow:

    • GitHub Actions:

      • Secrets are securely stored and injected into build commands.

    • Build arguments (-build-arg) are used to pass secrets to the
      Dockerfile.

    • Dockerfile:

      • ARG instructions receive build arguments.
      • ENV instructions set environment variables within the build context.
      • Multiple stages (builder, runtime) are used for efficiency.

    Potential Issues and Solutions:

    1. Incorrect Usage of ARG and ENV:

      • Fix: Move ENV instructions after FROM runtime to set variables for
        the runtime image.

    2. Environment Variables Not Persisting Between Stages:

      • Fix: Use COPY –from=builder to copy variables from the builder stage
        to the runtime stage.

    3. Virtual Environment Isolation:

    • Fix: Activate the virtual environment in your Python script before
      accessing variables.

    1. Typos and Case Sensitivity:

      • Fix: Meticulously check for consistent variable names throughout.

    solution:

     FROM TEST.com/library/python:3.10-slim as builder
  ... (build instructions)

  FROM TEST.com/library/python:3.10-slim as runtime
  ... (runtime setup)

  COPY --from=builder /usr/src/.venv /usr/src/.venv
  COPY --from=builder /usr/src/OIDC_CLIENT_SECRET /usr/src/OIDC_CLIENT_SECRET
  COPY --from=builder /usr/src/DB_PWD /usr/src/DB_PWD

  WORKDIR /usr/src

  ENV OIDC_CLIENT_SECRET=$(cat /usr/src/OIDC_CLIENT_SECRET)
  ENV DB_PWD=$(cat /usr/src/DB_PWD)

  CMD uvicorn --workers $WORKER_COUNT --host 0.0.0.0 --port 5000 main:app

    Remember:

    • Commit and push changes to GitHub Actions to trigger a new build.
    • Rebuild and redeploy your application to test the fixes.
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search