Question 81029 in Docker Question posted in
The official documentation can be found here.

No git inside apache airflow docker container

I need help regarding git inside my latest airflow, I am using this image https://hub.docker.com/layers/apache/airflow/latest/images/sha256-3e0ab13f9525974867cbb612ff632d332b838b05fac4e59bf83f700827744d7d?context=explore and running a container using that but when I am trying to execute this below task I am getting errors

current snippets

# Create a BashOperator to sync the Git repositories
git_sync_task = BashOperator(
    task_id='git_sync_task',
    bash_command='cd /opt/airflow/service1 && git pull origin main',
    dag=dag,
)

I am getting this error,

[2023-01-04, 13:09:30 UTC] {subprocess.py:75} INFO – Running command: [‘/bin/bash’, ‘-c’, ‘cd /opt/***/service1 && git pull origin main’] [2023-01-04, 13:09:30 UTC] {subprocess.py:86} INFO – Output:
[2023-01-04, 13:09:30 UTC] {subprocess.py:93} INFO – /bin/bash: line 1: git: command not found [2023-01-04, 13:09:30 UTC]{subprocess.py:97} INFO – Command exited with return code 127
[2023-01-04, 13:09:30 UTC] {taskinstance.py:1772} ERROR – Task failed with exception

when I am trying to install git manually inside airflow_airflow-webserver_1 container I am also getting permission-related error,

default@3df7bafd3a37:/opt/airflow$ apt-get install git E: Could not
open lock file /var/lib/dpkg/lock-frontend – open (13: Permission
denied) E: Unable to acquire the dpkg frontend lock
(/var/lib/dpkg/lock-frontend), are you root?

How can I fix that git error?

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    This is what I did finally to build a custom image because git is not present inside the latest apache airflow image. Another point to note here is we need the root permission and airflow permission to install some services. So this finally works for me-

    FROM apache/airflow:latest
USER root           <----------------------- see here
RUN apt-get update 
    && apt-get install -y --no-install-recommends 
    vim git curl unzip groff less 
    && apt-get autoremove -yqq --purge 
    && apt-get clean 
    && rm -rf /var/lib/apt/lists/*

COPY requirements.txt /requirements.txt
# AWS CLI installation
RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
RUN unzip awscliv2.zip && ./aws/install

USER airflow      <----------------------- see here
# Required package installation
RUN pip install --user --upgrade pip
RUN pip install --no-cache-dir --user -r /requirements.txt

# Install Poetry
RUN pip install poetry

  2. 0

    Not that the apache/airflow Docker images run under a non-root user. You can set your Docker image to run as root before installing git to avoid permission issues, but take under consideration security issues etc

    USER root
RUN apt-get update
RUN apt-get -y install git
...
...
# change back to base image user
USER 50000
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search