Question 84553 in Docker Question posted in
The official documentation can be found here.

How to modify 'docker-compose-local.yml' for it to install all requirements (needed to run Amazon MWAA environment locally)

I am running aws-mwaa-local-runner in order to run a local Apache Airflow environment (in Docker for Windows).

However, after creating the container using ./mwaa-local-env start, I repeatedly get the Broken DAG ModuleNotFoundError‘ However, when I check my /docker/config/requirements.txt file (see here, although my file has a few more requirements that I need in it). When I compare my /docker/config/requirements.txt file with the output of pip freeze command run in Airflow container, I can see those requirements I need for my DAGs are missing.

I tried to pip install my other requirements in Airflow container but to no avail.

Is there a way to modify docker-compose-local.yml file so it installs all of my requirements.txt when creating the container (i.e. running the Airflow)?

Is there maybe something I might be missing? Any help or suggestion would be greatly appreciated.

Tags:

2

Answers


  1. 0

    Look at this: https://github.com/aws/aws-mwaa-local-runner . You should run the requirements file located in dags, locally.

    pip install -r requirements.txt

    Login or Signup to reply.
  2. 0

    Add your extra requirements to dags/requirements.txt, not docker/config/requirements.txt. The former is installed every time you start the service, but the latter is only installed when you build or rebuild the image.

    Additionally, keeping your added requirements separate is important because you will need to upload the list to your MWAA environment.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search