Question 95493 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

EcsCreateClusterOperator import error in MWAA (Apache Airflow) – Amazon web services

I am trying to use class EcsCreateClusterOperator in MWAA but I get the following error:

Broken DAG: [/usr/local/airflow/dags/etl_basic_ecs/etl_ecs_basic_dag.py] Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/airflow/dags/etl_basic_ecs/etl_ecs_basic_dag.py", line 5, in <module>
    from airflow.providers.amazon.aws.operators.ecs import (
ImportError: cannot import name 'EcsCreateClusterOperator' from 'airflow.providers.amazon.aws.operators.ecs' (/usr/local/lib/python3.7/site-packages/airflow/providers/amazon/aws/operators/ecs.py)

This library works perfectly in my local environment installed with:

pip install apache-airflow-providers-amazon==5.0.0

I also added this library (version 5.0.0) to my requirements.txt file before deploying to MWAA, in the UI seems like the provider is installed:

Screenshot of installed Amazon Provider 5.0.0

The log of the update shows that the library is downloaded:

Installing library log

Here is the documentation of the class: https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/_api/airflow/providers/amazon/aws/operators/ecs/index.html this class was introduced recently (version 5.0.0).

By default MWAA (Airflow 2.2.2) has the Amazon Provider with an older version (2.4), seems like Airflow is really loading the older version of the library.

Any help would be appreciated, thank you.

Tags:

2

Answers


  1. 0

    MWAA with Airflow version 2.2.2 is constrained to apache-airflow-providers-amazon==2.4.0. See the constraints file for more details.

    References:

    1. Airflow Versions (MWAA)
    2. Constraints file
    Login or Signup to reply.
  2. 0

    I had the same issue with EksCreateClusterOperator:

    from airflow.providers.amazon.aws.operators.eks import (
    ImportError: cannot import name ‘EksCreateClusterOperator’ from ‘airflow.providers.amazon.aws.operators.eks’ (/usr/local/lib/python3.7/site-packages/airflow/providers/amazon/aws/operators/eks.py)

    Looks like is case sensitive. So instead of using EcsCreateClusterOperator change it to ECSCreateClusterOperator (capital ECS) how it is showing on airflow github. That fixed my issue.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search