I have created an AWS MWAA environment and correctly pointed it to an S3 bucket as well as an "dags" folder inside the S3 bucket.
I have created a simple dag file (python) and added it to my /dags/ directory (configured in my mwaa configurations).
However, when I open the Airflow UI, I do not see my dag available in the UI.
The steps I did to try and troubleshoot:
1- Give permission to the role associated with my MWAA environment to access my S3 Bucket
2- Restart my MWAA environment
3- Set my Airflow UI access to Public Internet instead of Private.
Permission is given to the role to S3FullAccess for read and write and permission is attached to the role.
2
Answers
It often take up to 5 minutes to scanning dag folder on S3 as default. Also check S3 bucket for DAG must be configured to "Block all public access", with Bucket Versioning "enabled".
Finally, you should check the log group of DAGProcessing in cloudwatch to find out any problem occurred with DAG file
Did you figure out the solution? I met with the same issue.