Question 195232 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – botocore.exceptions.NoRegionError: You must specify a region for EmrServerlessCreateApplicationOperator

I am trying to create a emr-serverless application through the EmrServerlessCreateApplicationOperator but I keep facing the error botocore.exceptions.NoRegionError: You must specify a region.

I am passing the region like below:

 create_app = EmrServerlessCreateApplicationOperator(
        task_id="create_spark_app",
        job_type="SPARK",
        release_label="emr-6.6.0",
        config={"aws_access_key_id":args["aws_access_key_id"], 
                "aws_secret_access_key": args["aws_secret_access_key"], 
                "aws_session_token": args["aws_session_token"],
                "region_name":'us-east-1'}
    )

Any help is appreciated. Thank you.

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    I was able to figure it out. I passed them as environment variables.

    os.environ["AWS_ACCESS_KEY_ID"]= YOUR VALUE
os.environ["AWS_SECRET_ACCESS_KEY"]= YOUR VALUE
os.environ["AWS_SESSION_TOKEN"]= YOUR VALUE
os.environ["AWS_DEFAULT_REGION"]= YOUR VALUE

    This worked.


  2. 0

    You’re specifying the region_name attribute as part of config argument to EmrServerlessCreateApplicationOperator. Have you considered specifying the region with AWS connection instead? When the connection ID isn’t specified with operator, it should default to aws_default.
    Refer to this for more context. https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/connections/aws.html#default-connection-ids

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search