Amazon Web Services Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – AWS Glue Job Not Running / Triggering by Terraform

I have been trying to orchestrate AWS Glue to read and write to S3 by Terraform.

After configuration in main.tf, the job needs manual trigger over UI or AWS CLI to get triggered.

Thank you

tf file

The Glue Job should run along by Terraform Script.

Tried to to add triggers in main.tf but did not work.

Tags:

2

Answers


  1. 0

    It doesn’t look like terraform allows to start a glue job natively. You will need to use something else to trigger it, like aws cli.

    You can then run:

    aws glue start-job-run --job-name example_job_2

    If you really want this integrated with terraform then use local-exec resource:

    resource "aws_glue_job" "Job_Glue" {
  name = "example_job_2"
  role_arn = "arn:aws:iam::123456789:role/service-role/AWSGlueServiceRole-crawl-mar"
  command {
    name = "Job 1"
    script_location = "s3://sample-csv/glue_pyspark.py"
  }
  provisioner "local-exec" {
    command = "aws glue start-job-run --job-name example_job_2"
  }
}
    Login or Signup to reply.
  2. 0

    You need to configure a glue trigger in main.tf

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search