Question 110026 in Json Question posted in
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

Json – Dataflow Template Metadata regex definition with escaped chars

I am creating a Dataflow Flex template and I would like to define the input parameters as documented here
https://cloud.google.com/dataflow/docs/guides/templates/using-flex-templates#metadata

I have configured my template with the following JSON

{
  "name": "My name",
  "description": "my long description",
  "parameters": [
    {
      "name": "outputTopic",
      "label": "Pub/Sub ingest topic",
      "helpText": "Pub/Sub topic to publish result"
    },
    {
      "name": "bigQueryProject",
      "label": "BigQuery project",
      "helpText": "BigQuery google project"
    },
    {
      "name": "tempLocation",
      "label": "GCS Temp Location",
      "helpText": "GCS Location for storing temporary files",
      "regexes": [
        "gs://.*"
      ]
    },
    {
      "name": "startDate",
      "label": "Start Date",
      "isOptional": true,
      "helpText": "Start date in the format YYYY-MM-DD or use 'YESTERDAY' as default",
      "regexes": [
        "^TODAY$|^YESTERDAY$|^\d{4}-\d{2}-\d{2}$"
      ]
    }
    }

It works properly as I can see all parameters properly defined if I use the Google Console

screenshot console

However, when I execute the Job with all validated parameters, the request fails for an invalid REGEX defined

"(fb57dedae5c9fead): Template metadata contains invalid POSIX regex '^TODAY$|^YESTERDAY$|^\d{4}-\d{2}-\d{2}$': invalid escape sequence: \d in \d.

it Looks Like that the double escaping in the regex needed to create a valid JSON file, is not well interpreted by Dataflow.
Thanks in advance

Tags:

2

Answers


  1. 0

    I think your | is not being interpreted as you expect.

    Can you try using this format? This is what I see being used often:

    ^(TODAY|YESTERDAY|\d{4}-\d{2}-\d{2})$

    Login or Signup to reply.
  2. 0

    The error you are facing is due to Invalid RegEx. As I mentioned in the comment you can solve this by using \\, (\\d) .

    Based on my understanding, you need to use \\, this will produce \ in the string and in the RegEx.You can understand this more by using the RegEx validator.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search