Question 177408 in Json Question posted in
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

Can JoltTransformJson in Apache NiFi delete a flowfile/object with a null field?

How can I transform(remove in this case) flowfile if one of the fields of the object is null using JoltTransformJson processor in nifi

here below is my data.

emp.json:

[
  {
    "Name" : "john Smith",
    "DOB" : "2000-07-14",
    "Salary" : 16000.0
  },
  {
    "Name" : "Sara Jacob",
    "DOB" : "2000-07-14",
    "Salary" : 12000.0
  },
  {
    "Name" : "Peter John",
    "DOB" : null,
    "Salary" : 20000.0
  },
  {
    "Name" : "Lia Poul",
    "DOB" : "2000-07-14",
    "Salary" : 18000.0
  }
]

I used SplitJSON processor to split the JSON File into multiple -separate FlowFiles and i connected the downstream with the JoltTransformJson processor. Next, I configured JoltTransformJson as below:
Jolt Transformation DSL: chain
Jolt Specification:

[
  {
    "operation": "remove",
    "spec": {
      "rowsToRemove": {
        "*": {
          "DOB": {
            "$null": ""
          }
        }
      }
    }
  }
]

But this expression only removed the DOB field, not the whole object and Flowfile.

Tags:

2

Answers


  1. 0

    Yes you can use JoltTransformJSON processor with the following transformation specs which contain notNull function that pricipally resolves the issue such as

    [
  { // produce a new attribute, namely "Nl" to check whether the DOB is null
    "operation": "modify-overwrite-beta",
    "spec": {
      "*": {
        "Nl": ["=notNull(@(1,DOB))", "NuLl"]
      }
    }
  },
  { // the following conditional spontaneously removes the object having "Nl" : "NuLl" pair
    "operation": "shift",
    "spec": {
      "*": {
        "Nl": {
          "NuLl": {
            "@2": "@3,Nl"
          },
          "*": {
            "@2": ""
          }
        }
      }
    }
  },
  { // get rid of the remaining "Nl" attributes
    "operation": "remove",
    "spec": {
      "*": {
        "Nl": ""
      }
    }
  }
]

    the demo on the site http://jolt-demo.appspot.com/ is

    enter image description here

    Login or Signup to reply.
  2. 0 
    [
  {
    "operation": "shift",
    "spec": {
      "*": {
        "DOB": {
          "null": {
            "@2": "@3.&"
          },
          "*": {
            "@2": ""
          }
        }
      }
    }
  }
]
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search