Question 108930 in Json Question posted in
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

Json – How to remove null values while extracting data with jq?

Hi I am having a kubernets audit log file with . The log file has data as json records. I would like to parse the filter out the records.

The following is a sample couple of records in the file

{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"60cc3bf1-a04e-4db3-a343-98aaaea8c4a5","stage":"ResponseComplete","requestURI":"/api/v1/serviceaccounts?limit=500u0026resourceVersion=0","verb":"list","user":{"username":"system:apiserver","uid":"7cded9c8-a35d-4e66-adf1-162ce37d5868","groups":["system:masters"]},"sourceIPs":["::1"],"userAgent":"kube-apiserver/v1.24.12 (linux/amd64) kubernetes/ef70d26","objectRef":{"resource":"serviceaccounts","apiVersion":"v1"},"responseStatus":{"metadata":{},"code":200},"requestReceivedTimestamp":"2023-04-06T15:10:46.594135Z","stageTimestamp":"2023-04-06T15:10:46.595016Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":""}}{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"1af73bde-3a0f-437d-a468-49da772d619d","stage":"ResponseComplete","requestURI":"/apis/batch/v1/namespaces/restricted-namespace/jobs?fieldManager=helm","verb":"create","user":{"username":"kubernetes-admin","groups":["system:masters","system:authenticated"]},"sourceIPs":["172.19.0.1"],"userAgent":"Go-http-client/2.0","objectRef":{"resource":"jobs","namespace":"restricted-namespace","name":"gateway-certgen","apiGroup":"batch","apiVersion":"v1"},"responseStatus":{"metadata":{},"code":201},"requestReceivedTimestamp":"2023-04-06T15:14:02.625749Z","stageTimestamp":"2023-04-06T15:14:02.632035Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"","pod-security.kubernetes.io/audit-violations":"would violate PodSecurity "restricted:v1.24": allowPrivilegeEscalation != false (container "certgen" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "certgen" must set securityContext.capabilities.drop=["ALL"]), seccompProfile (pod or container "certgen" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")"}}

I would like to filter out those records and print the values of the following field from each of the records.

.annotations.pod-security.kubernetes.io/audit-violations

i am using this command,

 cat kube-apiserver-audit.log | jq '.annotations."pod-security.kubernetes.io/audit-violations"'

however it gives the followign output

null
null
"would violate PodSecurity "restricted:v1.24": allowPrivilegeEscalation != false (container "certgen" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "certgen" must set securityContext.capabilities.drop=["ALL"]), seccompProfile (pod or container "certgen" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")"
null
null
null
null

any idea how can i remove the null values from the jq output ?
thank you

Tags:

2

Answers


  1. 0

    You can use select to filter.

    jq -r '.annotations."pod-security.kubernetes.io/audit-violations" | select(. != null)'
    Login or Signup to reply.
  2. 0

    The values filter (see the manual) does exactly that, filtering out nulls while keeping the "values":

    .annotations."pod-security.kubernetes.io/audit-violations" | values

    "would violate PodSecurity "restricted:v1.24": allowPrivilegeEscalation != false (container "certgen" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "certgen" must set securityContext.capabilities.drop=["ALL"]), seccompProfile (pod or container "certgen" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")"

    Demo

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search