Question 104782 in Azure Question posted in
The official documentation to get you started can be found here.

ingest JSON-STAT from RESTapi to csv on Azure Data Factory

I want to make a request to an RESTapi of the Norwegian statistics bureau.
According to their api I need to make a POST request (thats not usual but that is ok) to get the tables that I need. In the body of the POST request I can specify with kind of table I want.

What I get is a JSON-STAT file. I need some help about how to handle it on Azure Data Factory. I read on documentation that ADF supports just JSON when using REST API. Does it mean that it also supports JSON-STAT ? If so How can I handle it on the activity ( source/sink ).

Thanks in advance,
Nunotrt

Tags:

2

Answers


  1. 0

    Here is the sample process of JSON-STAT file ingest from REST API . Create a new REST dataset. enter image description here Create Linked services to the REST API. enter image description here Provide REST dataset as the source for the copy activity and also with the POST request method, providing the request body({ "query": [ { "code": "Region", "selection": { "filter": "agg:KommSummer", "values": [ "K-3006" ] } } ], "response": { "format": "json-stat2" } })

    enter image description here Create a linked services for sink dataset. enter image description here Create a file path for the sink dataset to get the file into the location. enter image description here enter image description here enter image description here Result we get as per the expectations. enter image description here

    Login or Signup to reply.
  2. 0

    If you want to convert a json-stat file to csv or parquet, it is an alternative to you use a python code as a part of your Data Factory job.

    The python library pyjstat makes it easy to convert json-stat to a DataFrame, and from DataFrame to any other file format.

    Here are an example with json-stat 1.2

    import pandas as pd
from pyjstat import pyjstat as pyj

# url
file_ssb = 'https://data.ssb.no/api/v0/dataset/1102.json'

# Upload data to memory
dataset = pyj.Dataset.read(file_ssb)

# Write json-stat to dataframe
df = dataset.write('dataframe')
print(df)

# Save as csv
df.to_csv('../Data/test_jsonstat.csv')
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search