A response.json() returns a JSON in the following format: { "workbooks": [ { "name": "WORKBOOK_A", "embeddedDatasources": [ { "upstreamTables": [ {"name": "WORKBOOK_A_TABLE_A"}]}, { "upstreamTables": [ {"name": "WORKBOOK_A_TABLE_B"}, {"name": "WORKBOOK_A_TABLE_C"}]}, { "upstreamTables": []}]}, { "name": "WORKBOOK_B", "embeddedDatasources": [ { "upstreamTables": […
I have a json that is three levels deep. I want to flatten it into a dataframe that has five columns. id name code level parent_id So: The part I struggle with is that I can extract each nested item,…
I have a JSON file to be imported into R. Here is a section of the data: {"data":[{"British Columbia":"BC","BC":"4.63"}, {"Alberta":"AB","AB":"4.15"}, {"Ontario":"ON","ON":"13.6"}]} I need to turn this data into a dataframe, but the import is producing many NA values. I tried…
I am trying to read data from mongodb in batch My compute cluster configurations are: Spark Version > 3.5.0 Scala Version > 2.12 I have installed mongodb connector library version is(through maven): org.mongodb.spark:mongo-spark-connector_2.12:10.4.0 I have my schemas defined in a…
Starting with a DataFrame that has a form such as this df = pl.DataFrame([{"SkuId":1}]) shape: (1, 1) ┌───────┐ │ SkuId │ │ --- │ │ i64 │ ╞═══════╡ │ 1 │ └───────┘ How can I write it to a JSON…
today i have a modification from another question that i did yesterday. The challenge is: How convert a CSV into a hierarchical json but the keys and values are different. It must be done with PYTHON (pandas and json library)…
Today I have a challenge at my school, it is convert a CSV file to a JSON file. This CSV has a table structure (which means contains the information from an oracle table in this example). So i have to…
I've used most of methods to load a json and the results I got didn't meet my needs. When I load the file with pandas read_json, I get a dataframe with the index and all fields in a single line,…
Dependency DAG Description Pretty straight forward, basically, I am reading some parquet files from disk using polars which are the source of data. Doing some moderately heavy duty processing (a few million rows) to generate an intermediate data frame, then…
I want to build a scrapper. And I wrote this code, it keeps on running into error. I would be grateful if anyone can help. import requests import pandas as pd def fetch_json(api_link, timeout=10): try: response = requests.get(api_link, timeout=timeout) if…