Unable to read bigquery table with JSON/RECORD column type into spark dataframe. ( java.lang.IllegalStateException: Unexpected type: JSON)
we are trying to read a table from Bigquery to spark dataframe. Strucute of the table is Following pyspark code is used for reading the data. from google.oauth2 import service_account from google.cloud import bigquery import json import base64 as bs…