skip to Main Content

Writing to Azure Blob Storage from local spark environment

I've been going round in circles trying to write to a blob storage account in azure. Currently i'm creating a spark session with the following setup: spark = SparkSession.builder .appName("Azure Blob Storage Access") .config("spark.jars.packages", "org.apache.hadoop:hadoop-azure:3.3.1 ,com.microsoft.azure:azure-storage-blob:11.0.1 ,org.apache.hadoop:hadoop-azure:3.4.0 ,org.apache.hadoop:hadoop-azure:3.3.1 ,org.eclipse.jetty:jetty-util:11.0.7 ,org.apache.hadoop.thirdparty:hadoop-shaded-guava:1.1.1…

VIEW QUESTION
Back To Top
Search