I am trying to connect to sql server from databricks using pyodbc facing below issue import pyodbc sql_server_url='server_url' sqlDB='db-dev' SQL_DB_USER='user' SQL_DB_PASS='123' conn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER=' + sql_server_url +';DATABASE=' + sqlDB + ';UID='+SQL_DB_USER+';PWD='+SQL_DB_PASS) Error: ('01000', "[01000] [unixODBC][Driver…
I am trying to import python files from databricks notebook. I am able to import the function but it keep giving me error - NameError: name 'col' is not defined. I have a file /Workspace/Shared/Common/common.py. The code in that file…
Hi It seems I could not load data correctly from DBFS is Databricks using Auto loader, at least it is not displaying the data -- 'Query returned no results'. Any help is welcome!
I am trying to write to Snowflake using Snowpark from Azure Databricks and I get the following error. This error does not seem related to Unity Catalog but some configuration. I have made sure I have imported required libraries. df_new.write.mode("append").saveAsTable("MyTableName")…
The following bicep deployment has suddenly started failing and I do not understand why var managedResourceGroupName = 'rg-metadata-${dataBricksName}' resource databrickworkspace 'Microsoft.Databricks/workspaces@2018-04-01' = { name: dataBrickName location: location sku: { name: dataBrickPricingTier } properties: { managedResourceGroupId: managedResourceGroup.id parameters: { enableNoPublicIp: {…
I am facing an issue where the expected date parition folder should be named in format date=yyyymmdd, but instead writing as - Sometimes for each parquet file created in delta path, it's creating a seperate folder. I don't see any…
I am currently developing a Python script to retrieve a comprehensive list of all the jobs that were executed yesterday. However, I'm encountering an issue with the script's pagination mechanism using tokens. Despite my attempts to loop through the pagination…
I have these following inputs. 1.Endpoint 2.Database 3.Collections Based on these inputs needs to establish a connection with Azure Cosmos Db from databricks Tried this approach: installed importted com.azure.cosmos.spark maven package in databricks cluster with the DBR of 11.3LTS and…
I have this tableConfig.json inside a ADLS container based location . It has table specific details. { "tableName":"employee", "databaseName": "dbo", "location" : "/mnt/clean/demo", "colsList" : ["emp_id","emp_name","emp_city"] } Now I want to read that tableConfig.json in azure databricks python notebooks. and…
com.databricks.common.client.DatabricksServiceHttpClientException: PERMISSION_DENIED: Invalid permissions on the specified KeyVault https://azkv*.vault.azure.net/. Wrapped Message: Status code 403, {"error":{"code":"Forbidden","message":"Caller is not authorized to perform action on resource.rnIf role assignments, deny assignments or role definitions were changed recently, please observe propagation time.rnCaller: name=AzureDatabricks;appid=2vf8v4a6-3304-4ab8-85cb-cd0e6f879c1d;oid=4e924d0e-ad49-4acc-baec-6d612e920502;iss=https://sts.windows.net/552af62d-5878-4131-8695-1e87b0f89945/rnAction: 'Microsoft.KeyVault/vaults/secrets/getSecret/action'rnResource: '/subscriptions/6b60a61d-3e92-4501-8794-fd7725066113/resourcegroups/streamproject/providers/microsoft.keyvault/vaults/azkv*/secrets/clientsecret'rnAssignment:…