I am having issues with a series of pipelines that build our data platform Spark databases hosted in Azure Synapse. The pipelines host dataflows which have 'recreate table' enabled. The dataflows extract data and are supposed to recreate the tables…
I need remove the all comments in a SQL view created in Azure SQL Data Warehouse. Create the query without comments (/* */ ; -- , ...); regular expression functions not working Query: CREATE VIEW [dimension].[v_dim_customer] AS SELECT -- primary…
Inside a Azure synapse pipeline, I need to find and compare the row count of the CSV file with summary file stored in Azure Blob storage. If the comparison fails then pipeline will stop, if the counts match then can…
2024-07-09T23:11:19.7718791Z Complete getting Artifacts From Template 2024-07-09T23:11:19.7719155Z Start deploying artifacts from the template. 2024-07-09T23:11:19.7722609Z Deploy dev_keyVault_linkedService of type linkedService 2024-07-09T23:11:19.9503411Z For Artifact: dev_keyVault_linkedService: ArtifactDeploymentTask status: 403; status message: Forbidden 2024-07-09T23:11:19.9506451Z Failed 2024-07-09T23:11:19.9507649Z deploy operation failed 2024-07-09T23:11:19.9509658Z An error occurred during…
I have created a stored procedure when I run it manually: EXEC createSQLServerlessView_gold_SalesOrderDetail; inside develop-> SQL script it works fine. But When I do integration-> pipeline -> stored procedure and choose the same procedure and then run it, it gives…
I have set up an Azure Synapse Serverless SQL Pool, with the goal of reading data from a Parquet data lake stored in an Azure Data Lake Gen2 container. Within Synapse Studio, I can create an SQL script as below,…
We have the azure-management library defined in the requirements.txt: azure-mgmt-eventhub==11.0.0 But when trying to import the library it is not found: This has been intermittent for other developers - but it is persistent for the clusters I have worked with.…
I have a file which is present as a unstructured txt in ADLS. I want to read the entire file content as text and pass it to a Stored Procedure in Azure data factory. Can this be done through Azure…
I have a Azure Synapse pipeline which we migrated from DEV environment to our Pre Prod environment. When I execute it via the debug option, the pipeline executes fine. However if I trigger it via 'Trigger Now' or via setting…
I have an Azure Synapse Workspace for which I have the owner role in the subscription, resource group, and Resource itself, and I am the Microsoft Entra admin for the Synapse Workspace, but a former employee had the Synapse Administrator…