In Azure Data Factory I created a pipeline with several copy data activities. As soon as one of them crashes I want that it continues to the next step and that it sends an e-mail with the error. The mail…
I am deploying to an azure synapse environment queries using sqlcmd to serverless pool. The environment contains an SQL database that my deploying account has access to. I am creating first the credentials to access a cosmosDB with : CREATE…
I have a Main pipeline in Synapse workspace which has 2 activities: 1st - Notebook activity 2nd - If Condition activity For the 1st one (Synapse notebook, spark pool, pyspark), I have a SQL cell like the following: It has…
I am not able to undertsand why we are writing with (doc nvarchar(max)) as rows JSON FILE {"date_rep":"2020-07-24","day":24,"month":7,"year":2020,"cases":3,"deaths":0,"geo_id":"AF"} {"date_rep":"2020-07-25","day":25,"month":7,"year":2020,"cases":7,"deaths":0,"geo_id":"AF"} {"date_rep":"2020-07-26","day":26,"month":7,"year":2020,"cases":4,"deaths":0,"geo_id":"AF"} {"date_rep":"2020-07-27","day":27,"month":7,"year":2020,"cases":8,"deaths":0,"geo_id":"AF"} My Code to open this JSON file start from here. select JSON_VALUE(doc, '$.date_rep') AS date_reported, JSON_VALUE(doc, '$.countries_and_territories') AS country,…
I've created a KQL query for a LogAnalytics Workspace and one for the Azure Resource Graph and now try to get the results scheduled to Synapse Analytics or Azure DataLake Storage Gen2. But from Synapse side I can only create…
I would like to use the COPY INTO statement to copy a table with it's column names from an external storage. Would like to achieve it using a generic procedure which can use for different tables. Here below you find…
I am trying to give a service principal SELECT access on my Azure Synapse SQL data. CREATE USER [MY_SERVICE_PRINCIPAL] FROM EXTERNAL PROVIDER WITH DEFAFULT_SCHEMA=[dbo] GO GRANT SELECT ON DATABASE :: MyDB TO [MY_SERVICE_PRINCIPAL]; This works fine, but it requires me…
I am making a rest call using a copy activity to write data to a datalake storage. Source is a rest dataset and sink is a json. If I use a http binary dataset it works fine, but then I…
I need to calculate the cost of the Azure Synapse Analytics. I have used the Azure Pricing Calculator but I could not figure it out. It shows close to USD 2,100. I have the following components as a part of…
I´ve created an Azure Synapse Analytics Pipeline that must be triggered by the creation of a file within a Azure Gen2 storage account. Somehow the blob creation event (i.e. when I upload the file in the corresponding container and folder)…