I'd like to generate a table that consists of list of all available tables and number of rows for each of them. Table Name Number of rows SCHEMA.TABLE_NAME1 30 SCHEMA.TABLE_NAME2 40 I've used the query below: select schema_name(tab.schema_id) + '.'…
I would like to delete the bottom two rows of an excel file in ADF, but I don't know how to do it. The flow I am thinking of is this. enter image description here *I intend to filter ->…
I am having a bit of a trouble with Synapse notebooks. I want to get a list of blob via pyspark script to dynamically decide which files I want to integrate. I cannot make this thing work in Synapse.. in…
I have a Azure Synapse SQL table which contains a list of companies. I have created an access matrix, security predicate and policy to filter data presented based on the user. Create FUNCTION spf.rls_securitypredicate_dimcompany(@Companycode AS nvarchar(50)) RETURNS TABLE WITH SCHEMABINDING…
I have a spark notebook which I am running with the help of pipeline. The notebook is running fine manually but in the pipeline it is giving error for file location. In the code I am loading the file in…
I have a two trigger synapse pipelines one which is scheduled at 03 am cst , What I'm looking now is the Second pipeline should trigger after the completion of the first pipeline i.e after 03 am cst. Is there…
When copying a file from S3 to AzureBlobStorage, I would like to add the date and time string in addition to the source file name. In essence, the S3 folder structure looks like this data/yyyy/mm/dd/files *yyyy=2019-2022, mm=01-12, dd=01-31 And when…
I am new to Azure. I am currently following a tutorial on Azure Synapse Analytics. So far I am learning about how the data is stored. My understanding so far is that the container is contained within a workspace and…
I am getting below error when deploying Synapse CI/CD pipeline using Azure DevOps. The deployment was working fine until I added a few more notebooks and it started failing. Notebooks run fine on lower environment. All paths in the notebooks…
I’m using a copy activity in azure synapse pipeline to copy and filter data from containerA/file1.csv to containerB/file2US.csv Similarly I’m using another copy activity to copy and filter data from containerA/file1.csv to containerB/file2IND.csv The same process for different regions. In…