Inside of my pipeline, I'm running a lookup to gather tables from one database to transfer to another. select table_schema,table_name from information_schema.tables where table_schema = 'schema1' OR TABLE_SCHEMA = 'schema2' Then I do a copy activity for each of those…
I'm trying to change the tier of a blob in storage account through ADF using Web Activity: I am refering to this link - https://learn.microsoft.com/en-us/rest/api/storageservices/set-blob-tier?tabs=azure-ad The required values used are as below: URL :https://myaccount.blob.core.windows.net/mycontainer/myblob?comp=tier (modified to my account and blob…
I need to update the Azure SQL Server table if another SQL Server tables is updated. For example: I have one Azure SQL table dbo.Azuretable and I also have a SQL Server table dbo.sqlservertable. Both these tables are part of…
How to split the string based on white spaces in DataFlow expression builder in ADF pipeline. A string can have 1 or more white spaces. Eg: I used split(name,’ ‘) : Thiswill split the word based on single space.Some times…
I am trying to copy data from a csv file into a database table. I created a pipeline in Azure Data Factory and added a Copy Activity. I set the source and the destination datasets. During the copy operation, I…
I have hard time to delete the folders in the data lake from adf pipeline. I tred similar method as I did for files by using get metadata activity and delete activity inside ForEach activity, could not work it out.…
I have an Airflow instance in Azure Data Factory. I can test the APIs from swagger. But when I try to test from postman, it's showing authentication error. Tried basic-auth with my azure credentials, but did not work. How do…
I am trying to create a data flow where i can remove the line break from a csv file and because of it I'm getting an error whenever I tried INSERTING those values in a table. The problem is I…
I want to truncate tables where the value of my column isTrun in the config table is True. The scripts need to run in the Sink pre-copy script of the copy activity. The dataflow is the following: The look-up activity…
Hi I am new to Azure ADF and trying to copy additional column values to a table. While doing so sometimes I am getting NULL values from source but while adding that to destination table I get following error. ErrorCode=TypeConversionFailure,Exception…