I am trying to delete folders that are older than 7 days from ADLS. The folders I want to delete have date as name and are under subfolders of the main folder. Please see below example for folder structure (raw…
Using the external call action to geocode address data using the geocodio api The GET request is returning a 200 Okay response and I can see in the geocodio portal that at least one result is being returned for each…
I have 2 cloud environments AWS and Azure. In AWS, I have a SQL Server instance inside a VPC (192.168.0.0/24). In Azure, I have a VNet (10.0.0.0/24). Both these 2 Virtual Network are connected through Site-2-Site VPN. I have tested…
I am trying to perform copy activity between ADLS dataset and ODBC sync dataset (POSTGRESQL) But, table is not creating automatically in target dataset and error coming like Failure happened on 'Sink' side. ErrorCode=UserErrorOdbcOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=ERROR [42P01] ERROR: relation "semantic_dev.dim_storage_location" does not…
Inside of my pipeline, I'm running a lookup to gather tables from one database to transfer to another. select table_schema,table_name from information_schema.tables where table_schema = 'schema1' OR TABLE_SCHEMA = 'schema2' Then I do a copy activity for each of those…
I'm trying to change the tier of a blob in storage account through ADF using Web Activity: I am refering to this link - https://learn.microsoft.com/en-us/rest/api/storageservices/set-blob-tier?tabs=azure-ad The required values used are as below: URL :https://myaccount.blob.core.windows.net/mycontainer/myblob?comp=tier (modified to my account and blob…
I need to update the Azure SQL Server table if another SQL Server tables is updated. For example: I have one Azure SQL table dbo.Azuretable and I also have a SQL Server table dbo.sqlservertable. Both these tables are part of…
How to split the string based on white spaces in DataFlow expression builder in ADF pipeline. A string can have 1 or more white spaces. Eg: I used split(name,’ ‘) : Thiswill split the word based on single space.Some times…
I am trying to copy data from a csv file into a database table. I created a pipeline in Azure Data Factory and added a Copy Activity. I set the source and the destination datasets. During the copy operation, I…
I have hard time to delete the folders in the data lake from adf pipeline. I tred similar method as I did for files by using get metadata activity and delete activity inside ForEach activity, could not work it out.…