I have a source table/excel/csv data and done ETL to target SQL table. Now if there is a new column introduced in the source data, how to replicate the data in target. Assume that there was already one load of…
How can I copy files from an input container to an output container in Azure blob storage, and sort them into three different folders based on their file size, with files less than 1KB going to the 1KB folder, files…
I need to create a File System trigger that whenever a new file is created in an on premise machine, it executes a pipeline in Data Factory. Such trigger requires a password that is currently stored in a Key Vault.…
I have a program the generates an .exe file. Unfortunately, I do not have any idea about how this program is implemented, and our team consumes it as a black box. I am trying to run this .exe file (which…
Would like to know a KQL query to detect changes in ADF triggers. Looking at Log Analytics since we are sending all logs to LA, I see several tables are created and populated, some are not. AzureActivity looked promising but…
I am trying to delete folders that are older than 7 days from ADLS. The folders I want to delete have date as name and are under subfolders of the main folder. Please see below example for folder structure (raw…
Using the external call action to geocode address data using the geocodio api The GET request is returning a 200 Okay response and I can see in the geocodio portal that at least one result is being returned for each…
I have 2 cloud environments AWS and Azure. In AWS, I have a SQL Server instance inside a VPC (192.168.0.0/24). In Azure, I have a VNet (10.0.0.0/24). Both these 2 Virtual Network are connected through Site-2-Site VPN. I have tested…
I am trying to perform copy activity between ADLS dataset and ODBC sync dataset (POSTGRESQL) But, table is not creating automatically in target dataset and error coming like Failure happened on 'Sink' side. ErrorCode=UserErrorOdbcOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=ERROR [42P01] ERROR: relation "semantic_dev.dim_storage_location" does not…
Inside of my pipeline, I'm running a lookup to gather tables from one database to transfer to another. select table_schema,table_name from information_schema.tables where table_schema = 'schema1' OR TABLE_SCHEMA = 'schema2' Then I do a copy activity for each of those…