Question 171307 in Azure Question posted in
The official documentation to get you started can be found here.

What value should I pass in Copy activity’s source param?

Whenever I am trying to preview the data it’s asking for value.

I’m new to ADF.

I have provided all the parameters correctly everywhere don’t know what I’m missing.
It’s going till the folder path correctly but not able to read files inside that folder. If I give one file name at parameter value, it’s previewing the data and if I giving anything else preview data getting failed.

Attaching the steps of the pipeline and parameter declarations.
enter image description here
enter image description here
enter image description here

Tags:

2

Answers


  1. 0

    @contains is a boolean function which will return true if the second string passed as parameter is a part of first string. In the fileName , you need to remove @contains .

    enter image description here

    If you want to use only certain file names , for example , only the file names which has ‘_data’ as suffix (say SalesFileMay2023_data) then either use ‘Filter activity’ after getMetadata activity to fetch only specific filenames and pass in foreach

    Or else use ‘Wildcard file path’ in copy activity and provide the correct pattern.

    Login or Signup to reply.
  2. 0

    While previewing the dataset, it will ask to provide the value for the parameters which involves dynamic content expressions like yours. Preview is only for checking the data.

    So, while previewing, we need to expect the value as per the dynamic expression of the parameter and provide a sample value manually. It will consider these values only while previewing and not when you debug. This applies to both dataset preview and copy activity(source or sink) preview.

    If your dynamic expression is valid, it will give expected results when you debug.

    Coming to your parameter expression, I agree with @AnnuKumari that your parameter expects a string(file name) and contains gives a boolean value.

    So, as per your expression, you need to filter certain file names and go through the below sample scerario to achieve your requirement.

    Here I will copy the files which starts with Nilesh from the below files.

    enter image description here

    Create a dataset and give the file path till the folder rakeshdata and give this dataset to Get Meta data activity and use Child items to get the files list.

    Give the Get Meta data putput child items array to the filter activity. Give the condition for it.

    Items: @activity('Get Metadata1').output.childItems
    Condition : @startswith(item().name, 'Nilesh')

    enter image description here

    Filter output array:

    enter image description here

    Now, Give this array to ForEach activity.

    enter image description here

    Inside ForEach, in the copy activity source use dataset with parameter as file name.

    enter image description here

    Create another dataset for the copy activity sink and do the same(provide sink folder and dataset parameter for sink file name) for it as above.

    In the source dataset parameter give the @item().name like below.

    enter image description here

    Do the same for sink of copy activity.

    Result after debug of the pipeline:

    enter image description here

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search