Question 105080 in Azure Question posted in
The official documentation to get you started can be found here.

Azure – Any way to batch-upload a set of files to ADLS gen2 storage?

I want to replace my blob storage with the ADLS Gen 2 functionality in Azure. Point is, I upload a batch of files in a build pipeline with the following command to a certain container in Blob:

az storage blob upload-batch 
          --destination "storage-container-files" 
          --account-name "accountname" 
          --source "./files"

This works perfectly fine. Now I want to do the same thing but then to Gen2 datalake, for which I have to use the az storage blob .... command. However I cannot find something similar to ‘upload-batch’, only single file upload commands. I would like not having to loop in the yaml or in a Python script.

Is there any simple way to batch upload like I could with blob?

Tags:

2

Answers


  1. 0

    We don’t have any cmdlet to upload a batch of file (set of files) for ADLS gen2 Account.

    We have az storage fs file upload cmdlet to upload a single file only.I would suggest you use AZCOPY.exe to upload bulk of files to your adls gen 2 storage account.

    For additional information, you can refer to these below links:

    1. How to upload file to azure storage account using AzCopy.
    2. How to upload bluk of files to adls gen2 account using python.
    Login or Signup to reply.
  2. 0

    Sure, there is:

    trigger:
- none

pool:
  vmImage: ubuntu-latest

steps:
- task: AzureCLI@2
  inputs:
    azureSubscription: 'testbowman_in_AAD'
    scriptType: 'bash'
    scriptLocation: 'inlineScript'
    inlineScript: |
      az storage fs directory upload -f xxx/dir --account-name bowmantest -s "files" --recursive --connection-string "xxx"

    This is my repo structure:

    enter image description here

    And successfully:

    enter image description here

    Refer to this:

    https://learn.microsoft.com/en-us/cli/azure/storage/fs/directory?view=azure-cli-latest#az-storage-fs-directory-upload

    Upload files or subdirectories to a directory in ADLS Gen2 file
    system.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search