Azure Question posted in
The official documentation to get you started can be found here.

Azure functions read, process and save file on arrival in blob

I have a use case where I have to process files when they arrive in blob container.

The files are in a wiresharkproprietary format and need to be processed with specific program (tshark).

My goal is to execute the shell command that converts files to csv:

tshark -r <input_filename> <some other params> > <output_filename>.csv

and then save output csv files to another blob container.

I know that I can set a blob trigger in azure functions to trigger the process, but I’m not sure how to save the output file to blob and make sure that tshark is available in the runtime.

Another question is: maybe there is a better way to do this rather than azure functions?

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    I managed to achieve my goal by doing the following:

    1. Push docker image with all dependencies installed to Azure Container Registry
    2. Create Event Grid Subscription to system topic (blob created in storage account with additional filters like container and file extension). This populates the storage queue with messages.
    3. Create Container App Job with storage queue trigger that uses the docker image and consumes messages in the queue. For storage account access, I put the Container App Environment in my VNET.

  2. 0

    there is a better way to do this rather than azure functions?

    Yes, you can use Azure Logic apps and Azure Automation Accounts .

    Below is the way how I got expected results:

    enter image description here

    This calls the automation account.

    In Automation account you can add tshark module and use them.

    enter image description here

    Then you can edit your runbook and include parameters in it get the name of file.
    In run book get blob using blob name got from from Logic apps.
    enter image description here

    enter image description here

    Here just printing name, you can write commands to get blob from blobstorage and use tsharks with it. You can also create temparory(SO-Thread where I used Temperoroy file) file in Runbook.

    Now use the blobname get the blob and then use tshark and then upload to another storage container.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search