skip to Main Content

I am looking to use python code deployed in Azure Cloud (e.g. on an Azure Function), to read files location from a shared/local network drive (on-premise). The network drive requires user based authentication (i.e. different users have access/permissions to different areas).

There is an express route connection established to the local network, and I also have confirmed looking at the firewall there is nothing blocking cloud access to the shared drives.

My question is – what is the best way to read these files from local network (on-premise) to within the Azure function using python? E.g. what packages, should be used and how can user credentials be entered also.

I am unable to access with smbprotocol and can’t see the files (e.g. using the os package), and believe its due to authentication.

2

Answers


  1. Seems to be an XY problem. Why do you want to list onprem files from an Azure function ?

    I’m not sure this is even possible. But the real question is, do you really want to expose the whole locale corporate file share ? Even using the express route ?

    Azure functions are a serverless Cloud natif deployment, that can connect safely and natively to Azure Blob storage.

    My suggestions :

    1. Use an Azure File share, that will be mounted to your Azure Function AND to your onprem services
    2. Or deploy an API onprem with samba access that exposes a route that get file location. Your Azure function can then call the API to access this information.

    Some related answer :

    Login or Signup to reply.
  2. You can achieve the task with python, and now that we know the connection requires authentication maybe we need to know more about the express route connection, the Azure Function (python) could require server-side and client-side connectors, however if we need to keep the current security, it is always a good idea to keep the same cloud infrastructure, we can implement a type of customer premises equipment (CPE) network connection (2 months ago this feature was still in preview):

    Client or local python:

    pip install smbprotocol #windows or #python3 -m pip install smbprotocol
    

    Linux will require Kerberos support: pip install smbprotocol[kerberos]

    Once Client or local python has already authenticated the access to the Network Drive, you can manage it as a normal local file to read, and then we should move to the server side (Azure Function), you need to create an Azure Function to be allowed to post a file:

    It will be a combination of: Creating your Azure Function to receive a FILE with a POST function and store it on Azure Blob or similar, or process it in the way desired (https://learn.microsoft.com/en-us/azure/azure-functions/functions-reference-python?tabs=asgi%2Capplication-level&pivots=python-mode-decorators#sdk-type-bindings-examples)

    The function I would use for blob, client side:

    def s_on_db(self,container_db,folder_db,file_db_local_complete_path,file_to_file_to_db):
        try:
            connection_string="BlobEndpoint=https://mystorageaccount2024.blob.core.windows.net/;QueueEndpoint=h..."
            blob_service_client = BlobServiceClient.from_connection_string(connection_string)
            container_client = blob_service_client.get_container_client(container_db)
            blob_client = container_client.get_blob_client(folder_db+"/"+file_to_file_to_db)
            start=time.time()
            #upload data
            block_list=[]
            chunk_size=4*1024*1024
            with  open(file_db_local_complete_path,'rb') as  f:
              while  True:
                    read_data = f.read(chunk_size)
                    if  not  read_data:
                        break  # done
                    blk_id = str(uuid.uuid4())
                    blob_client.stage_block(block_id=blk_id,data=read_data)
                    block_list.append(BlobBlock(block_id=blk_id))
    
            blob_client.commit_block_list(block_list)
            end=time.time()
        except Exception as e:
            print(e)
            pass
    

    The function uploads the file part by part, depending on the bandwidth and internet upload speed you can change it:

    Full code (I will test it later because I have to set all the environment in Azure and local):

    from smbprotocol.open import Open, CreateDisposition, FilePipePrinterAccessMask, FileAttributes, ShareAccess, ImpersonationLevel
    from smbprotocol.session import Session
    from smbprotocol.connection import Connection
    import psutil
    the_current_user=psutil.Process().username()
    
    driveinnetwork = 'NetCOmpanyDrive'
    username = the_current_user #this can be the_current_user, this may be a vulnerability, recommended to be used for IT Admin only
    password = 'password' #if each user is managed with EntraID, we will need MSAL instead of the actual password, so we do not expose the real one, or the IT Admin can enable the script as a hidden service
    shared_folder = 'shared_folder_drive'
    file_path = '/path/to/localfile.pdf'
    
    #or os.system(f"net use {driveinnetwork} /user:{username} {password}") #but this may be a security issue
    
    #client side connector
    connection = Connection(uuid.uuid4(), driveinnetwork, 445)
    connection.connect()
    session = Session(connection, username, password)
    session.connect()
    
    allids = session.tree_connect(shared_folder)
    
    # Access a file
    with Open(allids, file_path, desired_access=FilePipePrinterAccessMask.GENERIC_READ, create_disposition=CreateDisposition.FILE_OPEN, file_attributes=FileAttributes.FILE_ATTRIBUTE_NORMAL, share_access=ShareAccess.FILE_SHARE_READ, impersonation_level=ImpersonationLevel.Impersonation, oplock_level=0) as file:
        #read
        data=file.read() #can read all or by chunks for memory usage
    
    #Add Azure POST file to Azure Function, more mechanisms may be needed here to know when remote tries to read local and vice-versa
        
    def s_on_db(container_db,folder_db,file_db_local_complete_path,file_to_file_to_db):
        try:
            connection_string="BlobEndpoint=https://mystorageaccount2024.blob.core.windows.net/;QueueEndpoint=h..."
            blob_service_client = BlobServiceClient.from_connection_string(connection_string)
            container_client = blob_service_client.get_container_client(container_db)
            blob_client = container_client.get_blob_client(folder_db+"/"+file_to_file_to_db)
            start=time.time()
            #upload data
            block_list=[]
            chunk_size=4*1024*1024
            with  open(file_db_local_complete_path,'rb') as  f:
              while  True:
                    read_data = f.read(chunk_size)
                    if  not  read_data:
                        break  # done
                    blk_id = str(uuid.uuid4())
                    blob_client.stage_block(block_id=blk_id,data=read_data)
                    block_list.append(BlobBlock(block_id=blk_id))
    
            blob_client.commit_block_list(block_list)
            end=time.time()
        except Exception as e:
            print(e)
            pass
    
    
    #data will be sent to the cloud using the function s_on_db(container_db,folder_db,file_db_local_complete_path,file_to_file_to_db)
    
    #close connection
    session.tree_disconnect(tree_id)
    session.logoff()
    connection.disconnect()
    

    The CPE option could be easier and keeps the Microsoft EntraID Authentication schema, requires a work or school account and some O365 Licenses (EntraID P1 or P2)

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search