Azure Question posted in
The official documentation to get you started can be found here.

Possibility to load documents with Azure Document Intelligence from Azure Container Storage

I have a storage account with Azure Container Storage configured consisting of multiple pdf/word/excel files. I would like to use Azure Document Intelligence to semantically chunk these files.

Is there a possibility to load the files directly from Container Storage to Azure Document Intelligence using langchain? According to the langchain docs it seems like either file has to be locally available or public url has to be handed over.

Attempt:

# Prerequisite: An Azure AI Document Intelligence resource in one of the 3 preview regions: East US, West US2, West Europe

import os
from langchain_community.document_loaders import AzureAIDocumentIntelligenceLoader

file_path = "storage-path-to-file"
endpoint = os.getenv("DOCUMENTINTELLIGENCE_ENDPOINT")
key = os.getenv("DOCUMENTINTELLIGENCE_API_KEY")

loader = AzureAIDocumentIntelligenceLoader(
    api_endpoint=endpoint, api_key=key, file_path=file_path, api_model="prebuilt-layout"
)

documents = loader.load()

# Returns:
# Message: Invalid request.
# Inner error: {
#    "code": "InvalidManagedIdentity",
#    "message": "The managed identity configuration is invalid: Managed identity is not enabled  # for the current resource."
# }
Tags:

2

Answers


  1. 0

    Is there a possibility to load the files directly from Container Storage to Azure Document Intelligence using langchain? According to the langchain docs it seems like either file has to be locally available or public url has to be handed over.

    You can use the below code which loads the files directly from Azure Blob storage using Azure Blob URL + SAS token.

    Code:

    import os
from langchain_community.document_loaders import AzureAIDocumentIntelligenceLoader

#bloburl + ?Sastoken
URL = "https://venkat78932.blob.core.windows.net/test/24005356.pdf?sp=r&st=2024-07-09T12:41:41Z&se=2024-07-09T20:41:41Z&spr=https&sv=2022-11-02&sr=b&sig=eupT8WGH5ojQpXYd%2xxxxxD"
endpoint = os.getenv("DOCUMENTINTELLIGENCE_ENDPOINT")
key = os.getenv("DOCUMENTINTELLIGENCE_API_KEY")


loader = AzureAIDocumentIntelligenceLoader(
    api_endpoint=endpoint, api_key=key, url_path=URL, api_model="prebuilt-layout"
)

documents = loader.load()

print(len(documents))
print(documents[0])

    Output:

    1
page_content='Word Documents Templaten===nnn## Main heading:nnUse the Heading 1 style for primary headings so that screen readers can identify them as such.nnIf not already, manually change your heading 1 style to be:nn\- sans serif (e.g. Arial, Verdana, Trebuchet or Calibri),nn\- 16 pt, andnn\- BoldnnThen set this formatting as your default for this style.nnn## Sub Headings:nnUse Heading 2 style for sub headings.nnIf not already, manually change your heading 2 style to be:nn\- sans serif (e.g. Arial, Verdana, Trebuchet or Calibri...................tands out, and does not distort the shape of text as italics and underlining do. Finally, block capitals can be difficult to follow as block capitals remove the natural shape of words, turning them into blocks. Clear layout allows one to focus on the content of visual materials rather than the format.nnn## FurthermorennIf you use headings it makes the creation and upkeep of tables of contents easier (For automatic creation and updating go to: Insert - Reference - Index and Tables - Table of contents).n'

    enter image description here

    You can get the Azure Blob URL + SAS token from portal.

    Portal -> Storage account -> Container -> your file -> Generate sas token -> click Generate sas token and url

    Portal:

    enter image description here

    Reference:
    Azure AI Document Intelligence | 🦜️🔗 LangChain

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search