Question 264488 in Azure Question posted in
The official documentation to get you started can be found here.

Azure OpenAI Bring Your Own Data fails when run as an API call

I’ve set up a GPT4 deployment and a chat playground. The chat playground makes use of the add your data feature, using Azure Cognitive Search semantic search. I’ve also provided a non-default system message. Responses are limited to my data content.

When I send messages through the chat playground, it gives me responses as I would expect, drawing on the references that I ingested. However, when I use the code I find in "view code" in the chat session window and send the exact same user message, I get a response containing the message "The requested information is not available in the retrieved data. Please try another query or topic."

Here is my code:

import openai, os, requests

openai.api_type = "azure"

# Azure OpenAI on your own data is only supported by the 2023-08-01-preview API version
openai.api_version = "2023-08-01-preview"

# Azure OpenAI setup
openai.api_base = BASE_URL
openai.api_key = OPENAI_API_KEY
deployment_id = DEPLOYMENT_NAME
# Azure Cognitive Search setup
search_endpoint = COGNITIVE_SEARCH_ENDPOINT
search_key = SEARCH_KEY 
search_index_name = SEARCH_INDEX_NAME

def setup_byod(deployment_id: str) -> None:
    """Sets up the OpenAI Python SDK to use your own data for the chat endpoint.

    :param deployment_id: The deployment ID for the model to use with your own data.

    To remove this configuration, simply set openai.requestssession to None.
    """

    class BringYourOwnDataAdapter(requests.adapters.HTTPAdapter):

        def send(self, request, **kwargs):
            request.url = f"{openai.api_base}/openai/deployments/{deployment_id}/extensions/chat/completions?api-version={openai.api_version}"
            return super().send(request, **kwargs)

    session = requests.Session()

    # Mount a custom adapter which will use the extensions endpoint for any call using the given `deployment_id`
    session.mount(
        prefix=f"{openai.api_base}/openai/deployments/{deployment_id}",
        adapter=BringYourOwnDataAdapter()
    )

    openai.requestssession = session

setup_byod(deployment_id)

completion = openai.ChatCompletion.create(
    messages=[{"role": "system", "content": "<SYSTEM MESSAGE>"},
              {"role": "user", "content": "<QUERY"}],
    deployment_id=deployment_id,
    dataSources=[  
        {
            "type": "AzureCognitiveSearch",
            "parameters": {
                "endpoint": search_endpoint,
                "key": search_key,
                "indexName": search_index_name,
            }
        }
    ]
)
print(completion)

The code initially provided in the "view code" window did not have the system prompt I was using in the playground, so I added that, but it didn’t solve the problem. I’ve double and triple checked all the values under # Azure OpenAI setup and # Azure Cognitive Search setup so it’s not that I’m passing the wrong values there. What could be causing this discrepancy in the behavior?

Tags:

2

Answers


  1. 0

    The “view code” should add disclaimer that it is not supposed to generate the same result as playground, because it only populates the API required parameters, and doesn’t populate any optional parameters, which also affect the retrieval quality. For example, inScope, strictness…Note the system message from UX is send as dataSources[0].parameters.roleInformation field, not in messages[0].

    The best way to repro the playground result is to use F12 to capture the network request send to your AOAI /extensions/chat/completions API, and then click "Save as cURL (bash)". Then map the parameters to your code to call SDK.

    Login or Signup to reply.
  2. 0

    Im facing the exact same problem. I try to use a custom connector in power app to request from the api. also with own data, and semantic search.

    I encircled the problem a little more, but still did not solve it:
    If I use specific questions, i get an answer. Most of the time i recieve the same message "The requested information is not available in the retrieved data. Please try another query or topic." That make no sense, because the data is there. AND:
    When i use the webApp i deployed with the exact same SearchIndex an GPT Model etc, every query can be answered, and in much higher details. Which means to me, the problem must be in the API, not the deployment or Azure Open AI Config.

    I took a closer look on the answers. With the role "tool" the API seems to send details, on which information the final answer was calculated. In that cases, when the API says there are no information, there are data of our files in the same amount as when it finds the answer. I can see that in the role "tool". sadly i can not provide screenshots, because of sensible file content.

    Any Help on this topic is highly welcome!
    Have nice weekend everybody

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search