Azure Question posted in
The official documentation to get you started can be found here.

How can I write a POST request for Azure OpenAI that uses structured output?

I want to write a POST request for Azure OpenAI that uses structured output.

https://azure.microsoft.com/en-us/blog/announcing-a-new-openai-feature-for-developers-on-azure/ says:

Here’s an example API call to illustrate how to use Structured Outputs:

{
  "model": "gpt-4o-2024-08-06",
  "prompt": "Generate a customer support response",
  "structured_output": {
    "schema": {
      "type": "object",
      "properties": {
        "responseText": { "type": "string" },
        "intent": { "type": "string" },
        "confidenceScore": { "type": "number" },
        "timestamp": { "type": "string", "format": "date-time" }
      },
      "required": ["responseText", "intent", "confidenceScore", "timestamp"]
    }
  }
}

Regrettably, they don’t say how to use that API call. How can I write a POST request for Azure OpenAI that uses structured output?

I tried to use the Completion REST API:

import requests
import json
from datetime import datetime

# Set your Azure OpenAI endpoint and key
endpoint = "https://your-azure-openai-endpoint.com"
api_key = "your-azure-openai-key"
deployment = 'engine-name'
endpoint = f'https://{endpoint}/openai/deployments/{deployment}/completions?api-version=2024-06-01'

# Define the request payload
payload = {
    "model": "gpt-4omini-2024-07-18name",
    "prompt": "Generate a customer support response",
    "structured_output": {
        "schema": {
            "type": "object",
            "properties": {
                "responseText": { "type": "string" },
                "intent": { "type": "string" },
                "confidenceScore": { "type": "number" },
                "timestamp": { "type": "string", "format": "date-time" }
            },
            "required": ["responseText", "intent", "confidenceScore", "timestamp"]
        }
    }
}

# Send the request
headers = {
    "Content-Type": "application/json",
    "api-key": f"{api_key}"
}

response = requests.post(endpoint, headers=headers, data=json.dumps(payload))

# Handle the response
if response.status_code == 200:
    response_data = response.json()
    print(json.dumps(response_data, indent=4))
else:
    print(f"Error: {response.status_code}")
    print(response.text)

However, I get the error completion operation does not work with the specified model:

{"error":{"code":"OperationNotSupported","message":"The completion operation does not work with the specified model, gpt-4o-mini. Please choose different model and try again. You can learn more about which models can be used with each operation here: https://go.microsoft.com/fwlink/?linkid=2197993."}}

Replacing the endpoint to the chat completions REST API endpoint:

endpoint = f'https://{endpoint}/openai/deployments/{deployment}/chat/completions?api-version=2024-06-01'

yields the error

Error: 400
Unsupported data type

GPT-4o-2024-08-06 is not in API yet. Only available in playground according to https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models#early-access-playground-preview:

Azure customers can test out GPT-4o 2024-08-06 today in the new AI Studio early access playground (preview).

So I guess I should use another REST API with gpt-4o-mini-20240718, but which one?

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    Using gpt-4o-2024-08-06, which finally got deployed today on Azure, made it work. Code:

    import requests
import json
from datetime import datetime

# Set your Azure OpenAI endpoint and key
deployment = '[deployment name for gpt-4o 2024-08-06]'
endpoint = "your-azure-openai-endpoint.com"
api_key = "your-azure-openai-key"
endpoint = f'https://{endpoint}/openai/deployments/{deployment}/chat/completions?api-version=2024-06-01'

# Define the request payload
payload = {
    "prompt": "Generate a customer support response",
    "structured_output": {
        "schema": {
            "type": "object",
            "properties": {
                "responseText": { "type": "string" },
                "intent": { "type": "string" },
                "confidenceScore": { "type": "number" },
                "timestamp": { "type": "string", "format": "date-time" }
            },
            "required": ["responseText", "intent", "confidenceScore", "timestamp"]
        }
    }
}

# Send the request
headers = {
    "Content-Type": "application/json",
    "api-key": f"{api_key}"
}

response = requests.post(endpoint, headers=headers, data=json.dumps(payload))

# Handle the response
if response.status_code == 200:
    response_data = response.json()
    print(json.dumps(response_data, indent=4))
else:
    print(f"Error: {response.status_code}")
    print(response.text)

  2. 0

    Both the models GPT-4o-2024-08-06 and GPT-4o-mini-2024-07-18 are not in API yet for getting customized structured output.

    GPT-4o-mini-2024-07-18 supports response_format but it is not customized by giving json_schema, only allowed value is json_object and also you can give the system message that gives Json output.

    Below, is the sample for two cases.

    Case 1:

    Endpoint:

    https://xxxxx.openai.azure.com/openai/deployments/GPT-4o-mini-2024-07-18/chat/completions?api-version=2024-02-15-preview

    Sample Body:

    {
  "model": "GPT-4o-mini-2024-07-18",
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant designed to output JSON."
    },
    {
      "role": "user",
      "content": "Generate a sample customer support response with following details: [responseText,intent,confidenceScore,timestamp]"
    }
  ],
  "response_format": {
      "type": "json_object"
  }
}

    Output:

     "message": {
   "content": "{
     n  "responseText": "Thank you for reaching out! We're sorry to hear that you're experiencing issues with your order. Can you please provide your order number so we can assist you further?",
     n  "intent": "order_issue",
     n "confidenceScore": 0.92,
     n  "timestamp": "2023-10-10T14:35:00Z"n}",
   "role": "assistant"
            }

    Case 2:

    Sample body:

    {
  "model": "GPT-4o-mini-2024-07-18",
  "messages": [
    {
      "role": "system",
      "content": "Assistant is an AI chatbot that helps users turn a natural language list into JSON object. After users input a list they want in JSON object."
    },
    {
      "role": "user",
      "content": "Generate a sample customer support response with following details: [responseText,intent,confidenceScore,timestamp]"
    }
  ]
}

    Output:

    "message": {
      "content": "Here's a sample JSON object based on your details:nn```jsonn{n  "responseText": "Thank you for reaching out to our customer support. We are here to assist you with any questions or concerns you may have.",n  "intent": "customer_support_response",n  "confidenceScore": 0.95,n  "timestamp": "2023-10-05T14:30:00Z"n}n``` nnFeel free to modify any of the values as needed!",
      "role": "assistant"
            }

    I would recommend using first case, since you can load the Json object from the results.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search