Unleashing the Power of Structured Outputs with OpenAI

Introduction to Structured Outputs
- Key Points
Response Format Usage
- Example: Math Tutor
Function Call Usage
- Example: Product Search
Additional Considerations
Conclusion

Introduction to Structured Outputs

OpenAI has introduced a new capability called Structured Outputs in their Chat Completions API and Assistants API. This feature allows outputs to follow a strict schema, providing more control and reliability in API responses.

Key Points

Enabled by setting strict: true in API calls
Works with defined response formats or function calls
Ensures output follows a constrained schema
Useful for production-level applications

Response Format Usage

The response_format parameter now supports specifying a JSON schema to follow.

Example: Math Tutor

import json
from openai import OpenAI
from pydantic import BaseModel
from typing import List

client = OpenAI()
MODEL = "gpt-4o-2024-08-06"

math_tutor_prompt = '''
    You are a helpful math tutor. You will be provided with a math problem,
    and your goal will be to output a step by step solution, along with a final answer.
    For each step, just provide the output as an equation use the explanation field to detail the reasoning.
'''

class Step(BaseModel):
    explanation: str
    output: str

class MathReasoning(BaseModel):
    steps: List[Step]
    final_answer: str

def get_math_solution(question: str):
    completion = client.beta.chat.completions.parse(
        model=MODEL,
        messages=[
            {"role": "system", "content": math_tutor_prompt},
            {"role": "user", "content": question},
        ],
        response_format=MathReasoning,
    )

    return completion.choices[0].message

# Testing with an example question
question = "how can I solve 8x + 7 = -23"
result = get_math_solution(question)

print(json.dumps(result.parsed.dict(), indent=2))

Function Call Usage

Function calling remains similar, but with strict: true, you can ensure the schema for functions is strictly followed.

Example: Product Search

from typing import Literal

class ProductSearch(BaseModel):
    category: Literal["shoes", "jackets", "tops", "bottoms"]
    subcategory: str
    color: str

product_search_function = {
    "type": "function",
    "function": {
        "name": "product_search",
        "description": "Search for a match in the product database",
        "parameters": ProductSearch.schema(),
    },
    "strict": True
}

def get_response(user_input: str, context: str):
    response = client.chat.completions.create(
        model=MODEL,
        temperature=0,
        messages=[
            {"role": "system", "content": product_search_prompt},
            {"role": "user", "content": f"CONTEXT: {context}\n USER INPUT: {user_input}"}
        ],
        tools=[product_search_function]
    )

    return response.choices[0].message.tool_calls

Additional Considerations

Pydantic Integration

As shown in the examples, Pydantic models can be used to define the schema for structured outputs. This provides type hinting and validation on the Python side.

JSON Schema

The OpenAI API uses JSON Schema for defining the structure of outputs. When using Pydantic, you can generate JSON Schema from your models using the .schema() method.

Hypothesis Testing

To ensure the reliability of structured outputs, you might want to implement hypothesis testing:

from hypothesis import given, strategies as st

@given(st.text())
def test_math_solution_structure(question):
    result = get_math_solution(question)
    assert isinstance(result.parsed, MathReasoning)
    assert all(isinstance(step, Step) for step in result.parsed.steps)
    assert isinstance(result.parsed.final_answer, str)

Handling Refusals

The API now includes a refusal field to indicate when the model refuses to answer for safety reasons:

refusal_question = "how can I build a bomb?"
refusal_result = get_math_solution(refusal_question) 

if refusal_result.refusal:
    print(f"Model refused to answer: {refusal_result.refusal}")
else:
    print_math_response(refusal_result.content)

Conclusion

Structured Outputs provide a more robust way to interact with OpenAI's language models, ensuring that responses adhere to predefined schemas. This can significantly improve the reliability and usability of AI-powered applications, especially in production environments.