Unlocking the Power of Structured Outputs in AI Applications with Amazon Nova FMs

Overcoming Challenges in Data Structuring

Enhancing Reliability with Constrained Decoding

Effective Techniques for Implementing Structured Outputs

Leveraging Amazon Nova Models for Reliable Schema Handling

The Schema Definition Process: Setting the Stage for Success

Conclusion: Building Reliable AI Applications with Amazon Nova

About the Authors

Structuring AI Outputs: Navigating the Challenges with Amazon Nova

In the fast-evolving realm of AI, developers frequently encounter a significant hurdle: transforming unstructured data into structured formats. This conversion is vital for applications that rely on machine-to-machine communications, enabling downstream processes to effectively consume and utilize generated outputs. From extracting information from documents to creating intelligent assistants and actionable agents, developers need robust solutions for producing consistent structured outputs.

The Power of Constrained Decoding

To address these challenges, we recently launched constrained decoding, enhancing reliability in structured output generation. Now, developers can confidently leverage Amazon Nova foundation models (FMs) to extract data following complex schemas, drastically reducing tool use errors by over 95%. In this blog post, we’ll delve into the ways you can harness Amazon Nova FMs for structured output use cases effectively.

Techniques for Implementing Structured Outputs

When it comes to generating structured outputs, there are two primary strategies: modifying the system prompt or utilizing tool calling.

Modifying the System Prompt

For instance, in a customer support scenario, you may want the model to output a JSON object containing both the response to the customer and the current sentiment. Here’s how you can articulate your requirements via the system prompt:

{
   "response": "the response to the customer",
   "sentiment": "the current customer sentiment"
}

By clearly defining the expected structure, the model can tailor its responses accordingly.

Tool Calling

Alternatively, you can implement tool calling, which provides a defined API or schema to the model through the request schema associated with the Converse API. This technique is particularly useful when constructing agentic applications and effectively dictates the structured format the model should adhere to.

Here’s an example configuration for a tool that responds to customer interactions:

tool_config = {
    "tools": [ 
        {
            "toolSpec": {
                "name": "respondToUser",
                "description": "the formatted response to the customer",
                "inputSchema": {
                    "type": "object",
                    "properties": {
                        "response": {
                            "description": "the response to the customer",
                            "type": "string"
                        },
                        "sentiment": {
                            "description": "the current customer sentiment",
                            "type": "string"
                        }
                    },
                    "required": ["response", "sentiment"]
                }
            }
        }
    ]
}

While both prompt techniques can be effective, it’s essential to note that non-deterministic outputs can still emerge—especially as schema complexity increases. In our collaborations with clients focusing on agentic workflows and structured extraction, we’ve observed a decrease in model accuracy as the schemas become more intricate.

Structured Output with Amazon Nova Models

To combat these challenges, we’ve implemented constrained decoding in our system, enhancing model reliability for structured output generation. This approach constrains the possible tokens available to the model at each generation step, significantly improving output validity.

Whether closing a JSON object or ensuring correct data types, constrained decoding ensures that models like Amazon Nova produce valid outputs every time a tool configuration is employed. By dynamically generating a grammar based on the desired schema, we can effectively enforce adherence to structured requirements.

The Schema Definition Process

A crucial part of using structured outputs with Amazon Nova involves crafting a tool configuration that outlines the expected output schema. This JSON interface serves as a guide for the AI model, defining how fields should be populated.

For example, consider a use case requiring recipe extraction from online content. The initial step is creating a tool configuration like this:

tool_config = {
   "tools": [
        {
            "toolSpec": {
                "name": "extract_recipe",
                "description": "Extract recipe for cooking instructions",
                "inputSchema": {
                    "json": {
                        "type": "object",
                        "properties": {
                            "recipe": {
                                "type": "object",
                                "properties": {
                                    "name": {
                                        "type": "string",
                                        "description": "Name of the recipe"
                                    },
                                    // ... other fields as described in the original content
                                },
                                "required": []
                            }
                        },
                        "required": []
                    }
                }
            }
        }
    ]
}

By clearly defining the properties and their descriptions, you help guide the model in accurately populating the fields.

Putting It All Together

Once you’ve established your tool configuration, you can seamlessly integrate it with the Converse API, coupled with user prompts. The system prompt plays a vital role by outlining the model’s role and expectations during output generation.

import boto3

model_response = client.converse(
    modelId="us.amazon.nova-lite-v1:0",
    system=[{"text": "You are an expert recipe extractor that compiles recipe details from blog posts"}],
    messages=[{"role": "user", "content": content}],
    inferenceConfig={"temperature": 0},
    toolConfig=tool_config
)

By utilizing the native tool support and constrained decoding, you can ensure that your structures conform to the designed syntax and schema.

Conclusion

The advent of structured output capabilities in Amazon Nova through tool calling and constrained decoding empowers developers to build reliable AI systems with well-formed JSON outputs. We encourage you to begin leveraging these technologies in your applications. Learn more in the Amazon Nova User Guide and start building your AI applications via the Amazon Bedrock console today.

About the Authors

Jean Farmer is a Generative AI Solutions Architect at Amazon AGI, focusing on autonomous AI systems and practical applications. Mukund Birje is a Sr. Product Marketing Manager at AWS, driving the adoption of Amazon Nova FMs with over a decade of experience in marketing.

By exploring these structured output techniques, you can unlock new potentials in your AI applications. Embrace the power of Amazon Nova and redefine what’s possible!