Unlocking the Power of Customization: A Deep Dive into Amazon Nova Foundation Models at AWS Summit NYC

Exploring Model Customization Capabilities: A Comprehensive Overview

Direct Preference Optimization: Enhancing Model Outputs with Your Preferences

Streamlined Customization Workflow: Leveraging Amazon SageMaker AI

Adapting Amazon Nova Micro: A Practical Implementation Walk-Through

Preparing Your Dataset: Key Steps for Optimal Model Training

Fine-Tuning with DPO: Achieving Superior Model Performance

Evaluating Model Performance: A Comparative Analysis of Evaluation Metrics

Deploying Your Custom Model: Seamless Integration with Amazon Bedrock

Conclusion: Embracing Advanced AI Customization with Amazon Nova

Meet the Authors: Insights from the Team Behind Amazon Nova

Unlocking the Future of AI: Customizing Amazon Nova Models with DPO Recipes at AWS Summit NYC

At the recent AWS Summit in New York City, a significant announcement resonated throughout the AI community: Amazon introduced a comprehensive suite of model customization capabilities for the Amazon Nova foundation models. With the launch of ready-to-use recipes on Amazon SageMaker AI, users can now adapt various models—Nova Micro, Nova Lite, and Nova Pro—throughout the model training lifecycle, encompassing pre-training, supervised fine-tuning, and alignment.

A Sneak Peek into Customization Recipes

This multi-post series will delve into these customization recipes, starting with an alignment technique called Direct Preference Optimization (DPO). DPO allows you to finely tune model outputs based on your preferences through an intuitive method: supplying the model with two responses—one preferred and the other less so. By guiding the model with this structured feedback, you can foster outputs that resonate better with your brand’s tone, style, and guidelines. Depending on your data volume and budget, you can choose between parameter-efficient or full model DPO approaches.

Once trained, these customized models can be seamlessly deployed to Amazon Bedrock for inference, utilizing provisioned throughput, while the parameter-efficient version supports on-demand inference. The flexibility extends further—Nova customization recipes are compatible with SageMaker training jobs and SageMaker HyperPod, allowing you to tailor the environment to your scale requirements.

Streamlined Customization with SageMaker Training Jobs

Today, we’re focusing on a streamlined approach to customizing Amazon Nova Micro using SageMaker training jobs. Here’s how the workflow operates:

1. Selecting a Recipe: Users initiate by selecting a specific Nova customization recipe, which offers detailed configurations for training parameters, model settings, and distributed training strategies. Default configurations optimized for the SageMaker AI environment can be utilized or modified for experimentation.

2. API Request Submission: An API request is sent to the SageMaker AI control plane, carrying the chosen Amazon Nova recipe configuration.

3. Managed Compute Cluster Execution: SageMaker executes the recipe on a managed compute cluster through a training job launcher script. It provisions the necessary infrastructure and orchestrates distributed training, automatically decommissioning the cluster upon job completion.

This comprehensive architecture simplifies the customization experience, allowing users to swiftly define training parameters while SageMaker handles the underlying infrastructure—billed only for the duration of training in seconds.

Business Impact: Optimizing for Function Calling

The implementation walkthrough zeroes in on adapting the Amazon Nova Micro model for structured function calling, proving to significantly enhance performance metrics. Our findings indicate an 81% increase in the F1 score and up to 42% improvement in ROUGE metrics. Such enhancements mean that these models can effectively drive various business applications, like AI customer support systems, intelligent digital assistants, and workflow automation in fields such as e-commerce and finance.

Training the Model: Data Preparation & Execution

The initial step involves preparing the dataset using the nvidia/When2Call dataset, designed for training AI assistants to make optimal tool-use decisions. You’ll format user prompts, preferred outputs, and non-preferred outputs into a suitable training structure.

Next, the DPO training takes place on SageMaker, leveraging the PyTorch Estimator class for fine-tuning. You’ll identify the instance type, configure input channels, and submit the fitting job—all while maintaining oversight through Amazon CloudWatch logs for robust observability.

Evaluation and Results

After fine-tuning, model evaluation follows. Using evaluation recipes allows for an in-depth assessment of performance across various metrics. Our evaluation shows marked improvements:

• F1 Score: Increased by 81%
• F1 Quasi: Improved by 40%
• ROUGE Scores: Gains of 39% and 42% for ROUGE-1 and ROUGE-2, respectively.

Deployment on Amazon Bedrock

Finally, deploying the fine-tuned model using the Amazon Bedrock CreateCustomModel API enables on-demand inference with the integrated tooling. The process involves creating a custom model and configuring it for real-time operational efficiency.

Conclusion: A New Era of Customization

This blog post illustrates how businesses can leverage DPO recipes to customize Amazon Nova models effectively. The exploration highlights the capability to achieve dramatic performance improvements—culminating in a powerful tool for meeting industry-specifications and enhancing customer interactions.

To kickstart your journey with the Nova customization recipes, check out the SageMaker HyperPod recipes repository and Amazon Nova Samples repository. The AWS team is continually expanding this suite based on customer feedback, ensuring alignment with evolving machine learning trends.

About the Authors

• Mukund Birje: Sr. Product Marketing Manager at AWS.
• Karan Bhandarkar: Principal Product Manager, focusing on model customization.
• Kanwaljit Khurmi: Principal Worldwide Generative AI Solutions Architect.
• Bruno Pistone: Senior Generative AI/ML Specialist Solutions Architect.

Stay tuned for the next posts in this series, where we will explore additional customization techniques and real-world applications!