Announcing MLflow Version 3.10 Support in Amazon SageMaker AI MLflow Apps: Elevate Your Generative AI Development
Unlock Enhanced Experiment Tracking and Observability for Generative AI Workflows
What’s New in MLflow v3.10
Getting Started with SageMaker AI MLflow App v3.10
Migration
Conclusion
About the Authors
Unleashing the Power of Generative AI: Announcing MLflow v3.10 on Amazon SageMaker AI MLflow Apps
Today marks a significant milestone for AI developers and data scientists as we unveil Amazon SageMaker AI MLflow Apps’ support for MLflow version 3.10. This update not only enhances capabilities for generative AI development but also streamlines experiment tracking, ensuring that you can seamlessly transition from experimentation to production with enhanced efficiency and observability.
In this blog post, we’ll explore the groundbreaking features introduced in MLflow v3.10, guide you through setting up SageMaker AI MLflow Apps, and demonstrate how to harness these improvements for your generative AI applications.
What’s New in MLflow v3.10
MLflow 3.10 brings a wealth of targeted improvements to enhance the MLflow ecosystem, with a focus on generative AI application development and agentic workflows. Some of the most noteworthy features include:
Enhanced Tracing and Integration
This release significantly improves tracing for complex multi-turn workflows and offers tighter integration with popular LLM (Large Language Model) frameworks and libraries. This means easier tracking of interactions throughout your generative AI processes.
Streamlined Logging
Logging for generative AI interactions and invocations has been optimized. This simplification allows developers to record essential metrics without cumbersome manual input, resulting in more efficient workflows.
Advanced Evaluation Tools
The new mlflow.genai.evaluation() API introduces a programmatic interface that systematically measures and maintains generative AI quality throughout the developmental lifecycle. The built-in metrics cover critical areas like relevance, faithfulness, correctness, and safety, providing a comprehensive toolkit to evaluate and optimize your AI models effectively.
Improved Observability
The updates also include more granular trace filtering and search options, allowing for richer metadata capture that aids in debugging and root-cause analysis. Pre-built performance dashboards provide at-a-glance visibility into metrics like latency distributions, request counts, and quality scores—eliminating the need for time-consuming manual chart configurations.
These advancements create a robust environment for making informed decisions about resource allocation and operational costs as you scale your generative AI applications.
Getting Started with SageMaker AI MLflow App v3.10
For those who are new to Amazon SageMaker AI MLflow Apps, creating an instance is straightforward. You can get started through the SageMaker Studio console, AWS CLI, or API.
Prerequisites
To begin, ensure you have the following:
- An AWS account with access to Amazon SageMaker.
- AWS Identity and Access Management (IAM) configured for permissions.
- An Amazon S3 bucket for data storage.
Once you’re set up, navigate to the SageMaker Studio console, select the MLflow application, and choose “Create MLflow App.” Enter a name and modify your IAM role and S3 bucket configuration in the Advanced settings if needed.
After creation, you’ll receive an MLflow Amazon Resource Name (ARN) that allows you to connect with your newly created app instantly.
Installing Required Packages
To track experiments with your SageMaker AI MLflow App, install the necessary Python packages using pip:
pip install mlflow==3.10.1 sagemaker-mlflow==0.3.0Connecting to Your App
You can easily connect and start logging your AI experiments using the following code snippet (don’t forget to replace the placeholder ARN):
import mlflow
# Connect to your SageMaker MLflow App
mlflow_app_arn = "your_mlflow_app_arn"
mlflow.set_tracking_uri(mlflow_app_arn)
# Set your experiment
mlflow.set_experiment("your_genai_experiment")With these steps, you’re outfitted to utilize the enhanced capabilities of MLflow v3.10 in your existing code.
Migration
If you’re currently using an existing MLflow Tracking Server or App hosted on SageMaker or elsewhere, migrating to MLflow v3.10 is simple. Follow the instructions outlined in the blog post "Migrate MLflow tracking servers to Amazon SageMaker AI with serverless MLflow" to facilitate a smooth transition.
Conclusion
The introduction of MLflow v3.10 in Amazon SageMaker AI MLflow Apps is a significant leap forward, making enterprise AI development more efficient, observable, and manageable. Whether you are a seasoned developer or just starting with generative AI, the enhanced features of MLflow v3.10 offer invaluable tools to streamline your workflows.
Dive into Amazon SageMaker AI Studio today and create your first MLflow App to experience these innovations firsthand. The new MLflow version is also compatible with Amazon SageMaker AI serverless model customization and SageMaker Unified Studio, further enhancing your workflow flexibility.
We’d love to hear your feedback! Connect with us through AWS re:Post for SageMaker or your usual AWS Support contacts.
About the Authors
Sandeep Raveesh
Sandeep Raveesh is a GenAI GTM Specialist Solutions Architect at AWS. He collaborates with customers to enhance their LLM training and inference processes while developing solutions for industry challenges in the generative AI space.
Dana Benson
Dana Benson serves as a Software Development Manager focused on SageMaker AI ML and LLM observability. Before joining AWS, Dana was involved in developing Smart Home behaviors for Alexa.
Ruidi Peng
Ruidi Peng is a Software Development Engineer at AWS, dedicated to the Amazon SageMaker MLflow team. He emphasizes building scalable infrastructure that empowers customers to monitor and gain insights into their machine learning workloads, while enjoying outdoor adventures in his spare time.
Embark on your generative AI journey with SageMaker AI MLflow Apps today!