Deploying Mercury and Mercury Coder Foundation Models on AWS
Introduction
Today, we are excited to announce that Mercury and Mercury Coder foundation models (FMs) from Inception Labs are available through Amazon Bedrock Marketplace and Amazon SageMaker JumpStart. With this launch, you can deploy the Mercury FMs to build, experiment, and responsibly scale your generative AI applications on AWS.
About Mercury Foundation Models
Mercury is the first family of commercial-scale diffusion-based language models, offering groundbreaking advancements in generation speed while maintaining high-quality outputs. Unlike traditional autoregressive models that generate text one token at a time, Mercury models use diffusion to generate multiple tokens in parallel through a coarse-to-fine approach.
About Amazon Bedrock Marketplace
Amazon Bedrock Marketplace plays a pivotal role in democratizing access to advanced AI capabilities through several key advantages, including comprehensive model selection, a unified and secure experience, and scalable infrastructure.
Deploy Mercury and Mercury Coder Models in Amazon Bedrock Marketplace
Amazon Bedrock Marketplace provides access to over 100 foundation models. Follow these steps to access the Mercury models in Amazon Bedrock.
SageMaker JumpStart Overview
SageMaker JumpStart is a fully managed service that offers state-of-the-art FMs for various use cases and provides a collection of pre-trained models that you can deploy quickly.
Prerequisites
Check that you have access to the recommended instance types and that your SageMaker IAM service role has the necessary permissions to deploy the model.
Subscribe to the Model Package
Follow the steps to subscribe to the Mercury or Mercury Coder model package through AWS Marketplace.
Deploy Mercury and Mercury Coder Models on SageMaker JumpStart
For quick deployment, use SageMaker Studio to access the models, with options to deploy programmatically through the SageMaker Python SDK.
Deploy Mercury Using the SageMaker SDK
Learn how to deploy the model using the SDK, including necessary code snippets for effortless implementation.
Use Mercury for Code Generation
Discover how to leverage Mercury for generating functional code snippets quickly and efficiently.
Use Mercury for Tool Use and Function Calling
Explore Mercury’s advanced tool use capabilities, enabling it to intelligently perform calculations and call external functions based on your queries.
Clean Up
Follow these steps to avoid unwanted charges after deploying your models on Amazon Bedrock Marketplace and SageMaker JumpStart.
Conclusion
In this post, we explored how to access and deploy Mercury models using Amazon Bedrock Marketplace and SageMaker JumpStart, emphasizing their exceptional speed and performance for generative AI workloads.
About the Authors
Meet the experts behind this initiative, each contributing their unique expertise to enhance the capabilities of generative AI on AWS.
Unlocking the Future: Deploying Mercury Models on AWS with Ease
Today marks an exciting milestone for developers and AI enthusiasts alike with the announcement that Mercury and Mercury Coder foundation models (FMs) from Inception Labs are now available through the Amazon Bedrock Marketplace and Amazon SageMaker JumpStart. In this blog post, we’ll guide you through how to get started with these cutting-edge models, revolutionizing the way we approach generative AI applications on AWS.
About Mercury Foundation Models
Mercury is not just another AI model; it’s the first family of commercial-scale diffusion-based language models. Designed for speed and efficacy, the Mercury models outperform traditional autoregressive models by generating multiple tokens in parallel using a coarse-to-fine approach. This leap in technology translates to ultra-fast generation speeds of up to 1,100 tokens per second on NVIDIA H100 GPUs—up to 10 times faster than comparable models.
Key Features of Mercury Coder Models:
- High-Quality Code Generation: Supports multiple programming languages, making it your go-to for Python, Java, JavaScript, C++, PHP, Bash, and TypeScript.
- Fill-in-the-Middle Tasks: Particularly strong in code completion and editing workflows.
- Transformer-Based Architecture: Compatible with existing optimization techniques and infrastructure.
- Extended Context Length: Out of the box support for 32,768 tokens, extending up to 128,000 tokens with advanced techniques.
About Amazon Bedrock Marketplace
The Amazon Bedrock Marketplace is instrumental in democratizing access to advanced AI capabilities. Here are some advantages it brings:
- Comprehensive model selection: Choose from a broad range of models tailored for specific use cases.
- Unified and Secure Experience: Simplify integration using Amazon Bedrock APIs while employing robust security measures.
- Scalable Infrastructure: Managed endpoints allow for configurable scaling, optimizing costs and performance.
How to Deploy Mercury and Mercury Coder Models
Getting started with Mercury models on the Amazon Bedrock Marketplace is straightforward:
- Access the Console: Go to the Amazon Bedrock console and navigate to the Foundation models section.
- Model Catalog: Filter for Inception to access Mercury models, and check the details including capabilities and pricing.
- Subscribe and Deploy: Click “Subscribe” to the model and then “Deploy.” You will be prompted to enter deployment details such as endpoint name, instance type, etc.
- Test the Model: Once deployment is done, explore its capabilities in the Amazon Bedrock playground.
SageMaker JumpStart Overview
SageMaker JumpStart facilitates quick and effective deployment of state-of-the-art FMs across various applications. You can deploy Mercury and Mercury Coder models directly within SageMaker Studio or programmatically via the SageMaker Python SDK.
Prerequisites for Deployment:
Ensure that you have access to the recommended instance types:
- Check quotas via the Service Quotas console.
- Verify necessary IAM permissions for deploying the model.
Quick Start Guide: Subscribe to the Model Package
To subscribe to the Mercury model package:
- Access the Model Package: Open the listing page for Mercury or Mercury Coder.
- Review Terms: Accept the EULA and pricing.
- Configure Region: Choose the right AWS Region that supports your instance type.
Deploying with the SageMaker SDK
For a more programmatic approach, you can utilize the SageMaker SDK:
# Create the model package
endpoint_name = name_from_base("mercury-endpoint")
model = ModelPackage(role=role_arn, model_package_arn=package_arn, sagemaker_session=sagemaker_session)
# Deploy the Model
instance_type = "ml.p5.48xlarge"
deployed_model = model.deploy(initial_instance_count=1, instance_type=instance_type, endpoint_name=endpoint_name)Use Cases: Code Generation and Function Calling
Code Generation Example
Imagine asking the Mercury model to generate a simple tic-tac-toe game:
payload = {
"messages": [{
"role": "user",
"content": "Build a simple tic-tac-toe game."
}],
"max_tokens": 2500,
}
outputs = predictor.predict(payload)The model responds with a functional HTML, CSS, and JavaScript implementation at an impressive speed of 528 tokens per second.
Tool Use and Function Calling Example
Mercury models support advanced tool use. For instance, you can create a travel planning assistant capable of checking the weather and performing calculations:
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"parameters": {"location": "Tokyo", "unit": "celsius"}
}
},
{
"type": "function",
"function": {
"name": "calculate",
"parameters": {"expression": "1000 * 150"}
}
}
]This setup allows for seamless interaction with external APIs for enriched user experiences.
Clean-Up Resources
To avoid unintentional costs, it’s essential to clean up any resources after your testing:
Delete Amazon Bedrock Marketplace Deployment
- Navigate to Marketplace deployments on the Amazon Bedrock console.
- Select the endpoint to delete and confirm.
Delete SageMaker JumpStart Endpoint Code
Use the following code snippet:
sm.delete_model(ModelName=sm_model_name)
sm.delete_endpoint(EndpointName=endpoint_name)Conclusion
In this post, we explored how to access and deploy Mercury models using Amazon Bedrock Marketplace and SageMaker JumpStart. These advanced diffusion-based models offer unparalleled speed and precision, making them suitable for various generative AI applications.
Dive into the Amazon Bedrock Marketplace or SageMaker Studio to start your journey with Mercury models today!
About the Authors
Meet our expert team specializing in AI and cloud solutions, committed to helping you navigate the future of artificial intelligence and machine learning.
Feel free to share your thoughts or experiences with deploying Mercury models below!