Exciting Update: Amazon Bedrock Custom Model Import Now Supports Qwen Models!

Deploying Qwen 2.5 Models Efficiently on AWS

An Overview of Qwen Models: Key Features and Applications

Introduction to Amazon Bedrock Custom Model Import

Prerequisites for Importing Qwen Models into Amazon Bedrock

Use Case 1: Building a Qwen-Powered Coding Assistant

Use Case 2: Leveraging Qwen 2.5 VL for Image Understanding

Pricing Model for Amazon Bedrock Custom Model Import

Important Steps for Cleanup Post-Experimentation

Conclusion: Optimizing AI Workloads with Qwen and Amazon Bedrock

Meet the Authors: Experts Behind the Innovation

Unlock Advanced AI with Amazon Bedrock: Custom Model Import Now Supports Qwen Models!

We’re thrilled to share exciting news for developers and organizations leveraging AI capabilities within the AWS ecosystem. Amazon Bedrock’s Custom Model Import now supports Qwen models, including custom weights for the Qwen2, Qwen2_VL, and Qwen2_5_VL architectures. This development enables users to bring tailored Qwen models like Qwen 2, Qwen 2.5 Coder, Qwen 2.5 VL, and QwQ 32B into a fully managed, serverless environment without the hassle of managing infrastructure.

Deploying Qwen 2.5 Models with Amazon Bedrock Custom Model Import

In this post, we’ll explore how to deploy Qwen 2.5 models using Amazon Bedrock Custom Model Import, giving organizations access to state-of-the-art AI capabilities while optimizing costs effectively.

Overview of Qwen Models

The Qwen 2 and 2.5 families consist of large language models that cater to diverse needs with various sizes and specialized variants:

• General Language Models: Ranging from 0.5B to 72B parameters, available in both base and instruct versions for general tasks.
• Qwen 2.5 Coder: Specialized for code generation and completion.
• Qwen 2.5 Math: Focused on advanced mathematical reasoning.
• Qwen 2.5 VL (Vision-Language): Offers image and video processing capabilities, enabling rich multimodal applications.

Overview of Amazon Bedrock Custom Model Import

Amazon Bedrock Custom Model Import allows for seamless integration of your custom models alongside existing foundation models (FMs) via a single, serverless API. With this feature, you can access your imported models on-demand without the need to manage infrastructure.

Accelerate the development of generative AI applications by integrating custom models with Amazon Bedrock’s native tools such as Knowledge Bases, Guardrails, and Agents. This feature is currently available in various regions, including US-East (N. Virginia), US-West (Oregon), and Europe (Frankfurt).

Prerequisites

Before embarking on your Qwen model integration journey, ensure you meet the following prerequisites:

• An active AWS account
• An Amazon S3 bucket to store your Qwen model files
• Sufficient permissions to create Amazon Bedrock model import jobs
• Confirmation that your region supports Amazon Bedrock Custom Model Import

Use Case 1: Qwen Coding Assistant

Let’s delve into creating a coding assistant using the Qwen2.5-Coder-7B-Instruct model.

1. Obtain Model ID: Search for and copy the Model ID from Hugging Face.

2. Download the Model: Use the Hugging Face library to download:

   from huggingface_hub import snapshot_download
   
   snapshot_download(repo_id="Qwen/Qwen2.5-Coder-7B-Instruct", 
                     local_dir=f"./extractedmodel/")

3. Upload to S3: Upload the model files to your S3 bucket:

   aws s3 cp ./extractedfolder s3://yourbucket/path/ --recursive

4. Start Model Import Job:

   response = self.bedrock_client.create_model_import_job(
       jobName="uniquejobname",
       importedModelName="uniquemodelname",
       roleArn="fullrolearn",
       modelDataSource={
           's3DataSource': {
               's3Uri': "s3://yourbucket/path/"
           }
       }
   )

5. In the Console: You can also perform this process via the AWS Management Console.

Once imported, you can interact with the model effectively. For example, to list items in an S3 bucket, use:

prompt = "Write sample boto3 python code to list files in a bucket stored in the variable `my_bucket`"
messages = [
    {"role": "system", "content": "You are a helpful coding assistant."},
    {"role": "user", "content": prompt}
]

Use Case 2: Qwen 2.5 VL for Image Understanding

Qwen2.5-VL models combine vision and language understanding and are perfect for analyzing multimedia inputs.

1. Download the Model:

   from huggingface_hub import snapshot_download
   
   hf_model_id = "Qwen/Qwen2.5-VL-7B-Instruct"
   snapshot_download(repo_id=hf_model_id, local_dir="./{local_directory}")

2. Import to Amazon Bedrock using either Console or API.

3. Testing Vision Capabilities:

   def generate_vl(messages, image_base64, temperature=0.3, max_tokens=4096, top_p=0.9):
       processor = AutoProcessor.from_pretrained("Qwen/QVQ-72B-Preview")
       prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
   
       response = client.invoke_model(
           modelId=model_id,
           body=json.dumps({
               'prompt': prompt,
               'temperature': temperature,
               'max_gen_len': max_tokens,
               'top_p': top_p,
               'images': [image_base64]
           }),
           accept="application/json",
           contentType="application/json"
       )
   
       return json.loads(response['body'].read().decode('utf-8'))

Pricing Model

Amazon Bedrock Custom Model Import allows you to use your model weights without any import charges, but you will be billed for inference based on active model copies and their duration. Amazon Bedrock manages scaling automatically to optimize performance and costs.

Clean Up

To avoid unnecessary charges post-experimentation, delete your imported Qwen models from the console or API, along with any model files from your S3 bucket if they’re no longer required.

Conclusion

Amazon Bedrock Custom Model Import empowers organizations to harness sophisticated models like Qwen 2.5, benefitting from a streamlined serverless infrastructure. This allows your teams to focus on application development rather than managing complex environments. Coupled with AWS’s robust security and compliance features, this integration promises an ideal blend of performance and cost-effectiveness.

For further details, check out the Amazon Bedrock User Guide.

About the Authors

Ajit Mahareddy: Product and Go-To-Market leader with over 20 years of experience. Passionate about advancing Generative AI technologies.

Shreyas Subramanian: Principal Data Scientist specializing in generative AI and deep learning applications using AWS services.

Yanyan Zhang: Senior Generative AI Data Scientist focused on innovative AI/ML applications.

Dharinee Gupta: Engineering Manager at AWS Bedrock, enabling seamless model utilization in serverless solutions.

Lokeshwaran Ravi: Senior Deep Learning Compiler Engineer working on ML optimization and AI security.

June Won: Principal Product Manager focused on enhancing discoverability and usability of foundation models.

Get ready to unleash the power of advanced AI with the new capabilities available through Amazon Bedrock!