Understanding Granular Cost Attribution for Amazon Bedrock Inference: A Guide to Tracking and Optimizing Cloud Expenses

Key Takeaways:

• Overview of Amazon Bedrock’s new cost attribution feature
• How to effectively trace inference costs to specific IAM principals
• Step-by-step guide for implementing cost tracking and aggregation

Introduction

As AI inference gains prominence in cloud spending, understanding the driving factors behind these costs has become crucial for organizations eager to optimize finances. This post introduces Amazon Bedrock’s granular cost attribution capabilities.

How Granular Cost Attribution Works

• Understanding IAM Principal Attribution
• Incorporating Cost Allocation Tags for Better Insights

Tracking Costs by Access Patterns

Scenario 1: IAM Users and API Keys

Scenario 2: Applications with IAM Roles

Scenario 3: Federated Authentication

Scenario 4: LLM Gateway Implementations

Steps for Cost Aggregation and Reporting

• Setting Up Cost Tags
• Enabling IAM Principal Data in AWS Billing
• Using AWS Cost Explorer for Visualization

Conclusion

Leverage Amazon Bedrock’s cost attribution capabilities to enhance your financial planning and optimize AI inference expenses effectively.

About the Authors

• Profiles of the team behind Amazon Bedrock’s cost management features.

Understanding Granular Cost Attribution for Amazon Bedrock Inference

As AI inference offerings expand, they’re taking on a larger share of cloud expenses. This surge underscores the necessity for organizations to grasp precisely who and what are driving these costs, allowing for effective chargebacks, cost optimization, and financial forecasting. This brings us to Amazon Bedrock’s latest feature: granular cost attribution for inference calls.

What’s New?

Today, we’re thrilled to announce that Amazon Bedrock now automatically attributes inference costs to the Identity and Access Management (IAM) principals that initiate these calls. An IAM principal could be an IAM user, an application-assumed role, or a federated identity from providers like Okta or Entra ID. This attribution flows seamlessly into your AWS Billing, accommodating various models, without requiring additional resource management or altering your existing workflows.

Additionally, through optional cost allocation tags, you can further categorize costs by teams, projects, or any custom dimension using AWS Cost Explorer and AWS Cost and Usage Reports (CUR 2.0).

How Granular Cost Attribution Works

When you enable IAM principal data in your CUR 2.0 configuration, you’ll see which IAM principals are interacting with Amazon Bedrock and the costs associated with their usage. For instance:

In this example, Alice and Bob’s expenses for different models and token types are clearly detailed, allowing organizations to track usage effectively.

Adding Tags for Aggregation and Cost Exploration

To aggregate costs by team, project, or cost center, you can incorporate tags into your IAM principals.

• Principal Tags: These are attached directly to IAM users or roles and apply consistently across all requests from that principal.
• Session Tags: These are more dynamic, applied when a user or application assumes a role for temporary credentials.

After activating these tags in AWS Billing, they will begin appearing in CUR 2.0 and can be utilized in Cost Explorer.

Quickstart by Scenario

Your approach will depend on how users and applications engage with Amazon Bedrock. Here’s a breakdown:

Detailed Scenarios

1. Per-User Tracking with IAM Users and API Keys

   • Ideal for small teams or development environments. Each user’s spending is directly linked to their IAM credentials, making it easy to track costs.

2. Per-Application Tracking with IAM Roles

   • Suitable for production workloads. Applications assume specific IAM roles, providing visibility into costs associated with each application.

3. Per-User Tracking with Federated Authentication

   • Best for enterprises leveraging identity providers. Users can utilize a shared IAM role while still maintaining individual visibility into their costs.

4. Per-User Tracking Through an LLM Gateway

   • For organizations using a gateway that processes requests before contacting Amazon Bedrock. Implementing per-user session management is key for accurate cost attribution.

Getting Started

To utilize Amazon Bedrock’s new cost attribution feature, follow these steps:

1. Identify Your Access Pattern: Determine whether you’re using IAM users, API keys, or roles.
2. Enable IAM Principal Data: Configure your CUR 2.0 to include IAM principal data.
3. Add Tags for Aggregation: If aggregation is needed, implement tags within IAM users, roles, or configure your IdP for session tagging.
4. Analyze: Within 24-48 hours post-activation, your tags will be visible in Cost Explorer and CUR 2.0, allowing for sophisticated analysis.

Conclusion

Understanding where costs are accruing from inference requests is essential for effective financial planning, chargebacks, and optimization strategies. With Amazon Bedrock’s granular cost attribution, businesses can pinpoint spending back to specific users, applications, or identities. This visibility empowers organizations to not only track their costs but also to optimize their spending strategies moving forward.

About the Authors

Ba’Carri Johnson is a Sr. Technical Product Manager at Amazon Bedrock, dedicated to cost management and governance for AWS AI.

Vadim Omeltchenko serves as a Sr. Solutions Architect focused on helping AWS customers innovate.

Ajit Mahareddy has a rich background in product management, specializing in AI/ML products from leading technology firms.

Sofian Hamiti leads technological initiatives with over a decade of experience in AI solutions.

Stay tuned for more insights as we continue to empower organizations to optimize and scale their AI capabilities!