Architecting Multi-Tenant Agentic Applications with Amazon Bedrock AgentCore

1. Introduction to Multi-Tenant Architectures

2. Design Considerations for Building Multi-Tenant Agents

2.1 Agent Runtime Deployment: Dedicated vs. Shared

2.2 Shared vs. Tier-Specific vs. Fine-Tuned Models

2.3 Workflow Management Patterns: Silo, Pool, and Bridge

2.4 Multi-Tenant Retrieval Augmented Generation (RAG)

2.5 Tenant Context, Act-on-Behalf Patterns, and Token Propagation

2.6 Fine-Grained Access Control for MCP Tools and APIs

2.7 Hierarchical Namespace Isolation in Memory Management

2.8 Agent Identity, Trust, and Discovery

2.9 Cost Tracking per Tenant and Observability

2.10 Implementing Guardrails for Content Safety

3. Implementation Models: Silo, Pool, and Bridge

3.1 Silo Model Implementation with AgentCore

3.2 Pool Model Implementation with AgentCore

3.3 Bridge Model Implementation with AgentCore

4. Future Directions in Multi-Tenant Agent Architectures

5. Conclusion: Building Production-Ready Multi-Tenant Agents

6. About the Authors

Building Multi-Tenant Agentic Applications with Amazon Bedrock AgentCore: Part 1

In the fast-evolving landscape of Software as a Service (SaaS), the challenge of creating multi-tenant agentic applications extends well beyond standard concerns of security, governance, and responsiveness. The focus is shifting toward a nuanced architecture that encompasses tenant isolation, identity management, observability, and data handling, all while mitigating costs and addressing the noisy neighbor effect.

In this first installment of our blog series, we delve into the essential architectural considerations for building multi-tenant agentic applications using Amazon Bedrock AgentCore, a robust managed, serverless solution for developing and deploying these applications on AWS.

Design Considerations for Building a Multi-Tenant Agent

When developing secure multi-tenant applications, key architectural decisions must balance tenant isolation with operational efficiency, while adhering to security and compliance standards. Three primary patterns for tenant isolation—Silo, Pool, and Bridge—provide different trade-offs depending on your needs.

1. Agent Runtime Deployment: Dedicated vs. Shared

One of the foundational decisions in multi-tenant architecture concerns how to provision an agent runtime for tenants.

• Dedicated Runtime: This approach assigns a separate execution environment for each tenant, offering high isolation and simplifying compliance audits but at a higher operational cost.

• Shared Runtime: Here, multiple tenants share the same environment, which lowers costs but requires strict management of tenant context.

Amazon Bedrock AgentCore Runtime addresses these challenges with session-isolated microVM-based compute. By launching lightweight microVMs per session, it maintains persistent file systems while minimizing cost and latency. Tenant context is included using custom HTTP headers, ensuring that each agent is fully aware of its business logic and requirements.

2. Shared vs. Tier-Specific vs. Fine-Tuned Models

For many applications, shared foundation models (FMs) provide a good starting point, promising streamlined operations with updates affecting all tenants.

However, as tenant needs diverge, the hybrid approach of combining shared models for standard tiers with fine-tuned options for premium tenants can accommodate specialized use cases. Amazon Bedrock simplifies this further by offering customizable large language models, enabling SaaS providers to adapt their offerings based on tiered needs.

3. Flexible Workflow Management

Effective workflow management is crucial in multi-tenant applications, where various tenants can have distinct sequences of execution steps.

The three workflow patterns—Silo, Pool, and Bridge—offer different avenues for customization:

• Silo Pattern provides full independence but involves higher maintenance costs.
• Pool Pattern uses shared agent skills, maximizing efficiency.
• Bridge Pattern combines shared processes with tenant-specific execution, allowing for a balanced approach.

4. Multi-Tenant Retrieval Augmented Generation (RAG)

In RAG systems, careful decisions regarding data isolation can either foster efficiency or compromise security.

The silo pattern utilizes dedicated vector databases per tenant for maximum security, while the pool pattern favors shared databases with robust metadata-based filters. Amazon Bedrock Knowledge Bases offers fully managed RAG capabilities, supporting both approaches depending on your compliance needs.

5. Tenant Context Management

Robust multi-tenant identity management is essential, ensuring that tenant context flows securely through each architectural layer.

Using patterns like impersonation and the more secure act-on-behalf, service security can be enhanced. Encoding tenant context in JSON Web Tokens (JWTs) allows for explicit permissions, providing a layer of confidence that each agent won’t access unwanted resources.

6. Fine-Grained Access Control

Multi-tenant applications require rigorous access control to restrict tool usage based on tenant needs. Policies assessing tenant context in real-time allow for more dynamic access, reducing reliance on static permissions.

Amazon Bedrock’s AgentCore Policy evaluates every agent request against tailored policies, promoting both flexibility and security at the invocation layer.

7. Memory Management and Isolation

Implementing ten-tier memory management shields tenant data from leakage. Access control based on hierarchical policies allows for nuanced management of global, tenant, and user-specific data.

The AgentCore Memory framework supports this structured approach, ensuring smooth agent experiences with both short- and long-term memory strategies.

8. Agent Identity, Trust, and Discovery

Ensuring that agents can verifiably identify, trust, and discover one another is critical as they interact across organizational boundaries.

Amazon Bedrock offers a solution with its Identity framework that integrates with existing identity providers, refining agent identities and establishing trust through robust verification processes.

9. Cost Tracking and Observability

Lastly, effective cost tracking is paramount for accurate attribution in multi-tenant environments. Instrumentation at the application level captures essential metrics, which can be analyzed for performance insights.

Amazon Bedrock Observability offers seamless integration for detailed monitoring, enhancing visibility into agent workflows.

10. Guardrails for Content Safety

Implementing guardrails ensures safety and compliance through robust content validation mechanisms. Amazon Bedrock Guardrails provides configurable options for blocking malicious inputs and securing data across tenant boundaries.

Conclusion

Building production-ready multi-tenant agentic applications demands a comprehensive architectural approach that addresses isolation, identity, cost, and security across all levels. With Amazon Bedrock AgentCore, organizations can leverage these foundational elements to simplify the complexities of multi-tenant architectures into scalable, secure solutions.

In our forthcoming posts, we will take a deeper dive into practical implementation strategies for both the pool and silo deployment models, addressing every crucial design consideration outlined in this post.

We encourage readers to engage with the multi-tenant agents workshop for hands-on experience, diving into the nitty-gritty of building these powerful applications with Amazon Bedrock AgentCore.

About the Authors

Dhawal Patel – Principal Generative AI Tech Lead at AWS, specializing in AI, deep learning, and distributed computing.

Anubhav Sharma – Principal Solutions Architect at AWS, focusing on SaaS solutions and AI transformation.

Aswin Vasudevan – Senior Solutions Architect for Security, ISV at AWS, passionate about generative AI.

Sahil Thapar – Principal Solutions Architect at AWS, dedicated to crafting resilient applications.

Ujwal Bukka – Senior Partner Solutions Architect at AWS, driving partner success through innovative AI solutions.

Stay tuned for the next installment in our series!