Ensuring Data Privacy in Multi-Account Deployments with Amazon Bedrock
Introduction
Data privacy is a critical concern for software companies in the data management space. To build customer trust, companies must demonstrate their commitment to maintaining data confidentiality.
Challenges in Logging with Amazon Bedrock
Observability is crucial for AI implementations, and routing logs to customer accounts can present challenges.
Solution Overview
This section provides an outline of model invocations and how operations are managed in a multi-account environment.
Enabling Cross-Account Access with IAM Roles
A focus on using IAM roles to simplify model access and permission management across multiple customer accounts.
Writing Private Logs Using LangChain Callbacks
Implementing a custom logging solution with LangChain that ensures data privacy in multi-account configurations.
The AWS Shared Responsibility Model in Multi-Account Logging
Understanding the shared responsibility model is crucial for ensuring the security and privacy of data.
Conclusion
Best practices for building secure, scalable architectures while maintaining data privacy and observability in AI operations.
About the Authors
Learn more about the experts behind this insightful look into data privacy and AI solutions on AWS.
Securing Data Privacy in Multi-Account Architectures with Amazon Bedrock
Data privacy is a critical issue for software companies in the data management space. To earn and maintain customer trust, these companies must demonstrate that customer data will remain confidential and secure within controlled environments. One strategy employed by some organizations is the adoption of a multi-account architecture, where each customer’s data resides in a separate AWS account. This approach facilitates strict security boundaries, minimizes the risk of cross-customer data leakages, and supports compliance with pivotal regulations such as HIPAA and GDPR.
The Gold Standard: Multi-Account Deployment
Multi-account deployment represents the pinnacle of cloud data privacy. It allows software companies to ensure that customer data remains segregated, even at massive scale. This is underscored by the AWS Well-Architected Framework, which highlights the security isolation boundaries provided by AWS accounts.
As companies increasingly incorporate generative AI capabilities, like Amazon Bedrock, which offers managed foundation models complete with robust security features, they face unique challenges. Managing a multi-account architecture powered by Amazon Bedrock introduces complexities around access control, quota management, and operational visibility, particularly as the number of AWS accounts expands into double digits.
A Centralized Operations Approach
To address these challenges, organizations can configure a dedicated operations account. This approach centralizes management, ensuring that while customer data traverses managed services, it is stored solely within the respective customer accounts. By consolidating operations into a single account but maintaining data in isolated environments, organizations can streamline model access and quotas while preserving strict data boundaries.
Challenges in Logging with Amazon Bedrock
Effective observability is critical for AI implementations. Organizations cannot optimize what they fail to measure. Amazon Bedrock offers built-in logging capabilities for model invocations to Amazon CloudWatch or Amazon S3. However, routing logs presents two significant challenges:
- Logs may contain sensitive customer data that must not be stored in the operations account for longer than the required retention period, and
- CloudWatch limits the number of monitoring accounts to five.
So how can organizations build a secure, scalable logging solution that complies with privacy regulations?
Enabling Distributed Logging for Amazon Bedrock
This post introduces a solution for distributed logging in multi-account deployments of Amazon Bedrock. The goal is to maintain robust AI observability while ensuring strict privacy for data at rest. The key to this approach is to configure logging within customer accounts instead of the operations account.
Benefits of this Architecture
This architecture uses AWS Security Token Service (AWS STS), allowing customer accounts to assume specific IAM roles in the operations account when invoking Amazon Bedrock. For logging, LangChain callbacks capture invocation metadata directly in each customer’s account, keeping the operational account’s memory free of sensitive logs.
Steps in the Model Invocation Process
- The customer IAM role assumes the necessary role in the operations account via the AWS STS AssumeRole operation, establishing cross-account access.
- The operations account verifies whether the requesting principal from the customer account is authorized to assume the role based on its trust policy.
- Temporary credentials are issued to the customer account’s IAM role, enabling access to Amazon Bedrock in the operations account.
- The customer account invokes the Amazon Bedrock client, consuming model quotas from the operations account.
- Finally, metrics from the Amazon Bedrock client response are logged into the customer account’s CloudWatch via LangChain callbacks.
Enabling Cross-Account Access with IAM Roles
A crucial component of this solution is the IAM role for each customer in the operations account. This role allows the software company to manage permissions effectively, defining which models can be invoked, in which AWS regions, and the relevant quotas.
Given that enterprise customers may have multiple AWS accounts, this centralized approach supports consistent access policies while simplifying permission management and cost tracking.
Control via Trust Relationships
Carefully crafted trust relationships govern access, ensuring that only authorized accounts can assume specific roles. This prevents vulnerabilities such as the “confused deputy” problem. The operations account maintains security boundaries while providing clients the flexibility needed for multi-account environments.
Writing Private Logs with LangChain Callbacks
LangChain is a versatile open-source framework that enables developers to build complex applications by chaining data processing operations. By extending the BaseCallbackHandler class, developers can create custom behaviors for capturing logs without exposing sensitive data through operational accounts.
Example Implementation
The following Python script exemplifies a custom logging handler with LangChain:
class CustomCallbackHandler(BaseCallbackHandler):
def log_to_cloudwatch(self, message: str):
# Implementation to write metrics to CloudWatch
pass
def on_llm_end(self, response, **kwargs):
# Extract and log relevant metrics
input_tokens = response.llm_output.get("usage", {}).get("prompt_tokens", 0)
output_tokens = response.llm_output.get("usage", {}).get("completion_tokens", 0)
model_id = response.llm_output.get("model_id", None)
self.log_to_cloudwatch(f"Input tokens: {input_tokens}\nOutput tokens: {output_tokens}\nInvoked model: {model_id}")
def on_llm_error(self, error: Exception, **kwargs):
# Log errors encountered during model execution
print(f"Model error: {error}")This architecture enables comprehensive audit trails of AI interactions while maintaining strict data isolation, as logs never transit through or rest in the operations account.
The AWS Shared Responsibility Model
Understanding the AWS Shared Responsibility Model is crucial when implementing distributed logging for Amazon Bedrock in multi-account structures. While AWS provides underlying infrastructure security, customers remain accountable for protecting their data, configuring access controls, and enacting proper logging strategies.
The LangChain callback implementation shifts the responsibility toward customers, ensuring they handle appropriate encryption, retention periods, and access controls. This respects the multi-account design principle, keeping customer data isolated within their respective domains.
Conclusion
Building a secure, scalable multi-tenant architecture with Amazon Bedrock demands meticulous planning across various dimensions, including account structure, access patterns, and operational management. The distributed logging approach outlined herein allows organizations to maintain strict data isolation while benefiting from centralized AI operations.
By leveraging IAM roles, AWS STS, and LangChain callbacks, companies can establish a secure foundation that scales effectively while upholding privacy standards. Organizations should prioritize automation, monitoring, and governance early in the development process to mitigate technical debt as systems expand.
As generative AI continues to evolve, these architectural patterns offer a roadmap for preserving the highest standards of data privacy while innovating AI capabilities across diverse regulatory landscapes.
To learn more, explore the pertinent frameworks for securing AI implementations, ensuring your architecture remains resilient and compliant as your organization evolves.
About the Authors
Mohammad Tahsin is an AI/ML Specialist Solutions Architect at AWS, passionate about the latest AI/ML technologies. In his spare time, he enjoys gaming, digital art, and cooking.
Felipe Lopez is a Senior AI/ML Specialist Solutions Architect at AWS, with prior experience in industrial modeling and optimization.
Aswin Vasudevan is a Senior Solutions Architect for Security at AWS, enthusiastic about generative AI and serverless architecture, dedicated to helping clients achieve business value.