Transforming Data Analysis with AI: BGL’s Journey Using Claude Agent SDK and Amazon Bedrock AgentCore

Transforming Data Analysis with AI Agents: A Case Study from BGL

This post is cowritten with James Luo from BGL.

Data analysis is rapidly emerging as a pivotal application for AI agents. According to Anthropic’s 2026 State of AI Agents Report, 60% of organizations view data analysis and report generation as their most impactful usage of agentic AI, with 65% of enterprises considering it a top priority. However, businesses often encounter two major challenges:

1. Reliance on Data Teams: Business users lacking technical backgrounds depend on data teams for queries, generating time-consuming bottlenecks.
2. Limitations in Traditional Solutions: Conventional text-to-SQL solutions frequently deliver inconsistent and inaccurate results.

BGL, a leader in self-managed superannuation fund (SMSF) administration solutions, experienced these very challenges. Operating in 15 countries and serving over 12,700 businesses, BGL needed to efficiently process complex compliance and financial data to answer critical business questions like, “Which products had the most negative feedback last quarter?” or “Show me investment trends for high-net-worth accounts.”

In partnership with Amazon Web Services (AWS), BGL developed an AI agent utilizing the Claude Agent SDK hosted on Amazon Bedrock AgentCore. This agent empowers business users to extract analytic insights through natural language, all while meeting stringent security and compliance standards.

In this blog post, we delve into BGL’s process of creating a production-ready AI agent and explore three key aspects:

1. Establishing a Strong Data Foundation
2. Designing the AI Agent with Claude Agent SDK
3. Leveraging Amazon Bedrock for Scalable Execution

Setting Up Strong Data Foundations for AI-Driven Text-to-SQL Solutions

A common issue in the implementation of AI agents for analytics is overburdening the agent with tasks it shouldn’t manage alone. By attempting to grasp database schemas, manipulate complex datasets, interpret business logic, and analyze results, the AI is likely to produce inconsistent outcomes.

BGL tackled this head-on by leveraging its established big data architecture powered by Amazon Athena and dbt Labs. This system effectively processes and transforms terabytes of raw data. The Extract, Transform, Load (ETL) process builds analytic tables designed to answer specific business queries, creating a reliable source of truth for business intelligence (BI).

Streamlining Complex Data Transformation through AI Agents

The agent’s primary role is to interpret natural language queries and generate SQL SELECT statements against structured analytic tables. Additionally, it can write Python scripts for processing results and generating visualizations. This division of responsibility significantly reduces the likelihood of errors and offers several advantages:

• Consistency: Complex business logic, such as joins and aggregations, is managed by the data team in a deterministic manner. This allows the AI agent to focus solely on interpreting questions and generating basic SELECT queries.

• Performance: The pre-aggregated analytic tables are optimized with the right indexes. The AI agent can execute simple queries, enhancing response times even when dealing with sizable datasets.

• Maintainability: As business rules evolve, updates are only required in the data transformation logic. The AI agent automatically uses the updated analytic tables, ensuring it always operates from the same, current data source.

“Many people think the AI agent is so powerful that they can skip building the data platform; they want the agent to do everything. But consistent and accurate results require division of complexity.”
— James Luo, Head of Data and AI at BGL

Building AI Agents Using Claude Agent SDK with Amazon Bedrock

BGL’s developers have been using Claude Code, powered by Amazon Bedrock, as an AI coding assistant. The integration includes temporary, session-based access to mitigate credential exposure, aligning with compliance mandates for financial services.

Recognizing Claude’s extensive capabilities, BGL harnessed these features to create an analytics AI agent equipped with:

• Code Execution: For data processing and visualization of datasets returned from queries.

• Automatic Context Management: Maintaining context over long-running sessions without hitting token limits.

• Sandboxed Execution: Ensuring secure and isolated execution environments.

• Modular Knowledge Architecture: Utilizing a CLAUDE.md file for project context along with embedded Agent Skills for domain-specific expertise.

Why Code Execution Matters for Data Analytics

Standard querying tools often funnel extensive datasets directly into the AI’s context window, quickly reaching limits. BGL’s approach allows the agent to write SQL queries to fetch data and process the results directly in its file system. This avoids context window constraints while enabling comprehensive analysis.

Modular Knowledge Architecture: Using CLAUDE.md and SKILL.md Files

BGL’s implementation employs a structured approach to manage domain knowledge effectively.

• CLAUDE.md provides global context, outlining project configurations and defining the execution paths for SQL queries.

• SKILL.md files encapsulate specialized knowledge for each product line, such as compliance requirements specific to the BGL CAS 360 and Simple Fund 360 offerings.

The dynamic interaction between these configuration types allows the agent to operate efficiently while addressing queries relevant to specific domains.

High-Level Solution Architecture

To deliver a secure and scalable text-to-SQL experience, BGL hosts Claude Agent SDK on Amazon Bedrock AgentCore while maintaining its existing big data solution.

Workflow Overview

1. User Request: A business user submits a query via Slack.
2. Schema Discovery: The agent identifies relevant tables and trails SQL queries.
3. SQL Security Validation: A protective layer allows only SELECT queries, blocking any destructive commands.
4. Query Execution: Athena executes the query and results are stored in Amazon S3.
5. Result Download: The agent retrieves the CSV file to bypass context window limitations.
6. Analysis and Visualization: The agent processes the CSV and generates corresponding outputs.
7. Response Delivery: Insights and visualizations are shared back with the user in Slack.

Why Amazon Bedrock AgentCore?

Building an AI agent capable of executing arbitrary Python code involves significant infrastructure considerations. Amazon Bedrock AgentCore offers:

• Stateful Execution Sessions: Maintaining context for up to 8 hours allows for ongoing conversations with users.
• Framework Flexibility: Support for various agent architectures with minimal code.
• Enhanced Security Practices: Ensuring compliance-aligned operations at scale through robust identity and access management.

“AWS is investing in a forward-looking ecosystem that will facilitate seamless integration for everything we build now.”
— James Luo

Results and Impact

For BGL’s 200+ employees, this AI-driven approach signifies a transformational shift in how they access business intelligence. Product managers can now validate hypotheses immediately without delays, while compliance teams can readily identify risk trends. This democratization of data access converts data analysis from a bottleneck into a competitive advantage.

Conclusion and Key Takeaways

BGL’s journey illustrates how a robust data foundation, when coupled with agentic AI, can expand access to business intelligence. By deploying the Claude Agent SDK on Amazon Bedrock AgentCore, they created an AI agent that empowers employees to tap into valuable data insights.

Key Learnings:

• Invest in Data Infrastructure: A solid foundation allows the AI to focus on reliable logic rather than complex business rules.
• Organize Knowledge by Domain: Utilize Agent Skills to encapsulate specialized expertise while continuously iterating based on user feedback.
• Code Execution for Data Processing: Use the agent to write and execute code for handling large datasets effectively.
• Stateful Infrastructure: Utilize platforms like Amazon Bedrock to streamline persistent context management.

If you’re ready to build similar capabilities for your organization, explore the Claude Agent SDK and consider contacting your AWS account team for support in designing your architecture.

About the Authors

Dustin Liu is a solutions architect at AWS focused on financial services and insurance startups.

Melanie Li, PhD is a Senior Generative AI Specialist at AWS, helping customers leverage cutting-edge AI technologies.

Frank Tan is a Senior Solutions Architect with a strong interest in Applied AI and bridging technology with business outcomes.

James Luo is the Head of Data & AI at BGL, responsible for spearheading BGL’s data initiatives.

Dr. James Bland is a Technology Leader at AWS with extensive experience in driving AI transformations across sectors.

With these insights, we hope to inspire you to leverage AI-driven solutions for improved data analysis and business intelligence in your operations.