Transforming AWS Support Data into Actionable Insights: Enhancing Analytics with RAG and Custom Plugins

The Power of RAG in Operational Analytics

Addressing the Limitations of Traditional Analytics

Enhancing Insights with Structured Data Querying

Implementing Agentic AI for Advanced Query Responses

Key Implementation Steps for Amazon Q Business

Conclusion: The Future of Operational Data Analytics on AWS

Transforming AWS Support Data into Actionable Insights with RAG and Custom Plugins

As organizations strive to harness the value embedded in their AWS Support data, operational teams are increasingly seeking methods to convert raw support cases and health events into actionable insights. While traditional analytics tools may offer basic reporting capabilities, the need for sophisticated solutions that can process natural language queries about operational data has become paramount. Enter Retrieval-Augmented Generation (RAG) architecture, a powerhouse for optimizing large language model outputs by referencing authoritative knowledge bases. By utilizing semantic search and information retrieval capabilities, RAG enhances response accuracy and enriches the insight extraction process.

In our previous post, “Derive Meaningful and Actionable Operational Insights from AWS Using Amazon Q Business,” we explored a RAG solution implemented with Amazon Q Business. However, as effective as this approach is for semantic search, it can struggle with precise numerical analyses and aggregations. This post addresses those limitations by demonstrating how custom plugins can enhance Amazon Q to improve support analytics and incident response. By blending RAG’s strengths with structured data querying, we can achieve higher accuracy when analyzing AWS Support data. Instructions for deploying this solution can be found in our AWS Samples GitHub repository.

Understanding RAG’s Design for Analytics

RAG architecture shines in finding relevant information through context-based matching, enabling natural language interactions with external data. However, the fundamental architecture of vector similarity search prioritizes semantic relevance over the analytical aggregation capabilities, which can lead to inaccuracies. Let’s take a look at this limitation with a basic query:

Query: Give me the total count of support cases in Feb 2025

RAG-only Response:
Based on the provided JSON files, there were 190 AWS support cases documented in February 2025, calculated from the following case counts: …

The response indicates a calculated total of 190 cases, but this number is derived through potentially “hallucinated” aggregations.

Enhanced Structured Query Response:
Based on the query results, there were a total of 958 support cases in February 2025.

This stark contrast between the two outcomes underscores the limitation of RAG in performing numerical calculations. The gap only widens with more complex queries requiring categorical analysis.

Query: Give me OpenSearch support cases, distribution based on severity, opened in Jan 2025?

RAG-only Response:
Based on the support cases data from January 2025, here is the severity distribution for OpenSearch cases…

This response is vague and shows a poor categorical breakdown.

Structured Query Response:
Based on the query results for OpenSearch support cases opened in January 2025, here’s the distribution by severity: …

The precision offered by structured data querying clearly enhances our understanding of the support data.

Enhancing Q Support-Insights with Agentic AI

Building upon the Q Support-Insights (QSI) solution introduced earlier, we will look at ways to bolster analytical capabilities through the creation of custom plugins. This enhancement allows us to maintain the foundational elements of QSI while adding precise analytical processing through structured metadata querying.

QSI Overview

The Amazon Q Support Insights (QSI) solution comprises two main components:

1. Data Collection Pipeline

   • Support Collector Module using AWS Lambda functions
   • Automated real-time and historical synchronization of AWS Support data.
   • Stored in JSON format in a centralized Amazon S3 bucket.

2. Amazon Q Business Application Environment

   • Seamless integration of Amazon S3 for data sources.
   • User-friendly web experience and authentication through AWS IAM Identity Center.

Enabling Query Aggregation with Custom Plugins

Custom plugins can extend Amazon Q Business to merge semantic search with precise analytical capabilities. Here’s how we augment the QSI’s natural language processing with structured query capabilities:

• Analytical requests get converted into precise Amazon Athena SQL using Amazon Bedrock’s large language model (LLM).
• Queries run against structured metadata tables to provide exact numerical results alongside semantic search outcomes.

Metadata Processing

To enable precise querying capabilities, we’ve established a robust framework for structuring and indexing support data. This framework focuses on extracting critical structured metadata, such as dates, categories, and severity levels, making precise filtering and aggregation feasible.

Challenges Overcome by Enhanced Capabilities

Let’s take a glance at how the augmented QSI can tackle complex analytical queries that RAG alone cannot handle effectively.

Multi-dimensional Query Example:

Find accounts with frequent high-priority support cases since January 2024, excluding limit increase requests…

RAG-only Response:
Based on the provided data source, I cannot provide a complete analysis…

Conversely, with structured querying, we can generate comprehensive insights about accounts and services while providing actionable data for operational reviews.

Benefits and Impact

Here’s what the plugin architecture achieves:

• Precise Analytics: Exact counts and distributions replace approximate semantic matching.
• Contextual Analysis: Retains analytical context across conversation threads.
• Architectural Understanding: Enables better correlation of related issues through structured analysis.

Deploying the Amazon Q Business Application

Follow these simple steps for deployment, with detailed instructions available in the Amazon Q Business application creation module. Before you start, ensure you have the required prerequisites outlined in our AWS documentation.

Prerequisites

• Use AWS CloudShell or a local machine with AWS CLI.
• Set up two S3 buckets: one for AWS Support case data and the other for Lambda resource storage.
• An IAM Identity Center instance configured for user access.

Deployment Steps

You can easily deploy the solution using the provided deployment script:

# Clone the repository
git clone https://github.com/aws-samples/support-insights-with-amazon-q.git
cd q_application
chmod +x deploy_q_stacks.sh
./deploy_q_stacks.sh

Conclusion

By merging RAG’s semantic understanding with precise analytical capabilities via plugins, we’ve transformed Amazon Q Business into a formidable operational analytics platform. The examples presented illustrate how organizations can derive actionable insights from AWS Support data to foster better decision-making and proactive issue resolution. These methodologies extend beyond support data analytics and can facilitate improvements in any domain integrating structured and unstructured data sources.

Learn More

For further exploration, check the Amazon Q documentation for insights on building custom plugins and visit relevant resources for hands-on guidance.

Feel free to explore additional questions or provide feedback through AWS re:Post or AWS Support!

About the Authors

Chitresh Saxena is a Senior AI/ML specialist dedicated to helping organizations leverage AI/ML solutions on AWS.
Kevin Morgan serves as a Senior Enterprise Support Manager, committed to enhancing customers’ cloud adoption through technical guidance and support.

With this powerful blend of RAG and structured data querying, harnessing your AWS Support data has never been easier or more effective!