Exploring Non-Real-Time Generative AI Workflows: Buffered Asynchronous and Batch Processing Approaches

Introduction to Non-Real-Time Generative AI Workflows

Pattern 4: Buffered Asynchronous Request Response

REST APIs with Message Queuing

WebSocket APIs with Message Queuing

Pattern 5: Multimodal Parallel Fan-Out

Pattern 6: Non-Interactive Batch Processing

Conclusion: Architectural Patterns for Generative AI Applications

Exploring Asynchronous and Batch Processing in Generative AI Applications

In Part 1 of our series, we delved into three foundational patterns and best practices for developing real-time, interactive generative AI applications. However, the reality is that not every generative AI workflow necessitates instant responses. This post will explore two complementary approaches for non-real-time scenarios: buffered asynchronous processing for time-intensive individual requests and batch processing for scheduled or event-driven workflows.

Buffered Asynchronous Processing

Buffered asynchronous processing is invaluable when time-consuming tasks require precision. By leveraging an interactive delayed request-response cycle, buffered asynchronous integration allows for the creation of rich content while users patiently await results. Common applications of this approach include:

• Generating Multimedia Content: Creating video or music from text prompts.
• Scientific Analysis: Conducting in-depth medical or scientific analysis and visualization.
• Gaming and Virtual Worlds: Crafting immersive environments for gaming or metaverse experiences.
• Creative Graphics: Producing fashion and lifestyle visuals.

Buffering helps manage resources effectively, making this approach a strong choice for applications that must deliver high-quality outputs without immediate demands.

Batch Processing

The second approach, batch processing, tackles different challenges: handling extensive data sets based on predetermined schedules or event triggers. Useful scenarios include:

• Image Enhancement: Bulk processing of images for optimization.
• Report Generation: Automating the production of weekly or monthly reports.
• Customer Analysis: Analyzing customer reviews systematically.
• Content Creation: Streamlining social media content generation.

Non-interactive batch processing requires adherence to principles such as repeatability, scalability, parallelism, and dependency management. This approach is ideal when working with substantial data volumes, ensuring a streamlined workflow.

Pattern 4: Buffered Asynchronous Request-Response

Buffered asynchronous request-response patterns utilize event-driven architectures to enhance scalability and reliability in applications. Key benefits include:

• Performance: Improved throughput through concurrent processing.
• Scalability: Enhanced capacity via group processing.
• Reliability: Interactions among decoupled components, making systems more resilient.

Implementing this pattern typically involves using message queuing services, such as Amazon Simple Queue Service (Amazon SQS), to buffer requests and manage processing loads effectively. This approach shines when paired with WebSocket APIs, providing interactive updates and eliminating the need for client-side polling.

REST APIs with Message Queuing

To address scaling challenges with your LLM (Language Learning Model) endpoint, it’s advisable to employ an Amazon SQS queue to buffer messages. Here’s how it works:

1. The frontend sends messages to Amazon API Gateway REST endpoints.
2. These messages are passed to the SQS queue.
3. The API Gateway acknowledges receipt and returns a unique message ID to the frontend.
4. Middleware, possibly running on compute services like AWS Lambda or Amazon EC2, processes messages in batches, generating entries in Amazon DynamoDB.
5. Responses are stored back in DynamoDB, linked to the original message ID.

This system allows you to bypass limitations—such as the 29-second API Gateway request-response cycle—and enables seamless communication between components.

WebSocket APIs with Message Queuing

A variation of the previous pattern leverages WebSocket APIs, allowing middleware to send results back to the client immediately after they’re generated. This approach employs API Gateway to facilitate omni-channel communication, significantly enhancing user experience.

Pattern 5: Multimodal Parallel Fan-Out

For applications requiring interaction with multiple LLM models, data sources, or agents, the messaging fan-out pattern emerges as a robust solution. This approach:

• Distributes messages to various destinations in parallel.
• Minimizes overall generation time by breaking complex tasks into manageable sub-tasks.

Utilizing tools like Amazon EventBridge or Amazon SNS, you can implement rules-based message fan-out to orchestrate this parallel processing effectively.

Pattern 6: Non-Interactive Batch Processing

Non-interactive batch processing pipelines excel when handling large volumes of data efficiently. Often triggered by schedules or specific events, this pattern utilizes AWS Step Functions, AWS Glue, or similar compute services to create serverless data processing and inferencing pipelines. Key advantages include:

• Resource Optimization: Maximizing throughput and minimizing waste.
• Higher Automation Levels: Ensuring systematic processing through well-defined workflows.

Conclusion

This post has outlined six architectural patterns for constructing generative AI applications using AWS serverless services. By understanding and implementing these patterns, we can efficiently manage interactive real-time, asynchronous, or batch-oriented workloads while minimizing operational overhead.

As the generative AI domain evolves, expect to see emerging blueprints that further refine these patterns. Ultimately, deploying production-ready generative AI applications requires thoughtful consideration of architectural choices and the unique needs of your projects. Factors like response time, scalability, integration, reliability, and user experience are crucial when determining the most fitting patterns.

For further insights into serverless architectures, check out Serverless Land.

Stay tuned for the next parts of our series, where we delve deeper into the intricacies of building generative AI applications!