Revolutionizing Clinical Outcome Assessments: Enhancing Data Quality and Efficiency with AI at Clario

About Clario

Business Challenge

Solution

Solution Architecture

Benefits and Results

Lessons Learned and Best Practices

Next Steps and Conclusion

About the Authors

Revolutionizing Clinical Outcome Assessment Interviews with AI: A New Era for Clinical Trials

Clinical Outcome Assessment (COA) interviews play a crucial role in clinical trials, particularly in evaluating treatments for psychosis, anxiety, and mood disorders. These assessments not only gauge a treatment’s efficacy and safety but also often determine the overall success of a trial. Despite their importance, the traditional methods used to assess the quality of these outcomes can be complex and fraught with challenges.

In this blog post, we will explore how Clario has harnessed the power of AI, particularly through Amazon Bedrock and other AWS services, to enhance the analysis of COA interviews, thereby improving data quality and reliability while facilitating faster and more informed decision-making.

The Importance of Data Quality in COA Interviews

COA interviews require a high standard of data quality and reliability. Variability in interview evaluations, stemming from poor assessment techniques and inherent biases, can lead to unreliable results, ultimately increasing the risk of study failure. Traditional review methods, particularly labor-intensive audio and video recordings analysis, are cumbersome and often do not scale effectively across multilingual studies.

Recognizing these challenges, Clario sought to innovate their COA review methodology.

Business Challenges Addressed by Clario

Clario aimed to streamline their operations while enhancing data quality. Their key challenges included:

1. Standardization Across Global Operations: Ensuring uniformity in assessments across multilingual data and diverse geographical locations.

2. Managing Large Volumes of Data: Effectively handling extensive audio recordings while meeting strict regulatory and privacy standards.

3. Reducing Subjectivity: Delivering consistent and reliable behavioral health assessments, thereby minimizing site and rater bias.

4. Accelerated Decision-Making: Providing quicker insights to support timely, evidence-based decisions for clinical trial sponsors.

How Clario Harnessed AI for COA Assessments

To tackle these challenges, Clario embraced AWS’s artificial intelligence and machine learning capabilities. This move involved leveraging generative AI and features from Amazon Bedrock to create an AI-powered solution that automates COA interview analysis. The innovative components of their solution include:

• Speaker Diarization and Multi-Lingual Transcription: Automatically identifying and transcribing unique speakers in near real-time, even across multiple languages.

• Vector Databases and Semantic Search: Using advanced search techniques to evaluate interview quality and provide relevant insights.

• Automated Assessment Reviews: Maintaining regulatory compliance while expediting the review process without sacrificing quality.

AI-Powered Solution Architecture

Clario’s solution architecture incorporates several efficient steps:

1. Data Collection: COA interview recordings are securely uploaded to Amazon S3, ensuring data integrity and privacy through encryption.

2. Automated Analysis: The AI Orchestration Engine extracts audio, applies speaker diarization models via Amazon SageMaker, and transcribes audio using Amazon Bedrock.

3. Semantic Retrieval: Transcriptions are vectorized and stored in Amazon OpenSearch, enabling advanced querying capabilities to extract relevant segments of dialogues.

4. Quality Evaluation: Using a graph-based agent system on Amazon EKS, the solution follows a structured checklist to analyze interviews, generate findings, and compile quality ratings.

Benefits and Outcomes

The implementation of Clario’s AI-powered solution has yielded significant improvements:

• Operational Efficiency: A potential decrease in manual review efforts by over 90%.

• Quality Improvements: Achieving near-complete data coverage through automated reviews, enabling targeted interventions where necessary.

• Business Impact: Shortening the turnaround time from weeks to hours, enhancing data reliability for regulatory submissions, and reducing the risks of study failures.

Lessons Learned and Future Directions

Through its journey, Clario has gleaned insights that can guide similar AI implementations:

1. Responsible AI Development: Ensuring AI outputs are validated against source documents for accuracy before human review is vital in healthcare.

2. Continuous Model Evaluation: Regular performance assessments are necessary to maintain high-quality standards.

3. Scalable Architecture: Utilizing serverless, cloud-based solutions enhances security and compliance while prioritizing scalability.

Conclusion: A Transformative Leap Forward

Clario’s innovative use of AI to enhance COA interview assessments marks a significant leap forward in clinical trial data analysis. By harnessing the robust capabilities of Amazon Bedrock, Clario is setting new standards in clinical data quality and reliability.

As Clario continues to expand its AI-powered functionalities, it envisions revolutionizing other areas within neuroscience studies relying on clinical interviews. This transformation empowers stakeholders to make well-informed decisions, accelerating the development of life-changing therapies for patients in need.

About the Authors

Alex Boudreau
Director of AI at Clario, leading the development of the company’s advanced GenAI platform. With a background in deep learning and cloud engineering, he drives innovations in AI technology across diverse applications.

Cuong Lai
Technical Team Lead for Clario’s Generative AI team. Experienced in software engineering and cloud-native solutions, he is passionate about advancing generative AI technologies.

Praveen Haranahalli
Senior Solutions Architect at AWS, specializing in architecting secure and scalable cloud solutions. He leverages nearly two decades of IT experience to help organizations embrace AI/ML solutions.

Clario’s pioneering efforts offer a glimpse into the future of clinical trials, showcasing how technology can be harnessed to improve both operational efficiencies and patient outcomes.