Advancing ADHD Diagnosis: The Integration of Machine Learning and Mobile Technology by Qbtech
Revolutionizing ADHD Diagnosis: The Story Behind QbMobile
Co-written with Dr. Mikkel Hansen, Qbtech
The assessment and diagnosis of Attention Deficit Hyperactive Disorder (ADHD) have long depended on clinical observations and behavioral evaluations. These traditional methods, while valuable, can often be complex and time-consuming. Enter Qbtech, founded in 2002 in Stockholm, Sweden—this innovative company is transforming ADHD diagnosis by integrating objective measurements with clinical expertise. With the completion of over one million tests across 14 countries, Qbtech’s FDA-cleared and CE-marked products, QbTest (in-clinic) and QbCheck (remote), have become widely accepted tools for objective ADHD testing. Their latest initiative, QbMobile, utilizes smartphone technology to deliver clinical-grade ADHD assessments directly to patients.
The Challenge: Democratizing Access to Objective ADHD Assessment
ADHD affects millions worldwide, yet traditional diagnostic processes often involve lengthy wait times and multiple clinic visits. Although Qbtech has made strides with its existing in-clinic and remote solutions, opportunities to expand access via smartphone technology became evident. Qbtech faced the task of transforming raw camera feeds and motion sensor data from various smartphone hardware into clinically validated ADHD assessments, ensuring they provided objective diagnostic value akin to their established tools.
This effort required the processing of complex multimodal data streams, extracting meaningful features, and training models to maintain accuracy across diverse device variations while adhering to stringent healthcare regulations.
Building the AI Model: From Raw Data to Clinical Insights
Qbtech’s mobile ADHD assessment leverages machine learning techniques to analyze multiple data streams. The team selected Binary LightGBM as their primary algorithm to develop the ADHD assessment model.
The resulting model utilizes 24 input features derived from face tracking, head movements during tests, user interaction patterns, and demographic information. This scale captures the nuanced attention, hyperactivity, and impulsivity patterns typical of ADHD across various patient populations.
To facilitate this, Qbtech employed three robust frameworks:
- LightGBM: Primary algorithm for machine learning.
- Scikit-learn (sklearn): Tool library for data processing and model development.
- SHAP (SHapley Additive exPlanations): Methodology to assess feature importance.
By utilizing a dataset of approximately 2,000 samples, each encompassing about 50 MB of data, the team ensured diverse representation across intersectional groups through careful stratification.
Training Performance and Evaluation
Although model training requires just one minute of computation time, the real challenge lay in transforming raw samples into structured features. Qbtech benefited significantly from Amazon SageMaker AI, which accelerated this process by reducing feature extraction time.
The team adopted rigorous evaluation metrics—sensitivity (85.7%), specificity (74.9%), and PR-AUC (73.2%)—to ensure clinical validity, employing nested cross-validation for hyperparameter tuning. Given the absence of an absolute ground truth in ADHD diagnostics, the solution offers clinicians consistent, objective data to bolster their diagnostic decisions.
Scaling Feature Engineering with Amazon SageMaker AI
A notable improvement in Qbtech’s development process was implementing cloud-based parallel processing. By adopting asynchronous processing, the team enabled multiple tests to run simultaneously, vastly improving efficiency.
The feature engineering pipeline converts raw data into time series for each source, generating various features. For example, face position data computes statistics like minimum, maximum, and mean movement within 30-second windows. This method allowed Qbtech to reduce processing time from two days to just 30 minutes.
from multiprocessing import Pool, cpu_count
def uuids_to_dataset(df_uuid):
"""Process all files into a dataset"""
with Pool(cpu_count()) as p:
r = list(p.imap(uuid_to_features, df_uuid["uuid"].to_list()))
df = pd.concat(r)
df = df.sort_values(by="uuid").reset_index(drop=True)
return dfThis parallel processing approach resulted in a staggering 96% reduction in computational time, allowing continuous iterations and improvements during model development.
Data Pipeline: From Smartphone to Clinical Decision
The data pipeline begins with raw sensor data in various formats. Qbtech employs AWS Glue jobs to extract and transform this heterogeneous data, ensuring quality and consistency across different devices.
Feature Extraction and Selection
The feature engineering process yielded approximately 200 features from the raw data, streamlined to just 24 for the final model. A systematic selection method involved analyzing class separation through histograms and leveraging SHAP analysis to confirm clinically meaningful interactions with diagnoses.
End-to-End Latency
For a clinical tool to be effective, results must be delivered swiftly. Qbtech’s pipeline achieves results in under a minute, supporting real-time clinical decisions and enhancing the patient experience.
Development Efficiency Gains
The most significant improvement was the feature engineering time reduction from two days to just 30 minutes, enabling Qbtech to complete 20 development iterations more efficiently.
Clinical Impact: Comparative Performance
Validation of QbMobile against Qbtech’s established products indicates that the smartphone-based assessment maintains high clinical standards. The shift to mobile assessments allows for comprehensive remote diagnostics, reducing logistical barriers for patients.
Deployment and Continuous Improvement
Qbtech packages trained models into Docker images, deploying them to AWS ECR via GitHub releases. They ensure consistent performance across devices through regular validation checks.
Security and Monitoring for Healthcare Compliance
Qbtech’s deployment incorporates stringent security and monitoring measures, encrypting data and ensuring patient privacy through anonymity. Continuous monitoring safeguards against security threats and helps meet healthcare requirements.
Looking Forward: Scaling for Global Impact
Qbtech anticipates QbMobile’s broad adoption, planning to utilize SageMaker’s elastic scaling capabilities to manage performance bottlenecks as usage grows. Regular updates will refine the model, integrating new features to bolster diagnostic accuracy.
Future Research Directions
Qbtech plans to expand data inputs and explore neural networks for enhanced assessment capabilities. With insights from over one million tests, they aim to continuously improve mobile assessments and explore broader applications beyond ADHD.
Conclusion
Qbtech’s implementation of QbMobile showcases significant strides toward accessible, objective ADHD assessments. By harnessing the capabilities of Amazon SageMaker AI and AWS Glue, they reduced feature engineering time by 96% while ensuring their model meets clinical standards.
This innovation not only enhances technical efficiency but also improves accessibility to ADHD assessments, reducing wait times and fostering better patient-provider interactions.
As global mental health challenges continue to escalate, Qbtech’s cloud-based AI approach illustrates how modern technology can broaden access to specialized healthcare services, paving the way for other organizations aiming to enhance patient outcomes through AI solutions.
For more insights into building healthcare AI solutions on AWS, check out the documentation for Amazon SageMaker AI and AWS Glue, or connect with AWS healthcare specialists to discuss your specific needs.
About the Authors
Antonio Martellotta is a Senior Solutions Architect at AWS, advising private equity firms on digital value creation through cloud and AI. He specializes in data strategy and analytics.
Dr. Mikkel Hansen is the Medical Director and CMO at Qbtech, focusing on integrating objective technologies in ADHD diagnosis and management. He collaborates with regulatory bodies to improve ADHD care worldwide.