Revolutionizing Shopping with Just Walk Out Technology by Amazon: A Multi-Modal AI Approach
Revolutionizing Shopping with Just Walk Out Technology by Amazon
Since its launch in 2018, Just Walk Out technology by Amazon has completely transformed the shopping experience. Imagine entering a store, picking up the items you need, and simply walking out without having to wait in line to pay. This revolutionary checkout-free technology is now available in over 180 third-party locations worldwide, spanning various industries such as travel, sports, entertainment, and healthcare.
The latest generation of Just Walk Out technology is powered by a multi-modal foundation model (FM) that leverages a transformer-based architecture similar to that used in generative artificial intelligence (AI) applications. This advanced model enables retailers to automatically generate highly accurate shopping receipts using data from multiple inputs such as overhead video cameras, weight sensors on shelves, digital floor plans, and catalog images of products.
The Challenge: Complex Shopping Scenarios
One of the key challenges in developing the Just Walk Out system was ensuring accuracy in complex, long-tail shopping scenarios. Previous generations of the system utilized a modular architecture that segmented the shopper’s visit into discrete tasks. While this approach delivered accurate receipts, it required significant engineering efforts to address new, complex situations, limiting scalability.
The Solution: Just Walk Out Multi-Modal AI
To address these challenges, a new multi-modal FM was introduced specifically for retail environments. This enhanced model improves accuracy and generalization to new store formats, products, and customer behaviors. By incorporating continuous learning, the system adapts and learns from challenging scenarios to maintain high performance.
Key elements of the Just Walk Out multi-modal AI model include flexible data inputs, multi-modal AI tokens to represent shopper journeys, and the ability to continuously update receipts based on shopper interactions. Training the FM involved feeding vast amounts of data into the model and utilizing auxiliary tasks to enhance its performance.
Training the Just Walk Out FM
Effective training of the FM involved selecting challenging data sources, leveraging auto labeling for efficiency, pre-training the model on diverse tasks, and fine-tuning the model to optimize performance. The data flywheel methodology continuously improves the model by identifying and incorporating high-quality, challenging cases.
Conclusion
The introduction of multi-modal AI represents a significant advancement for Just Walk Out technology. This innovative approach simplifies and scales AI systems, moving away from traditional modular architectures. The new system sets a higher standard for accuracy and applicability across different store environments, ultimately enhancing the shopping experience for customers worldwide.
For more information on Just Walk Out technology and AWS AI services, visit the official Amazon announcements and product pages. The future of shopping is here, powered by cutting-edge AI innovation.
About the Authors
Tian Lan is a Principal Scientist at AWS, leading research efforts for Just Walk Out technology.
Chris Broaddus is a Senior Manager at AWS, overseeing research projects related to Just Walk Out technology and deep learning.