Revolutionizing Vessel Schedule Planning with Machine Learning: A Partnership Between Hapag-Lloyd and AWS

Revolutionizing Shipping with Machine Learning: Hapag-Lloyd’s Predictive Vessel Assistant

Co-written with Thomas Voss and Bernhard Hersberger from Hapag-Lloyd.

In today’s global economy, timely and efficient shipping is crucial. Hapag-Lloyd, a leading shipping company, operates over 308 modern vessels, transporting 11.9 million TEUs (twenty-foot equivalent units) annually. With a workforce of 16,700 across 400 offices in 139 countries, Hapag-Lloyd plays a key role in connecting continents, businesses, and people. In this blog post, we delve into how Hapag-Lloyd has leveraged machine learning (ML) to enhance its scheduling and operational efficiency.

The Challenge of Schedule Reliability

For Hapag-Lloyd, maintaining accurate vessel schedules is paramount. Schedule reliability is defined as the percentage of vessels arriving within one calendar day of their estimated arrival time. Traditionally, the company relied on rule-based and statistical methods based on historical transit patterns, which fell short in accounting for real-time variables like port congestion and weather conditions.

When unpredictable events occurred—like the infamous blockage of the Suez Canal in March 2021—manual intervention became necessary, disrupting operations and customer service.

Building a Robust Machine Learning Solution

Recognizing the limitations of their existing approach, Hapag-Lloyd embarked on developing a machine learning solution to predict vessel arrival and departure times more accurately. Implementing this system involved several challenges:

1. Dynamic Shipping Conditions: The estimated time of arrival (ETA) model needed to adapt to changing variables such as weather conditions and unexpected port-related delays.

2. Data Integration at Scale: The solution required integration of extensive historical voyage data with real-time data sources, including port congestion and vessel tracking.

3. Robust MLOps Infrastructure: A strong MLOps framework was necessary for continuous model performance monitoring and rapid updates.

Solution Overview: Machine Learning in Action

To tackle these challenges, Hapag-Lloyd’s new ML solution processes various data sources:

• Internal Data: Stored in a data lake, this includes vessel schedules, port performance metrics, and real-time port congestion information.
• External Data: Automatic Identification System (AIS) data provides streaming updates on vessel positions.

The company developed a multi-step solution using specialized models for different legs of a vessel’s journey:

1. Ocean to Port (O2P) Model: Predicts the time needed for a vessel to reach the next port, factoring in distance, speed, and historical leg durations.

2. Port to Port (P2P) Model: Forecasts sailing times between ports, considering weather and seasonal trends.

3. Berth Time Model: Estimates the time a vessel will spend at port.

4. Combined Model: Integrates inputs from the first three models to compute expected deviations from the original schedule.

Utilizing Amazon SageMaker, Hapag-Lloyd implemented an efficient training pipeline that allows for systematic version control and rapid model updates.

Inference Solution Walkthrough

The inference mechanism uses a hybrid approach combining batch processing for nightly predictions with real-time API capabilities.

• Daily Batch Inference: Automated processes trigger the updating of the internal data, merging it with predictions from the ML models.

• Real-time API: This API handles intraday updates, allowing for immediate responses to schedule changes.

This architecture not only ensures high availability, with a 99.5% uptime, but also enhances response times to enhance user experience.

Outcomes and Achievements

Hapag-Lloyd’s ML-powered assistant significantly improves upon previous solutions. With an estimated mean absolute error (MAE) decrease of 12%, the new system boasts enhanced accuracy, climbing two positions in international schedule reliability rankings.

Key Benefits:

• Real-time Updates: Business experts can interact with the system seamlessly, achieving response times in the hundreds of milliseconds.

• Enhanced Reliability: The predictive capabilities allow for better logistics planning and decreased manual intervention.

Conclusion

Hapag-Lloyd’s journey into machine learning showcases the transformative power of data in logistics. By developing an ML solution to revolutionize vessel scheduling, Hapag-Lloyd not only improves operational efficiency but also enhances the reliability promised to its customers.

For further insights into architecting ML workloads at scale, we recommend checking out the AWS blog post on governing the ML lifecycle using Amazon SageMaker.

Acknowledgments

We would like to recognize the dedicated contributions of Michal Papaj and Piotr Zielinski from Hapag-Lloyd in advancing this project, particularly in the areas of data science and engineering.

About the Authors

Thomas Voss
A data scientist at Hapag-Lloyd, Thomas leverages his academic background and industry expertise to drive business innovation through data science.

Bernhard Hersberger
Leading the AI Hub team at Hapag-Lloyd, Bernhard integrates AI solutions across the company, ensuring comprehensive responsibility from issue identification to solution deployment.

Gabija Pasiunaite, Jean-Michel Lourier, Mousam Majhi
Each with unique expertise, these AWS professionals focus on delivering scalable ML solutions, maximizing our data capabilities through innovative architectures.

This collaborative piece reflects Hapag-Lloyd’s commitment to advancing the shipping industry through technology. Join us in exploring the future of logistics!