Revolutionizing Networking for Generative AI: Introducing UltraCluster 2.0
Delivering low-latency, large-scale networking is key to enabling the training and running of complex generative AI models efficiently. With the demand for AI-powered applications on the rise, it’s crucial for tech companies to invest in networks that can handle the scale and speed required for these workloads.
At AWS, we understand the importance of reducing network latency and enhancing performance for our customers. That’s why we have taken a unique approach by building our own network devices and operating systems for every layer of the stack. This level of control not only allows us to improve security, reliability, and performance but also enables us to innovate at a faster pace.
One of our key innovations in this space was the introduction of the Elastic Fabric Adapter (EFA) in 2019. This custom-built network interface provides operating system bypass capabilities to Amazon EC2 instances, allowing customers to run applications with high levels of inter-node communication at scale. EFA utilizes the Scalable Reliable Datagram (SRD) protocol, designed specifically by AWS, to deliver high performance and low latency.
Most recently, we launched the UltraCluster 2.0 network, designed specifically for generative AI workloads. This upgraded network supports over 20,000 GPUs with a 25% reduction in latency compared to its predecessor. Built in just seven months, UltraCluster 2.0 delivers tens of petabits per second of throughput with a round-trip time of less than 10 microseconds. This new network has resulted in at least a 15% reduction in model training time, showcasing its efficiency and effectiveness for large-scale AI projects.
By investing in custom-built network devices and software, we have been able to develop innovative solutions that meet the demands of AI workloads. Our commitment to delivering low-latency, large-scale networking solutions continues to drive advancements in the field of artificial intelligence and propel our customers towards groundbreaking discoveries and innovations.