Analysis of Nvidia
(ticker NVDA)
Latest news
Courtesy of GlobeNewswire, October 28, 2024, 19:00
SANTA CLARA, California, October 28, 2024 (GLOBE NEWSWIRE)
- Today, NVIDIA announced that the xAI Colossus supercomputer cluster, consisting of 100,000 NVIDIA Hopper Tensor Core GPUs in Memphis, Tennessee, has achieved this massive scale through the use of NVIDIA Spectrum-X™ Ethernet. This networking platform is designed to deliver superior performance for multi-tenant, hyperscale AI factories using standard Ethernet for remote direct memory access (RDMA) networking.
Colossus, the world’s largest AI supercomputer, is used to train xAI’s Grok family of large language models, and the chatbots are offered as a feature for X Premium subscribers. xAI is in the process of doubling the size of Colossus to 200,000 NVIDIA Hopper GPUs.
The supporting infrastructure and state-of-the-art supercomputer were built by xAI and NVIDIA in just 122 days, compared to the typical timeframe for systems of this size, which can take anywhere from many months to years. It took 19 days from the moment the first rack was installed to the start of training.
Network performance
While training the extremely large Grok model, Colossus achieves unprecedented network performance. Across all three levels of the network fabric, the system showed no increase in application latency or packet loss due to flow collisions. It maintains 95% throughput, made possible by Spectrum-X congestion control. This level of performance cannot be achieved at scale using standard Ethernet, which generates thousands of flow collisions while delivering only 60% of the data throughput.
“AI is becoming mission-critical and demands improvements in performance, security, scalability, and cost-effectiveness,” said Gilad Shainer, senior vice president of networking at NVIDIA. “The NVIDIA Spectrum-X Ethernet networking platform is designed to provide innovators like xAI with faster processing, analysis, and execution of AI workloads, which in turn accelerates the development, deployment, and time-to-market of AI solutions.”
“Colossus is the world’s most powerful training system,” Elon Musk said on X. “Great work by the xAI team, NVIDIA, and our many partners and suppliers.”
“xAI has built the world’s largest and most powerful supercomputer,” said an xAI spokesperson. “NVIDIA Hopper GPUs and Spectrum-X allow us to push the boundaries of AI model training at massive scale, creating a super-accelerated and optimized AI factory based on the Ethernet standard.”
The Spectrum-X platform
At the heart of the Spectrum-X platform is the Spectrum SN5600 Ethernet switch, which supports port speeds of up to 800 Gbps and is based on the Spectrum-4 switch ASIC. xAI decided to combine the Spectrum-X SN5600 switch with NVIDIA BlueField-3® SuperNIC cards to achieve unprecedented performance.
Spectrum-X Ethernet networking for AI provides advanced features that deliver highly efficient and scalable bandwidth with low latency and short tail latency, previously exclusive to InfiniBand. These features include adaptive routing with NVIDIA Direct Data Placement technology, congestion control, as well as enhanced AI fabric visibility and performance isolation - all of which are key requirements for multi-tenant generative AI clouds and large enterprise environments.
Good luck with your investments
The AVI Team
Discussion on Telegram
No comments yet. Be the first!
Sign in with Telegram. Your comment will appear under the post in the channel.