Microsoft releases Azure ND H100 v5 virtual machine with 8 NVIDIA H100 GPUs

Microsoft Azure AI infrastructure received an upgrade on November 21, launching a new ND H100 v5 virtual machine series. This virtual machine is equipped with NVIDIA H100 Tensor Core graphics processing unit (GPU) and low-latency network

According to reports, Microsoft has newly released the Azure ND H100 v5 virtual machine (VM) series, which is now available in the Eastern United States and the United States Available in the South Central Azure Region.

This series of products uses the latest NVIDIA H100 Tensor Core GPU and NVIDIA Quantum-2 InfiniBand network to handle increasingly complex artificial intelligence workloads. As part of Microsoft's continued deep investment in artificial intelligence, Microsoft is leveraging an AI-optimized 4K GPU cluster with plans to scale it to hundreds of thousands of new GPUs over the next year

We are on this site The features of the ND H100 v5 VM virtual machine series are compiled above:

• AI super computing GPU: Equipped with eight NVIDIA H100 Tensor Core GPUs. This series can significantly speed up AI models compared to previous generation virtual machines.
• Next-generation computer processing unit (CPU): CPU performance is critical for AI training and inference, and Microsoft has chosen the fourth-generation Intel Xeon Scalable processor.
• Low-latency network: Using NVIDIA Quantum-2 ConnectX-7 InfiniBand, each GPU has 400Gb/s and each VM has 3.2 Tb/s of cross-node bandwidth.
• Optimized host-to-GPU performance: Delivers 64GB/s of bandwidth per GPU via PCIe Gen5.
• Large-scale memory and memory bandwidth: DDR5 memory is the core of this series of virtual machines, providing higher data transfer speed and efficiency.

Microsoft said that when this series of virtual machines uses the new 8-bit FP8 floating-point data type to perform matrix multiplication operations, the speed is increased by six times, which is faster than the previous generation FP16. ND H100 v5 VM helps achieve up to 2x faster inference for large language models such as the BLOOM 175B end-to-end model.

The above is the detailed content of Microsoft releases Azure ND H100 v5 virtual machine with 8 NVIDIA H100 GPUs. For more information, please follow other related articles on the PHP Chinese website!