TapTechNews July 3rd news, according to the Taiwanese media Commercial Times, the upstream chip end of NVIDIA H200 has entered the mass production stage since the latter half of the second quarter and is expected to start large-scale delivery after the third quarter.
According to TapTechNews' previous report, OpenAI held a seminar in San Francisco in April, and NVIDIA's chief executive, Jensen Huang, attended in person and jointly announced the delivery of the first Nvidia DGX H200.
H200, as an iterative upgrade product of H100, is based on the Hopper architecture and adopts the HBM3e high-bandwidth memory technology for the first time, achieving a faster data transfer speed and larger memory capacity, showing significant advantages for large language model applications.
According to the data released by NVIDIA officially, when handling complex large language models such as Meta's Llama2, H200 has improved the generative AI output response speed by up to 45% compared to H100.
The supply chain points out that currently, the client orders to be shipped still mainly focus on H100 of the HGX architecture, and the proportion of H200 is limited. Due to the allocation system, the arrival time of the main H100 GPU of each terminal system partner is up to 20 weeks at the longest.
It is reported that B100 will be shipped in the first half of next year. NVIDIA CEO Jensen Huang previously said that from B100 onwards, the cooling technology of all future products will be changed from air cooling to liquid cooling. The TDP of NVIDIA H200 GPU is 700W, and conservatively estimated, the TDP of B100 is close to 1000 watts. Traditional air cooling cannot meet the cooling demand during the chip working process, and the cooling technology will be comprehensively innovated to liquid cooling.