Nvidia finally has some AI competition as Huawei shows off data center supercomputer that is better "on all metrics"
www.pcguide.com
Huawei is flexing its networking muscles with its newly unveiled CloudMatrix 384 supercomputer, and it looks like Nvidia could be in trouble.

With, I think, a massive grain of salt since this info is unverified and direct from the manufacturer…

Huawei’s official presentation claims their Cloudmatrix 385 supercomputer delivers 300 PFLOPS of computing power, 269 TB/s of network bandwidth, and 1,229 TB/s of total memory bandwidth. It also achieves 55 percent model fitting utilization (MFU) during training workloads and offers 2.8 Tbps of inter-card bandwidth, heavily emphasizing its strength in networking.

| Spec            | NVL72 (Nvidia) | CloudMatrix 384 (Huawei) | Better? (%) |
|-----------------|----------------|--------------------------|------------|
| Total compute   | 180 Pflops     | 300 Pflops               | 67%        |
| Total network bw| 130 TB/s       | 269 TB/s                 | 107%       |
| Total mem bw    | 576 TB/s       | 1,229 TB/s               | 113%       |
  • enumerator4829@sh.itjust.worksEnglish
    7·
    8 days ago

    Please note that the nominal FLOP/s from both Nvidia and Huawei are kinda bullshit. What precision we run at greatly affect that number. Nvidias marketing nowadays refer to fp4 tensor operations. Traditionally, FLOP/s are measured with fp64 matrix-matrix multiplication. That’s a lot more bits per FLOP.

    Also, that GPU-GPU bandwidth is kinda shit compared to Nvidias marketing numbers if I’m parsing correctly (NVLink is 18x 10GB/s links per GPU, big ’B’ in GB). I might read the numbers incorrectly, but anyway. How and if they manage multi-GPU cache coherency will be interesting to see. Nvidia and AMD both do (to varying degrees) have cache coherency in those settings. Developer experience matters…

    Now, the real interesting thing is power draw, density and price. Power draw and price obviously influence TCO. On 7nm, I guess the power bill won’t be very fun to read, but that’s just a guess. The density influences network options - are DAC-cables viable at all, or is it (more expensive) optical all the way?

    • taladar@sh.itjust.worksEnglish
      1·
      8 days ago

      I guess it is useful for the US’ love for avoiding the units everyone lese is using that they have a unit that represents a lifetime worth of flops, the Trump?