Broadcom takes a Tomahawk to Nvidia's AI networking empire

Broadcom began shipping its answer to Nvidia's upcoming Quantum-X and Spectrum-X switches on Tuesday: the Tomahawk 6. The chip doubles the bandwidth of its predecessor and comes in both standard and co-packaged optics flavors.

The launch comes as Broadcom looks to play a bigger role in AI networks — a space where Ethernet has historically trailed Nvidia's InfiniBand interconnect tech.

That's starting to change. Over the past year, we've seen a number of high-profile AI clusters pop up using Ethernet for their scale-out fabrics — the network used to connect GPU nodes to one another so that work can be distributed across them. xAI's 200,000 GPU Colossus supercomputer in Memphis, Tennessee, is just one example.

In fact, as of this year, Dell'Oro Group analyst Sameh Boujelbene estimates that Ethernet has surpassed InfiniBand's share, she told The Register.

Broadcom not only faces competition from Nvidia's Quantum-X InfiniBand switches, but also its Spectrum-X family of Ethernet kit, which our sibling site The Next Platform projects could rival Cisco's datacenter networking revenues within a year or so.

To meet this challenge, Broadcom is accelerating the rollout of its Tomahawk 6 (TH6) switch ASIC. Normally, it takes about two years from the time that Broadcom starts "shipping" chips to integrators to when the first appliances actually hit the market. This time around, Pete Del Vecchio, product line manager for Broadcom's Tomahawk line, tells El Reg that we can expect to see the first TH6-based switches as early as this summer with volume production ramping in the first half of next year.

Fatter SerDes, more ports, bigger flatter fabrics

In terms of speeds and feeds, TH6 looks quite similar to Nvidia's new photonic Spectrum ASICs due out next year.

Both chips offer 102.4Tbps of aggregate bandwidth and make use of 200Gbps PAM4 serializer/deserializers (SerDes). This means a single switch package will now be able to support up to 512 ports at 200Gbps, or, if you're prioritizing bandwidth over density, up to 64 ports at 1.6Tbps.

We say package because, unlike Tomahawk 5, which was monolithic, Broadcom's latest flagship has moved to a chiplet architecture, which breaks out the SerDes from the package processing and traffic management. And that means you'll be able to get this thing with 1,024 SerDes running at 100Gbps if you really want to.

According to Del Vecchio, Tomahawk 6's greater density means Broadcom can now support scale-out networks at 200Gbps with up to 128,000 GPUs using a two-layer topology, where three tiers would have been needed previously. And TH6 isn't limited to plain old RDMA over converged Ethernet (RoCE) either - we're told the chip will also support the emerging Ultra Ethernet standard.

This has the benefit of minimizing hops and therefore latency, simplifying congestion control, and halving the power required by the network, he added.

This is mainly a benefit of the chip's increased bandwidth and not unique to Broadcom. Other 102.4Tbps ASICs, like those from Nvidia, should also enjoy similar improvements in efficiency.

Ethernet fabrics are usually associated with the scale-out networks, but Broadcom is also positioning Tomahawk 6 as an alternative to Nvidia's NVLink for scale-up networks.

These networks usually offer higher aggregate bandwidth and are used to pool the compute and memory of multiple physical accelerators into one logical one. For example, Nvidia's HGX B200 platform used NVLink to stitch eight GPUs together, while its NVL72 racks, as their name suggests, utilize multiple NVLink switches to connect 72 GPUs.

Broadcom argues the same can be achieved using Ethernet on TH6, which can support scale-up fabrics up to 512 GPUs at 200Gbps. If you need more than that, you'll need to add more switches or start aggregating ports.

This is actually what Intel did with its Gaudi accelerators. Each Gaudi3 OAM module featured 24 200GbE ports. Three ports were dedicated to the scale-out network, while the remainder enabled each of the eight accelerators to communicate with each other at roughly 1TBps of bidirectional bandwidth.

That said, Ethernet-based scale-up networks may not catch on, as the emerging Ultra Accelerator Link standard is gaining momentum among chipmakers like Intel and AMD.

Broadcom prepares a co-packaged optical assault

In addition to conventional switches, Tomahawk 6 will also be available with co-packaged optics (CPO) on board.

The technology essentially takes the lasers, digital signal processors (DSPs), and retimers traditionally found in pluggable transceivers — for which you'd have one per port — and packages them alongside the switch ASIC.

Broadcom has been investing in CPO for years now, having first introduced the tech with its Humboldt family of switches back in 2021. However, it only began shipping CPO switches with its Tomahawk 5 Bailly platform last year.

Broadcom's latest CPO platform, teased back at Computex, enables up to 200Gbps of optical bandwidth per lane, which aligns perfectly with TH6.

For large-scale AI clusters, CPO offers a number of advantages. Most notably, it eliminates the need for expensive and power-hungry pluggable optics, at least at the switch. So, it's no surprise Broadcom and Nvidia are so keen on the tech: The less power spent on networking, the more you have for GPUs.

CPO also has the advantage of allowing for greater port densities. You can pack more fiber ports onto the front of a switch than you can QSFP cages.

Along with lower power and higher density, eliminating pluggables means fewer points of failure and lower overall latencies, since there are fewer interconnects between the fiber and ASIC.

With that said, CPO is still in its infancy, and there remain valid concerns over the increased complexity, issues with serviceability, and the overall blast radius of such a vertically integrated system. If a pluggable fails, it's relatively cheap to swap out. But, what happens if the photonic chiplet or one of the lasers fails? That could take down the entire switch fabric.

This is one of the reasons most CPO switch designs we've seen to date still use user-replaceable lasers that can be swapped out in the event of a failure. However, Manish Mehta, VP of marketing for Broadcom's optical systems group, says early evidence suggests many of these concerns are unfounded.

"The continuous wave laser source has been disaggregated, but even there it's not really failing," he told us.

• Dell has $14BN AI server backlog, warns projects are 'nonlinear'
• Nvidia opens up speedy NVLink interconnect to custom CPUs, ASICs
• Nvidia scores its first DOE win since 2022 with Doudna supercomputer
• The House of Zen joins the co-packaged optics race with Enosemi buy

How TH6 stacks up to Nvidia's Quantum-X and Spectrum-X

At GTC this spring, Nvidia revealed that its next-generation of Quantum-X InfiniBand and Spectrum-X Ethernet switches would ditch QSFP cages for CPO and gobs of MPO fiber ports.

Nvidia's Quantum-X photonic switches are expected to hit the market later this year and will offer 144 ports of 800Gbps InfiniBand connectivity, which you might have noticed is more than Broadcom's TH6.

However, it's important to remember that the Quantum 3450-LD is an appliance, while Tomahawk 6 is just an ASIC. Nvidia actually uses four Quantum-X CPO packages to achieve its 115Tbps of bandwidth. As The Next Platform, pointed out in its GTC coverage, there should probably be six ASICs in there for a full non-blocking configuration, but somehow Nvidia managed to make it work with four.

A far better comparison to TH6 is Nvidia's Spectrum SN6810, which uses a single ASIC, and, as we mentioned earlier, boasts many of the same speeds and feeds, including support for up to 128 ports at 800Gbps.

If you need more than that, Nvidia's SN6800 quadruples the bandwidth to 409.6Tbps, which works out to 512 MPO ports good for 800Gbps apiece. That's enough bandwidth for a single appliance to support a cluster of 64 DGX B200s.

Broadcom's Del Vecchio was keen to point out that, while Nvidia's Spectrum switches rely on its own ConnectX superNICs or BlueField data processing units (DPU) to achieve its InfiniBand-like latencies, TH6's Cognitive Routing functionality means it can achieve similar results using any modern NIC.

Nvidia says its Quantum-X photonic switches should hit the market in the second half of this year, while its Spectrum-X CPO kit will arrive in 2026. That's roughly in line with when Broadcom expects its partners to begin shipping its first CPO-enhanced TH6 Ethernet switches.

But if you need a CPO switch today, it should be noted that several network vendors, including Micas Networks, already offer switches based on Broadcom's 51.2Tbps Tomahawk 5-Bailly platform. ®