There is little doubt that Nvidia (NVDA)is currently the leading company in artificial intelligence (AI) computing infrastructure, after many years of capital investments.
But can Nvidia maintain its lead, or will its current dominance be challenged?
Lately, there is evidence that a few competitors are poking their heads up over their foxholes. To understand whether these companies are legitimate challengers to Nvidia’s crown, we first need to understand the basics of how an AI supercomputer works.
In simple terms, an AI supercomputer is a cluster of graphic processing units (GPUs) doing lots of calculations. However, there are also some central processing units (CPUs) needed to tell the computer what to do. Plus, some memory chips are required - both DRAM (short-term) and NAND (long-term).
The cost breakdown for an AI supercomputer is roughly 50% to 60% on GPUs, then 10% to 15% on CPUs and DRAM chips, and finally 5% to 10% on NANDs.
Infiniband and Networking
Importantly, all of these chips need to be connected to each other. This can be done either with Nvidia’s InfiniBand, or with Ethernet cables. This is known as “networking,” and it's where the final 10% to 15% of the hardware cost goes.
Nvidia acquired InfiniBand when it bought Mellanox for $6.9 billion in a deal finalized in April 2020. An early innovator in high-performance interconnect technology, Mellanox pioneered the InfiniBand interconnect technology, which along with its high-speed Ethernet products is now used in over half of the world’s fastest supercomputers, as well as in many leading hyperscale data centers.
This was a brilliant move by Nvidia, as it positioned itself to be able to offer all of the hardware needed by any customer looking to build an AI computer.
Data centers in the future will be set up as giant computing engines with tens of thousands of compute nodes, designed with their interconnects for optimal performance. The goal is to have connections with as much bandwidth as possible. That way, data can be transferred rapidly.
While the GPU portion of Nvidia’s business gets most of the headlines, the networking business has been a major part of its success.
During the latest earnings conference call, the company highlighted strong sales of its networking parts, which are increasingly important as companies build clusters of tens of thousands of chips that need to be connected. Nvidia said that it had $3.2 billion in networking revenue, primarily its InfiniBand products, which was over three times higher than sales in the year-earlier period.
It is in this specific area where Nvidia may come under threat.
Infiniband vs. Ethernet
It has been thought that InfiniBand was better for AI than Ethernet because it could transfer data directly between memory chips without having to go through the CPU, lowering its latency.
When it comes to AI workloads, bandwidth and latency are crucial, and InfiniBand is really low latency owing to an architecture that reduces packet loss. By comparison, packet loss is a given in Ethernet networks. However, InfiniBand tends to lag Ethernet in terms of raw bandwidth ceilings.
It's similar, in a way, to either traveling on an interstate highway (Ethernet) with higher speed limits, but the potential for collisions that can hold up traffic; or traveling on local roads (InfiniBand) which are slower, but clear of congestion.
One main reason InfiniBand has been so popular is because it has been sold alongside Nvidia’s GPUs, which have been in short supply. Customers were happy to buy InfiniBand if it meant they could also get their hands on the chips. But now the backlog on GPUs is falling, so there might be less incentive to use InfiniBand.
In its recent earnings call, Ethernet networking company Arista Networks(ANET)talked up its ability to outcompete InfiniBand in AI computing.
“We are progressing well in four major AI Ethernet clusters that we won versus InfiniBand recently. In all four cases, we are now migrating from trials to pilots, connecting thousands of GPUs a year,” explained Arista CEO Jayshree Ullal.
Arista also argued that its Ethernet cables are around 10% faster at job completion than InfiniBand.
Networking Growth
There is no doubt Arista is doing well.
In the first quarter, Arista’s revenue rose 16% year on year, while earnings per share increased by 44%. This growth should accelerate as AI infrastructure spending increases. Around 40% of Arista’s business comes from Microsoft (MSFT) and Meta (META), which are raising capex spending on AI.
This isn’t to say that Nvidia is about to be replaced any time soon as the leading AI infrastructure company. Most developers are using Nvidia’s CUDA program, which has become the de facto language for AI engineers, and a huge advantage for Nvidia.
Nvidia is still leading the AI race, but its operating margin of more than 50% has attracted fierce competition. In networking, that is coming from Arista as well as Broadcom (AVGO).
I believe the outlook is better for Arista, with expected revenue contributions from AI later this year, and that contribution jumping to $750 million in 2025. I expect this to be just the tip of the iceberg for the company.
Here is what Morningstar analyst William Kerwin wrote earlier this month: “We raise our fair value estimate for wide-moat Arista Networks… as we raise our forecast for strong and long-term growth for the firm arising from generative artificial intelligence networking. We believe Arista will significantly benefit from generative AI spending, resulting from its best-of-breed high-speed Ethernet switches. We expect Arista to drive and benefit from a transition toward Ethernet—and away from Nvidia’s Infiniband technology—in generative AI networks over the next five years. We forecast generative AI spending to augment growth in high-speed data center switching and for Arista to maintain a strong market share.”
I totally agree. ANET stock is a buy at its current price in the low $300s.
On the date of publication, Tony Daltorio did not have (either directly or indirectly) positions in any of the securities mentioned in this article. All information and data in this article is solely for informational purposes. For more information please view the Barchart Disclosure Policy here.