Intel says it will miss its AI goals with Gaudi 3 — unbaked software leaves Intel’s $500 million AI goal unachievable as competitors rake in billions

Intel’s Gaudi 3 accelerator for AI offers tangible performance improvements compared to its predecessors. Given its advantages amid relatively low prices, Intel expected sales of these products to exceed half a billion dollars this year. However, as the new unit was formally launched in late September and the software was not precisely baked, Intel said this week that this shipment milestone would be missed. Still, some Gaudi 3 accelerators will be available at IBM Cloud.

“While the Gaudi3 benchmarks have been impressive, and we are pleased by our recent collaboration IBM to deploy Gaudi 3 as a service on IBM Cloud, the overall uptake of Gaudi has been slower than we anticipated, as adoption rates were impacted by the product transition from Gaudi 2 to Gaudi 3, and software ease of use,” said Pat Gelsinger, chief executive of Intel, at the company’s earnings call with analysts and investors. “As a result, we will not achieve our target of $500 million in revenue for Gaudi in 2024.”

Intel’s Gaudi 3 relies on two interconnected chiplets housing 64 tensor processing cores, designed with a 256×256 matrix structure that uses FP32 accumulators and eight matrix engines using 256-bit wide vector capabilities. It also includes 96MB of internal SRAM cache, offering data transfer rates up to 19.2 TB/s. 

Additionally, Gaudi 3 has 24 networking interfaces running at 200 GbE and 14 media processors capable of handling video and image formats like H.265, H.264, JPEG, and VP9 for visual data processing. The chip has 128GB of HBM2E memory across eight stacks, delivering a high bandwidth of 3.67 TB/s. Compared to its predecessor, Gaudi 3 marks a substantial leap forward as Gaudi 2 contained only 24 tensor cores, two matrix engines, and 96GB of HBM2E memory.

The new Gaudi 3 accelerator offers tangible performance advantages over Gaudi 2 and can even challenge Nvidia’s H100 (at least when this GPU does not use sparsity) in some cases. What is no less important is that Gaudi 3 is significantly cheaper than the H100. Earlier this year, Intel disclosed that a kit featuring eight Gaudi 3 chips on a baseboard would be priced at $125,000, roughly $15,625 per chip. In comparison, an Nvidia H100 card is currently priced at $30,678, around two times higher.

However, despite all the advantages that Gaudi 3 has, it looks like Intel’s software was not exactly ready for prime time, which slowed down hardware purchases. Now, Intel expects Gaudi 3 sales to ramp up in 2025.