Nvidia: what’s so good about the tech firm’s new AI superchip? | Technology sector

The chipmaker Nvidia has extended its lead in artificial intelligence with the unveiling of a new “superchip”, a quantum computing service, and a new suite of tools to help develop the ultimate sci-fi dream: general purpose humanoid robotics. Here we look at what the company is doing and what it might mean.

What is Nvidia doing?

The main announcement of the company’s annual develop conference on Monday was the “Blackwell” series of AI chips, used to power the fantastically expensive datacentres that train frontier AI models such as the latest generations of GPT, Claude and Gemini.

One, the Blackwell B200, is a fairly straightforward upgrade over the company’s pre-existing H100 AI chip. Training a massive AI model, the size of GPT-4, would currently take about 8,000 H100 chips, and 15 megawatts of power, Nvidia said – enough to power about 30,000 typical British homes.

With the company’s new chips, the same training run would take just 2,000 B200s, and 4MW of power. That could lead to a reduction in electricity use by the AI industry, or it could lead to the same electricity being used to power much larger AI models in the near future.

What makes a chip ‘super’?

Alongside the B200, the company announced a second part of the Blackwell line – the GB200 “superchip”. It squeezes two B200 chips on a single board alongside the company’s Grace CPU, to build a system which, Nvidia says, offers “30x the performance” for the server farms that run, rather than train, chatbots such as Claude or ChatGPT. That system also promises to reduce energy consumption by up to 25 times, the company said.

Putting everything on the same board improves the efficiency by reducing the amount of time the chips spend communicating with each other, allowing them to devote more of their processing time to crunching the numbers that make chatbots sing – or, talk, at least.

What if I want bigger?

Nvidia, which has a market value of more than $2tn (£1.6tn), would be very happy to provide. Take the company’s GB200 NVL72: a single server rack with 72 B200 chips set up, connected by nearly two miles of cabling. That not enough? Why not look at the DGX Superpod, which combines eight of those racks into one, shipping-container-sized AI datacentre in a box. Pricing was not disclosed at the event, but it’s safe to say that if you have to ask, you can’t afford it. Even the last generation of chips came in at a hefty $100,000 or so a piece.

skip past newsletter promotion

after newsletter promotion

What about my robots?

Project GR00T – apparently named after, though not explicitly linked to, Marvel’s arboriform alien – is a new foundation model from Nvidia developed for controlling humanoid robots. A foundation model, such as GPT-4 for text or StableDiffusion for image generation, is the underlying AI model on which specific use cases can be built. They are the most expensive part of the whole sector to create, but are the engines of all further innovation, since they can be “fine-tuned” to specific use cases down the line.

Nvidia’s foundation model for robots will help them “understand natural language and emulate movements by observing human actions – quickly learning coordination, dexterity, and other skills in order to navigate, adapt, and interact with the real world”.

GR00T pairs with another piece of Nvidia tech (and another Marvel reference) in Jetson Thor, a system-on-a-chip designed specifically to be the brains of a robot. The ultimate goal is an autonomous machine that can be instructed using normal human speech to carry out general tasks, including ones it hasn’t been specifically trained for.

Quantum?

One of the few buzzy sectors that Nvidia doesn’t have its fingers in is quantum cloud computing. The technology, which remains at the cutting edge of research, has already been incorporated into offerings from Microsoft and Amazon, and now Nvidia’s getting into the game.

But Nvidia’s cloud will not actually be connected to a quantum computer. Instead, the offering is a service that uses its AI chips to simulate a quantum computer, ideally allowing researchers to test their ideas without going to the expense of accessing the (rare, expensive) real thing. But down the line, Nvidia will provide access to third party quantum computers through the platform, it said.