Hopper58 schreef op 26 november 2024 08:44:
Interessant artikel over de shift in AI van Training naar Inference:
'At some point, AI’s emphasis will shift from training to inference, the process of generating answers or results from the models. Many in the industry now believe that dedicating more computing power to inference can provide similar benefits to training.
“We are seeing the emergence of a new scaling law…with inference-time compute,” Nadella said at Microsoft’s Ignite conference.
The inference focus has big implications for Nvidia. While training is uniquely suited to the company’s GPUs, inference might be more readily handled by AI processors from Nvidia peers like Advanced Micro Devices
AMD and Intel, by custom chips from Amazon, or by a range of chip start-ups.
Nvidia is hardly unaware of the threat. It emphasized in its recent earnings report that inference makes up around 40% of its data-center revenue and is growing fast. It says that its NVL72 server system delivers a fourfold improvement in AI model training but up to a 30 times improvement in inference compared with previous systems. The new NVL72 stitches together 36 GB200 Superchips, with each GB200 connecting two Blackwell GPUs to an Nvidia Grace CPU. '
www.marketwatch.com/articles/ai-scali...