Nvidia shares fall 6 percent as IBM and Supermicro launch AI speed boost

Nvidia, the chip designer, saw its shares fall 6 percent in morning trading on Thursday, even as the company announced a major new collaboration with Super Micro Computer, the server maker, and IBM, the tech giant, to speed up artificial intelligence systems.

The three companies revealed a joint storage system designed to let artificial intelligence applications reuse previously processed information instead of calculating everything from scratch. According to a Thursday press release carried by Yahoo Finance, the technology combines Supermicro flash storage servers, IBM software, and Nvidia Dynamo inference frameworks and Spectrum-X networking.

Testing on a massive 120-billion-parameter artificial intelligence model showed the setup delivered up to 56 times faster Time-to-First-Token performance on prompts containing 130,000 tokens. This key metric measures how long an artificial intelligence system takes to generate its very first response after receiving a prompt from a user.

More in MarketsMassive tech sell-off wipes $2.5 trillion from the stock market

The new system relies on persistent key-value cache storage to hold information generated during earlier processing runs. This innovation helps artificial intelligence systems handle longer conversations and complex workloads without overloading graphics processing unit and central processing unit memory. The companies also tested the technology under heavy network traffic and simultaneous user requests, building the system to share cached data smoothly across multiple graphics processing unit clusters.

For your portfolio, a sharp drop in a market heavyweight like Nvidia can ripple quickly through index funds and technology holdings. Check your retirement accounts today to see how the broader chip sector pullback touches your investments.

Photo: Coolcaesar, CC BY-SA 4.0, via Wikimedia Commons