Maybe it won't - I say "time will tell" because we really do not know how much LLMs will be demanded in 10 years. Nvidia's stock skyrocketed because they were incidentally prepared for an enormous increase in demand the moment it happened. Now that expectations are cooling down and Sam Altman is signalling that AGI is a long ways off, the math that justified designing NPU/TPU hardware in-house might not add up anymore. Even if you believe in the tech itself, the hype is cooling and the do-or-die moment is rapidly approaching.
My overall point is that I think Nvidia played smartly from the start. They could derive profit from any sufficiently large niche their competitors were too afraid to exploit, and general purpose GPU compute was the perfect investment. With AMD, Apple and the rest of the industry focusing on simpler GPUs, Nvidia was given an empty soapbox to market CUDA with. The big question is whether demand for CUDA can be supplanted with application-specific accelerators.
> The big question is whether demand for CUDA can be supplanted with application-specific accelerators.
At least for AI workloads, Google's XLA compiler and the JAX ML framework have reduced the need for something like CUDA.
There are two main ways to train ML models today:
1) Kernel-heavy approach: This is where frameworks like PyTorch are used, and developers write custom kernels (using Triton or CUDA) to speed up certain ops.
2) Compiler-heavy approach: This uses tools like XLA, which apply techniques like op fusion and compiler optimizations to automatically generate fast, low-level code.
NVIDIA's CUDA is a major strength in the first approach. But if the second approach gains more traction, NVIDIA’s advantage might not be as important.
And I think the second approach has a strong chance of succeeding, given that two massive companies—Google (TPUs) and Amazon (Trainium)—are heavily investing in it.
Not really, no. Over the past several years, JAX was used in only 3% of top publications. PyTorch in 60%. There's no trend to suggest that JAX has "reduced the need" for anything, except for Google itself. https://paperswithcode.com/trends
Feel free to suggest a more reliable proxy if you don't like this one. What this means is your chance to find a person who knows/has experience with JAX are very low.
It's weird to me that folks think NVDA is just sitting there, waiting for everyone to take their lunch. Yes, I'm totally sure NVDA is completely blind to competition and has chosen to sit on their cash rather than develop alternatives...</s>
My overall point is that I think Nvidia played smartly from the start. They could derive profit from any sufficiently large niche their competitors were too afraid to exploit, and general purpose GPU compute was the perfect investment. With AMD, Apple and the rest of the industry focusing on simpler GPUs, Nvidia was given an empty soapbox to market CUDA with. The big question is whether demand for CUDA can be supplanted with application-specific accelerators.