Tbh my experience is in the non AI uses, recently I was looking at Gaussian splatting tools and it seemed the majority of it was CUDA only. I’m also still bothered AMD for ages claimed my card (5700xt) would be getting rocm but just abandoned it.
But even in the amd stack things (like ck and aiter) consumer cards are not even second class citizens. They are a distance third at best.
If you just want to run vllm with the latest model, if you can get it running at all there are going to be paper cuts all along the way and even then the performance won't be close to what you could be getting out of the hardware.
When they say that AI will replace programmers, I think of this article and come to terms with my own job security.
Most of this stuff is just central knowledge of the language that you pick up over time. Certainly, AI can also pick this stuff up instantly, but will it always pick the most efficient path when generating code for you?
Probably not, until we get benchmarks into the hot path of our test suite. That is something someone should work on.
I've had my own dealings with this awful company. Including Wally.
Let's just say that none of this comes as any surprise.
Now, what people should be asking is how much Jensen knew. In May he said there was nothing going on. But the videos of the Chinese guy holding H1/200's ... never got to him?
Also interesting how they waited until just after GTC...
reply