It's great at one thing: memory. And that's interesting because memory is a commodity, but they still make bank on just being able to access it.

Memory is a commodity but access to high bandwidth memory is expensive whether it's HBM, or LPDDR/DDR connected to many memory channels.

Its not priced in a linear way wrt to bom cost.

That's because memory channels cost money. Memory controllers are more complex. Lastly, chips that can make use of high bandwidth VRAM are both of the above.

Memory chips are a commodity, that I agree. Though HBM is trending towards not being a commodity.

Memory controllers are die area. mm2 die space is linear bom cost.

DRAM+mm2 bom will have a different slope to just DRAM bom but still basically linear. Nonlinear pricing is pure market segmentation.

  Memory controllers are die area. mm2 die space is linear bom cost.

Ignoring the design and platform support that comes with higher bandwidth memory controllers.

And this is not also linear or fixed cost, why?

I don't think you know how industry pricing works. Wafers have a price, double mm2, double the price of chip in bom.

Yes. Bigger die size. More complex memory controller designs. More memory lanes. More software support for higher memory lanes. All adds up.

I'm explaining why despite memory being a commodity, high memory bandwidth VRAM cost is not cheap.

Memory controllers are copy paste. It's not more complex.

It's more expensive in a linear way wrt bom.

What I ended up doing is buy 3090's in bulk from ex mining rigs, stick new fans in them and run 14 of them connected to an old Supermicro chassis using very nice PCI express splitters. I figured out just what the sweet spot was in terms of PCI bandwidth and maxed it out on RAM (512G). The end result was pretty usable and allowed me to run some of my own benchmarks without sharing any data with the usual suspects. The bigger problem was power delivery, the box had four pretty beefy power supplies running on two different phases. It's retired now because we're much too busy with other stuff but that was a lot of fun and gave me a much better angle on all these new developments than I would have gotten otherwise. Total cost of the system was less than one RTX Pro 6000 and it had a lot more VRAM. The whole thing looked terrible though :)