I’m starting to wonder where the most is for any of these models.
Sure they are not cheap to train. But if open weight models continue to be trained and continue to become available on cheaper hardware, how do dedicated AI companies protect their margins?
OpenAI found the answer: artificially curtail the supply of DRAM wafers (by buying 40% of world's supply without necessarily having a feasible plan to make use of all of them], to prevent consumers from getting access to gpus with more and more memory, which could allow them to get dangerously close to the state of art while running local AI
Rather than an increase in VRAM of consumer gpus, we are seeing a decrease, which is pretty optimal for OpenAI
Sure they are not cheap to train. But if open weight models continue to be trained and continue to become available on cheaper hardware, how do dedicated AI companies protect their margins?