memory
Rampant AI demand "supplemented by the start of a recovery of traditional compute and storage infrastructure demand"
For LLMs “GPU RAM is actually one of our most valuable commodities. It's frequently the bottleneck, not compute…”
Garnet’s server can talk to any existing Redis client owing to its use of the RESP protocol. Its storage layer is a customised fork of Microsoft’s own open source FASTER project
"Cache misses have this weird massive non-linear effect into how much work the GPUs are doing, because we suddenly need to start recomputing all this stuff."
"There’s a great deal of innovation taking place right now in NAND, with all major flash manufacturers demonstrating significant density increases this year"
Micron is cutting spending on memory chip fabrication by 50% in 2023 and slashing 10% of its workforce, or some 5,000 jobs, with the semiconductor firm’s CEO Sanjay Mehrotra warning the memory market is facing “the most severe imbalance between supply and demand in both DRAM and NAND
