$PENG is bringing its MemoryAI KV cache server to Japan that offloads the KV cache from GPU memory so inference clusters stop buying GPUs just to get more HBM.
Worth noting that the system runs on $NVDA Dynamo and can reportedly reduce memory costs by more than 85%. https://t.co/vyvVbsSZY6
