Micron First to Sample 256GB SOCAMM2 Memory

Micron recently became the first in the industry to sample a single 256GB SOCAMM2 memory module, currently the world's highest-capacity low-power DRAM (LPDRAM) module for servers, designed specifically for AI data centers and high-performance computing applications.

This product utilizes the world's first single-chip 32Gb LPDDR5x DRAM Die, increasing capacity by one-third compared to the previous generation 192GB module. With this module, an 8-channel server CPU can achieve a single-chip memory capacity of up to 2TB, capable of handling larger context windows and complex inference workloads, meeting the stringent memory capacity and bandwidth requirements of large AI models.

In terms of power consumption and space footprint, SOCAMM2 exhibits significant advantages: power consumption is only one-third that of traditional RDIMMs, and the physical footprint is also only one-third, effectively increasing rack density and reducing total cost of ownership. The module measures 14x90mm, supports liquid-cooled server architectures, and offers both maintainability and scalability.

In terms of performance, Micron's internal testing shows that when used for KV cache offloading in a unified memory architecture, the 256GB SOCAMM2 can improve the first token generation time for long-context, large-language model inference by more than 2.3 times (tested using an Llama3 70B model, context length of 500,000, and 16 concurrent users). In discrete CPU applications, LPDRAM's performance per watt under high-performance computing workloads is more than three times that of mainstream memory modules.