Samsung Electronics has recently unveiled the LPDDR5X-PIM architecture and its detailed performance specifications, targeting on-device AI applications such as smartphones and AI PCs. This technology enables certain computations to be processed directly within the memory, eliminating the need for repeated data transfers to the processor. The LPDDR5X-PIM delivers a PIM bandwidth of up to 614 GB/s with a capacity of 16 GB. By comparison, conventional DRAM offers a bandwidth of only 76.8 GB/s, meaning the PIM technology achieves an eightfold increase in bandwidth.Samsung conducted tests using Meta's 8-billion-parameter model "Llama 3.1," with the following results:
- Processing time: Reduced from 12.3 seconds on conventional LPDDR5X to 5.4 seconds, representing an approximately 2.3x speedup.
- Tokens per second (TPS): Increased from 27 to 81.3, a threefold improvement over existing products.
