SK hynix Presents 'New Spectrum' for AI Infrastructure

Lim Eui-cheol, Vice President of SK hynix in charge of Solution AT, recently delivered a presentation at the "AI Infra Summit" held in the United States. He stated that as AI workloads become increasingly diverse, the existing GPU-HBM architecture alone is no longer sufficient to meet new demands. SK hynix is preparing new solutions combining software and hardware, such as PIM (Processing-in-Memory), HBF(High Bandwidth Flash), and SALT-KV (Semantic-Aware Lifecycle Tiering for KV Cache), to address the needs of these evolving workloads.

PIM is a technology that embeds computational functions within memory to reduce data movement between the processor and memory.  It is regarded as a next-generation solution that eases the bottleneck in data transfer (the Memory Wall) while improving both computational performance and power efficiency. At its booth, SK hynix exhibited the "AiM" chip (which adds computing capability to memory), the "AiMX" accelerator card equipped with multiple chips, and a server equipped with AiMX cards.

As the AI paradigm shifts from training to inference, there is a growing need for a new memory tier between HBM, which offers high bandwidth, and SSDs, which can store large volumes of data. HBF is a next-generation NAND solution that applies TSV technology like HBM, simultaneously delivering the large capacity of conventional storage and high bandwidth. 

SALT-KV divides an LLM's KV cache into context-based segments and comprehensively assesses their reuse value and storage cost. Based on this, it places data in the most suitable storage tier among HBM, DRAM, and SSD, raising utilization efficiency within limited memory resources. The exhibition also featured a demo in which visitors used an LLM model with the SALT-KV solution to create and run a game.

SK hynix stated that it can fully meet the increasingly diverse needs of customers in step with the pace of AI advancement.