The global flash and memory sector saw intense activity over the weekend, featuring a Micron-Apple dispute, Anthropic’s chip sourcing talks with SK Hynix, a new Nvidia-SK Hynix partnership, SK Hynix’s 3D DRAM progress, and Samsung’s CXL memory pooling innovation.
SK Hynix dominated recent industry headlines. Citrini analyst Jukan confirmed the firm is developing 3D stacked DRAM, backed by strategic hiring of 3D-stacked DRAM-on-logic design engineers via its San Jose U.S. subsidiary. The recruitment targets talent capable of co-designing 3D-stacked DRAM logic dies with multiple U.S.-based customers.
The new architecture layers 3D DRAM stacks atop system semiconductor processors. Integrating DRAM and logic on a single die delivers faster inter-chip connectivity and lower power consumption than traditional package-on-package (PoP) bonding of separate memory and logic packages.
Jukan notes the technology targets AI workload devices requiring high memory bandwidth and low power draw, most notably smartphone mobile application processors — positioning Apple as a key potential customer for SK Hynix.
SK Group has expanded its strategic partnership with Nvidia via formal letters of intent, forming a collaboration valued at over $500 billion. SK Telecom will build a 2-gigawatt Nvidia Vera Rubin DSX AI Factory as a new cloud provider to meet global compute demand. The Vera Rubin chips will adopt SK Hynix’s HBM4 memory, with the first AI Factory scheduled to launch in 2027.
The partnership aims to accelerate large-scale AI infrastructure development, covering sovereign, physical, agentic and enterprise AI services, and address surging AI demand across South Korea and the broader Asia-Pacific region. SK Hynix will also launch a long-term AI memory partnership with Nvidia, securing stable next-generation AI memory supply for Nvidia while scaling SK Hynix’s production capacity. Both sides will co-develop and optimize HBM and other AI memory solutions for LLM training, agentic and physical AI scenarios.
SK Group Chairman Chey Tae-won stated the collaboration will leverage SK Hynix’s AI memory and SK Telecom’s infrastructure strengths to build a world-class AI factory, transforming South Korea from an AI adopter into a global AI innovation hub.
Former Intel CEO Pat Gelsinger openly criticized HBM during a Korean industry fireside chat, calling it an imperfect memory solution. He pointed out HBM’s key flaws: thermal stacking issues, connectivity limits constrained by GPU edge shorelines, poor power efficiency, and bit inefficiency — sacrificing roughly four memory bits per effective operational bit. Gelsinger nonetheless acknowledged HBM is the best viable solution available today.
SK Hynix SVP Hoshik Kim partially endorsed Gelsinger’s view, stating HBM cannot fundamentally resolve AI’s inherent memory wall bottleneck. He clarified HBM serves only as a temporary mitigation measure, while SK Hynix actively develops alternative solutions to tackle the core memory wall challenge.
Per Bloomberg, leading U.S. AI firm Anthropic is negotiating with SK Hynix for dedicated memory chip supply to support its custom in-house AI processors. SK Group Chairman Chey Tae-won revealed the news at a July 2024 San Francisco AI Summit, where Anthropic CEO Dario Amodei also spoke, praising Anthropic’s ambitious hardware development goals amid widespread industry efforts to break Nvidia’s GPU dominance. Amodei did not publicly respond to the comments.
Samsung has achieved near-native DRAM performance for AI inference workloads using external CXL pooled memory for KV cache offloading, per its official tech blog. LLMs rely on KV cache to store computed attention keys and values to cut inference latency and redundant computing, yet large models demand hundreds of gigabytes of cache, easily exceeding on-board AI processor memory. Conventional SSD or network-based offloading eases capacity limits but introduces extra latency and bandwidth overhead.
CXL technology breaks traditional DRAM physical constraints via high-bandwidth coherent PCIe interconnects, enabling scalable memory pooling across multiple devices through CXL switches. Samsung’s CMM-D (CXL Memory Module-DRAM) is purpose-built for this expanded memory architecture.
Samsung tested its CMM-D modules with Nvidia RTX PRO 6000 Blackwell GPUs over PCIe Gen5, building a 1TB CXL memory pool with vLLM and LMCache software optimizations. Tests verified the pool supports large-scale KV cache offloading with near-DRAM performance. In single-GPU setups, optimized CXL pooling matched DRAM performance as the LMCache backend. In eight-GPU multi-GPU environments, it retained 92% of native DRAM throughput with far larger memory capacity.
Further testing compared a 512GB local DRAM setup and 1TB CXL pool under rising KV cache loads. Local DRAM suffered severe performance drops from repeated cache recomputation once capacity was exceeded, while the CXL pool maintained stable performance and supported vastly larger cache footprints. Full test details are available in Samsung’s downloadable white paper.
Samsung notes current CXL switch deployments require host-level kernel and software adjustments for LMCache and other components. Still, CXL 3.0-based memory pooling shows strong application potential, with ongoing ecosystem maturation via Intel DMR, AMD Venice and new CXL 3.0 switch platforms to eliminate existing technical limitations.
Finally, a growing rift has emerged between Micron and Apple. Citing WSJ reports, Citrini’s Jukan reveals Apple is lobbying the White House for approval to deploy China’s entity-listed CXMT DRAM in its devices, while Micron is filing counter-appeals to block CXMT chip adoption. The friction stems from a long-standing grudge held by Micron CEO Sanjay Mehrotra, dating back to his Sandisk tenure when Apple forced low flash memory pricing during a buyer’s market. With the current memory market favoring suppliers, Mehrotra is reportedly unwilling to ease pricing and supply constraints for Apple.
Beijing Qianxing Jietong Technology Co., Ltd.
Sandy Yang/Global Strategy Director
WhatsApp / WeChat: +86 13426366826
Email: yangyd@qianxingdata.com
Website: www.qianxingdata.com/www.storagesserver.com
Business Focus:
ICT Product Distribution/System Integration & Services/Infrastructure Solutions
With 20+ years of IT distribution experience, we partner with leading global brands to deliver reliable products and professional services.
“Using Technology to Build an Intelligent World”Your Trusted ICT Product Service Provider!
Your message must be between 20-3,000 characters!
English
