Meta and CXL tech supplier Panmnesia say AI data centers need viewing as a single co-ordinated processing resource, not as a co-located set of independent servers.
The AI data center should be a tightly-coupled resource, with a single coherence domain, like a CPU chip. This is different from existing, loosely-coupled, request-driven, enterprise data centers, which have many coherence domains. AI data centers can execute a single job across hundreds, even thousands, of GPUs, roughly similar to a high-performance computing workload involving co-ordinated processor cores, memory and network links.
Panmnesia and Meta have jointly proposed a next-generation AI datacenter architecture like this, in which the datacenter operates like a single chip. The work appears in NREE, a Nature Portfolio journal.
Myoungsoo Jung
Myoungsoo Jung, CEO of Panmnesia, said: “As AI systems continue to scale, the ability to connect large numbers of accelerators and memory devices quickly and efficiently is becoming just as important as the performance of individual accelerators. This research outlines a direction for next-generation AI infrastructure, where CXL enables the entire datacenter to operate as a single computing system.”
The components on a single processing chip, cores, etc., are designed and placed so that they have uniform link paths in a single coherence domain and do their work in a timed, co-ordinated and controlled way. In contrast existing data centers have server processors and storage in rack shelves with in-rack and between-rack network links and switches. There is no control structure so that server processing is co-ordinated.
Chip-like datacenter
Panmnesia says: “In current datacenters, accelerators inside a rack are joined by fast scale-up interconnects, while connections that leave the rack — and connections to devices other than accelerators — depend on slower scale-out networks. Measurements of such environments show heavy-tailed latency distributions, with 99th-percentile round-trip latency roughly five times the median. This is what holds the overall job back, and the more devices participate, the more often and more severely it occurs.”
For an AI datacenter to operate efficiently and speedily there needs to be control and co-ordination both in-rack and between racks of GPUs, their memory and storage. In large-scale AI infrastructure, it says, reducing latency variation between devices so that the datacenter as a whole behaves predictably matters as much as improving individual-device performance or link speed.
Panmnesia LAU
Meta and Panmnesia are proposing CXL be used, as the basis for this, with new concepts enabling it to operate at the multi-GPU-rack level. Extended CXL provides cache coherence between racks of accelerators in their scheme, supporting a larger number of devices than Nvidia’s rack-scale, NVLink-based GB200 NVL72 and UALink.
This defines the basic rules for joining devices together but not data routing and latency. They propose three dedicated hardware elements to reduce and limit latency variation:
High-fan-out non-blocking switch: connects many devices at once, reducing the number of hops and keeping path lengths similar regardless of the source.
Link acceleration unit (LAU): moves repetitive protocol processing at each connection point onto a dedicated hardware pipeline, making hop-level behavior more regular and bounding latency variation.
Fabric controller: applies the same ordering policy for handling requests across the entire system, so that transactions are processed consistently no matter which device they pass through.
Panmnesia high fan-out and non-blocking switch
The fabric controller (a combined CXL/PCIe controller) and the LAU have completed silicon validation, and the fabric switch has been fabricated as a physical silicon chip, with pre-release silicon now being supplied. This demonstrates that the proposed architecture holds at the level of manufacturable silicon.
Their architecture groups CPUs, accelerators, memory, and switches by function into trays, groups trays into pods, and connects pods through a fabric — a regular tray–pod–fabric hierarchy designed to preserve fixed-hop, more consistent communication paths and timing.
Panmnesia Fabric Controller
Compared to Nvidia’s design, in which one CPU is coupled to two accelerators over NVLink-C2C, the rack interior is connected by NVLink, and servers and racks are joined by a scale-out network such as Ethernet or InfiniBand, their scheme:
Provides an 8x increase in accelerators directly coordinated by a single CPU; from 2 to 16,
Enables up to 960 accelerators to operate together in a single coherence domain,
Reduces data access latency from the microsecond level to several hundred nanoseconds; approximately an order of magnitude lower,
The failure replacement unit becomes a malfunctioning single device instead of a server. Separating resources by type allows the system to replace only the malfunctioning devices rather than an entire server, avoiding wasted resources and prolonged operational halts.
Panmnesia has already implemented the architecture's core components in silicon, completed validation, and is now preparing them for commercial supply. Future developments are looking at optical interconnects to increase speed and scalability.
Bootnote
The NREE article reference is available.
Beijing Qianxing Jietong Technology Co., Ltd.
Sandy Yang/Global Strategy Director
WhatsApp / WeChat: +86 13426366826
Email: yangyd@qianxingdata.com
Website: www.qianxingdata.com/www.storagesserver.com
Business Focus:
ICT Product Distribution/System Integration & Services/Infrastructure Solutions
With 20+ years of IT distribution experience, we partner with leading global brands to deliver reliable products and professional services.
“Using Technology to Build an Intelligent World”Your Trusted ICT Product Service Provider!
Your message must be between 20-3,000 characters!
English
