Microsoft has announced the implementation of Remote Direct Memory Access (RDMA) over the USB4 interface, enabling multiple personal computers to be combined into a single computing system. The new feature is primarily aimed at locally running large language models that do not fit into the RAM of a single device. Essentially, enthusiasts and developers are given a tool to build a home AI cluster without the need to purchase specialized server hardware or high-speed InfiniBand or RoCE network cards.
Direct Memory Access Without the Network Stack
RDMA technology allows data to be transferred directly between the RAM areas of two systems, bypassing a significant portion of the traditional network stack. This ensures minimal latency and significantly reduces the load on the central processing unit, which in standard network exchanges must participate in every data copying cycle. This is why RDMA has long been an integral component of storage systems in data centers and distributed computing clusters. Now, this technology has found a new application in the consumer and workstation segment. According to tests, the latency when using RDMA over USB4 is 28–50 microseconds, whereas with a standard Ethernet connection it reaches 116–194 microseconds.
Connection Architecture and Hardware Requirements
Implementing this solution requires a USB4 interface, which provides significantly higher bandwidth compared to previous USB generations. This means that only reasonably modern computers equipped with matching controllers can be linked this way. Technically, Microsoft utilizes the peer-to-peer device connection mechanism already present in Windows 11 over USB4: each system creates its own USB4 domain, after which the Windows connection manager establishes a peer-to-peer tunnel between the devices. A specialized RDMA software component links data transfer queues with the ring buffers of the USB4 controller's DMA, avoiding additional memory copying operations. Two pairs of ring buffers are allocated for each connection. When combining three computers, one of them must be equipped with two USB4 ports.
Demonstration Test Results
To demonstrate functionality, Microsoft tested the technology on mini-PCs powered by the Ryzen AI Max+ 395 processor with 128 GB of RAM. Three connected computers successfully ran the Qwen3-235B-A22B model with 4-bit quantization. Microsoft reported a generation speed of around 18.5 tokens per second. Meanwhile, AMD, using its own hardware configuration, achieved 21.98 tokens per second. The discrepancy in figures is explained by differences in hardware configuration and test bench setups between the two companies.
Open Source and Future Prospects
The NetworkDirect provider for USB4, as well as the llama.cpp-based test platform used for the demonstration, are open-source projects. This means the developer community can independently adapt the solution, refine drivers, and create custom cluster configurations. For local AI labs, small research groups, and enthusiasts working with large language models, this solution can become an affordable alternative to cloud GPU clusters by utilizing existing desktop hardware.