Zero-Copy GPU Inference on Apple Silicon
Recent work by Agam Brahma details a significant breakthrough in GPU-accelerated computing on Apple Silicon: the ability to perform zero-copy inference from WebAssembly (Wasm). This advancement bypasses the traditional performance bottlenecks associated with transferring data between the CPU, the Wasm runtime, and the GPU.
The Traditional Bottleneck
Typically, running GPU inference with Wasm involves a costly data transfer process. Because Wasm operates within a sandboxed environment, accessing data from the GPU requires copying data out of the Wasm module's linear memory, into host memory, and then copying it across the PCIe bus to the GPU. This introduces two copies, latency, and impedance mismatches between the isolated Wasm environment and the hardware accelerator.
Apple Silicon's Unified Memory Architecture
Apple Silicon's Unified Memory Architecture (UMA) fundamentally changes this dynamic. UMA provides a single pool of physical memory accessible by both the CPU and GPU, eliminating the need for a dedicated bus. This means a pointer accessible by the CPU can also be directly accessed by the GPU, reading from and writing to the same DRAM.
The Three-Link Chain
The core innovation lies in establishing a reliable and efficient chain to utilize this direct memory access. Brahma validated this approach in three steps:
mmapfor Page Alignment: Utilizingmmapto acquire page-aligned memory, a requirement for GPU access.- Wasm Linear Memory Mapping: Mapping the Wasm module's linear memory directly to the
mmap-allocated pages. - Metal Framework Integration: Leveraging Apple's Metal framework to access the shared memory from the GPU.
Diagram of the three-link zero-copy chain: image omitted due to site embedding policy; open the original article (Abacusnoir) (opens in a new tab) to view it. Photo/source: [https://abacusnoir.com/2026/04/18/zero-copy-gpu-inference-from-webassembly-on-apple-silicon/ (opens in a new tab)]
Brahma emphasizes the importance of validating each link individually to ensure the entire pipeline functions correctly.
Driftwood: Stateful AI Inference
This zero-copy capability is the foundation of a project called Driftwood, designed for stateful AI inference. By treating Wasm as the control plane and the GPU as the compute plane, Driftwood aims to achieve near-zero overhead between the two. The article describes a scenario where a Wasm module fills a matrix in its linear memory, the GPU performs computations on it, and the Wasm module directly observes the results via the same memory pointer – all without any data copies.
Why It Matters
This development has several significant implications for developers and the broader tech landscape:
- Performance Gains: Eliminating data copies dramatically reduces latency and improves performance for AI inference and other GPU-accelerated tasks. This is particularly impactful for applications requiring frequent data transfers.
- Simplified Development: The zero-copy approach simplifies the development process, removing the need for complex memory management and data synchronization between the CPU, Wasm runtime, and GPU.
- New Architectural Possibilities: The ability to treat Wasm as a control plane and the GPU as a compute plane opens up possibilities for novel application architectures. The system's architecture closely resembles function shipping, but with the VM providing safety and portability.
- Apple Silicon Advantage: This innovation highlights a key advantage of Apple Silicon's UMA over traditional discrete GPU setups. It suggests Apple Silicon may become a particularly attractive platform for AI and machine learning workloads.
It's important to note that this work is still in its early stages. Further investigation and optimization are needed to fully realize the potential of this approach. The article does not detail the performance benefits quantitatively, but the conceptual advantage is clear. It will be interesting to track the development of Driftwood and see how this zero-copy capability is applied in real-world applications. Finally, it remains uncertain how well this approach scales to larger models and more complex workloads. The author indicates they are still “poking at it” and further research is required.