Apple Silicon Macs and MacBooks have faced significant challenges regarding external GPU (eGPU) support, particularly for AI and ML developers. Tiny Corp has introduced a groundbreaking solution, successfully enabling an AMD eGPU to function through USB3, a technology that lacks inherent PCIe capabilities. This innovation utilizes libusb, allowing compatibility across Windows, Linux, and macOS, including Apple Silicon devices.
Traditional eGPU Connections
Normally, GPUs are connected via PCIe slots or through Thunderbolt/USB4 connections, which facilitate PCI Express tunneling. Due to this infrastructure, many external GPU solutions are constrained, limiting their support for older systems and laptops. Devices powered by Apple Silicon cannot utilize external GPUs because of insufficient driver support and fundamental differences within their architecture. Consequently, users have encountered obstacles while performing AI tasks, particularly with prompt processing.
Technical Specifications and Requirements
For successful operation of an eGPU over a USB3 interface, users need an adapter based on the ASM2464PD chip and an AMD GPU. Tiny Corp’s testing employed the ADT-UT3G adapter, which typically supports only Thunderbolt 3, Thunderbolt 4, or USB 4 connections straight out of the box. It appears that a custom firmware was likely developed to adapt this setup for USB3 use without direct PCIe communication. The controller seemingly translates PCIe commands into USB packets and vice versa.
Implementation and Limitations
This innovative approach is somewhat unconventional, sidestepping kernel-level GPU drivers and necessitating particular hardware. The method uses USB3, not originally designed for GPU data transfer, effectively shifting computation tasks from the system to the eGPU. However, this system has a limitation: data transfer speeds are capped at 10 Gbps, significantly slowing the loading of models compared to standard PCIe connections.
Future Prospects and Compatibility
As the solution operates on custom user-space drivers, it is currently limited to AMD’s RDNA 3/4 generation of GPUs, although there may be prospects for RDNA 2 support in the future. The USB3 eGPU capabilities have been incorporated into Tiny Grad’s master branch, providing users with the opportunity to experiment if they possess compatible hardware. Further technical information is anticipated from Tiny Corp as developers refine the code.