Swiftlet Runs an 80B Qwen Model in 4.3 GB of RAM on Mac

Swiftlet is a new open-source inference project targeting Apple hardware. The headline numbers: an 80B Qwen model running inside 4.3 GB of RAM on Mac, and a 35B model running on-device on iPhone.

Those figures are possible through aggressive quantization. Extreme-low-bit compression trades some precision for a memory footprint small enough to fit on unified-memory Apple Silicon and mobile DRAM alike. The tradeoff is real, but for a large class of local inference tasks the output quality remains usable.

The practical consequence is significant for solo builders and on-device product teams. Running an 80B-class model previously required a GPU workstation or a cloud API. Swiftlet moves that threshold to a MacBook. Running a 35B model on an iPhone removes the network requirement entirely, which matters for latency-sensitive or privacy-sensitive applications.

The project targets the Qwen model family specifically. Qwen models from Alibaba have become a common substrate for local inference work because of their strong multilingual performance and permissive licensing relative to comparably sized Western alternatives. Swiftlet builds on that foundation and optimizes the delivery layer for Apple's hardware stack.

For engineers already working with llama.cpp or MLX, Swiftlet is worth evaluating as a narrower, hardware-specific alternative. It does not aim to be a general-purpose inference engine. The scope is deliberately constrained to Apple Silicon, which lets it optimize more aggressively than cross-platform runtimes.

The repository is available on GitHub. If the memory numbers hold under real workloads, this shifts what is feasible for offline-first Mac and iOS applications without any server dependency.

Source