Quantized Qwen3-TTS on CPU: Benchmarks and the Graviton Cost Breakdown
A scored comparison of six CPU-quantized Qwen3-TTS runtimes, real-time-factor numbers measured on 4 vCPU/15GB hardware, and the AWS infra move (Fargate to Batch Spot Graviton) that cut TTS hosting to ~$5-7/mo.
Fast, high-quality text-to-speech on CPU-only hardware is now viable for podcast-length synthesis. Quantized Qwen3-TTS running on AWS Graviton Spot instances produces output quality comparable to GPU inference, at roughly $5-7/mo in compute.
Library comparison
Six CPU-targeting Qwen3-TTS runtimes, scored on license, model sizes, documented CPU real-time-factor (RTF), and cloning support:
| Rank | Candidate | Score | License | Sizes | CPU RTF (documented) | "ryan" preset | Transcript cloning |
|---|---|---|---|---|---|---|---|
| 1 | gabriele-mastrapasqua/qwen3-tts | 9 | MIT | 0.6B + 1.7B | real, multi-CPU (0.6B int8 sub-realtime; 1.7B int8 ~1.8-2.4x) | yes (default) | audio-only + optional .qvoice ICL text |
| 2 | Serveurperso/qwentts.cpp | 8 | MIT+Apache-2.0 | 0.6B + 1.7B | none (hardcoded thread /2, needs source patch) | yes | first-class --ref-text ICL |
| 3 | predict-woo/qwen3-tts.cpp | 5 | NONE (open issue) | 0.6B only | 1.94x (Ryzen 5 3600, 4T), ~3GB RAM | no | no |
| 4 | koboldcpp | 4 | AGPL-3.0 | 0.6B+1.7B | none (Vulkan only) | not found | via predict-woo underneath |
| 5 | HaujetZhao/Qwen3-TTS-GGUF | 5* | MIT | LLM-portion only | none | - | - |
| - | Danmoreng/qwen3-tts(.cpp) | 2 | - | - | - | - | - |
gabriele-mastrapasqua/qwen3-tts wins on the only criterion that matters for production: real, reproducible CPU RTF numbers across multiple CPUs, MIT-licensed, both model sizes, with a runtime --int8/--int4 quant switch and a --serve persistent HTTP mode (no reload between requests).
RTF numbers
Measured in-sandbox on 4 vCPU / 15GB, gabriele's 0.6B model:
| Mode | Quant | RTF |
|---|---|---|
| Non-cloning ("ryan" preset) | int8 | 3.07x |
| Non-cloning ("aiden" preset) | int8 | 2.85x |
| Non-cloning ("ryan" preset) | int4 | 3.26x |
| Cloning (reference audio) | int8 | 3.17x |
| Cloning (bootstrap-ref) | int8 | 2.50x |
RTF here means seconds of compute per second of audio - lower is faster. int4 was slower than int8 (dequantization overhead outweighs the smaller weights), so int8 is the shipped precision everywhere.
The production budget was ≤2.67x (fit a 45-minute episode in under 2 hours); only the bootstrap-ref case cleared that in the sandbox. On real Graviton hardware in production, throughput clears the budget comfortably. For reference, vendor-published numbers put Neoverse-N1 (Graviton2) at RTF ≈1.28x - that figure is upstream-claimed, not independently measured here.
Hardware and cost
TTS synthesis moved off ECS Fargate (x86) onto AWS Batch running EC2-Spot Graviton instances (c7g/c8g, arm64). The real bottleneck on Fargate wasn't raw compute - it was the model reloading from disk on every chunk, ~85-90 times per episode for a 2.35GB int8 model. A persistent --serve HTTP server eliminated the reload entirely.
Batch's EC2-Spot pool is what unlocks Graviton instance selection (Fargate has no arm64 Spot option with this flexibility). The Batch compute environment lists both c7g.4xlarge/8xlarge/12xlarge (Graviton3) and c8g.4xlarge/8xlarge/12xlarge (Graviton4), so Spot's capacity-optimized allocator can land on whichever pool has room. The binary is compiled with -mcpu=neoverse-v1 - the Graviton3 ISA, forward-compatible with Graviton4, but not neoverse-v2, which SIGILLs on c7g hardware.
A GPU tier existed briefly for a batched-inference experiment and has since been retired entirely - no CUDA dependency remains anywhere in the pipeline. Current steady-state cost for the Batch Spot Graviton TTS pipeline (bursty preprocessing + synthesis jobs, Spot-capacity-optimized): ~$5-7/mo.
Where it stands today
0.6B-int8 is the standard voice model for all users, free and paid. int8 runs everywhere; int4 was evaluated and abandoned as slower on Graviton's i8mm path. Output quality at int8 - and even int4 - is on par with what the GPU pipeline produced, at a fraction of the cost. A larger 1.7B model with voice cloning is planned as a paid-tier option; it is not yet shipped.
For the full model-selection breakdown (presets vs. cloning vs. voice design), see the voice cloning guide.
Hear the quality yourself
My free podcast tool runs this same quantized Qwen3-TTS pipeline - fast CPU voices at a fraction of GPU hosting cost.
