AlphaLlama runs602 tokens/son Qwen3.8-Flash-Next · one RTX 5090
Qwen3.8-Flash-Next is a ~125B MoE model, run as an IQ3_XXS GGUF (the same file for both engines) and streamed from system memory. Fast mode (output not token-identical), best task, editing a C++ file: 602 vs Strata's 425 tokens/s (1.42×), same GPU and model.
Qwen3.8-27B: 659 tokens/s on one RTX 3090
Fast mode (not token-identical), best task: rewriting a Python file. On code rewrites, 483 vs Ollama's 112 (4.3×); with exact output, 159 vs 43 (3.7×).
Qwen3.6-35B-A3B: 1,175 tokens/s on one RTX 3090
Fast mode (not token-identical), best task: renaming across a C header. On code edits, 913 vs Ollama's 233 (3.9×); with exact output, 409 vs 155 (2.6×).
Try it now
./alphallama-serve -m model.gguf