LLM leaderboard
Runs are grouped by model and quantization, and ranked by decode speed within each group. Repeated runs of the same machine and backend are folded into their best one; untick the box to see them all. Click a column header to rank by another column. ↑ higher is better, ↓ lower is better. Filters are applied together (AND).
* Prefill and TTFT were measured with the server's prompt cache on (runs before 2026-11-14, and router runs), so they time a cache hit and are likely too fast. Decode is not affected. Prompt is the prompt length per request: prefill rises and decode falls with it, so compare runs of a similar prompt size.
Other modalities
Non-LLM Lemonade recipes -- text-to-speech, speech-to-text, image/audio/3D generation, embeddings, reranking, classification. Each has its own throughput metric, so these aren't ranked against the table above. Every metric here is higher-is-better; for audio, × real time is seconds of audio per second of processing.
| Date | Profile | Model | Backend | Engine | Metric ↑ | Valid |
|---|
Router cost runs
A collection.router run with a mixed prompt set: where the router sent each request, what that cost,
and what sending every request to the baseline model would have cost. A measured baseline sends each prompt
once to the baseline model and prices its own token counts; an estimated one reuses the routed model's counts,
which understates a reasoning model's cost. Prices are USD per million tokens, from Lemonade's own cost metadata or
the submitter's --cost; non-cloud models count as free. Costs are shown per 1,000 requests.
| Date | Profile | Router | Saved ↑ | Cost / 1k req | Baseline / 1k req | Baseline | Split | Local | Requests | Valid |
|---|---|---|---|---|---|---|---|---|---|---|
| 2026-11-14 | Radeon-AI-PRO-R9700 | user.LM-D-three-tier | 63.6% | $0.12 | $0.34 | DeepSeek-V4-Pro (measured) | Qwen3-1.7B-GGUF 28% · Qwen3.5-35B-A3B-GGUF 38% · DeepSeek-V4-Pro 35% | 65% | 120 | valid |
| 2026-11-14 | Radeon-AI-PRO-R9700 | user.LM-C-llm-classifier | 1.1% | $0.16 | $0.16 | Qwen3-235B-A22B-Instruct-2507 (measured) | Qwen3-235B-A22B-Instruct-2507 95% · Qwen3.5-35B-A3B-GGUF 5% | 5% | 120 | valid |
| 2026-11-14 | Radeon-AI-PRO-R9700 | user.LM-A-keyword-code-to-cloud | 53.6% | $0.04 | $0.09 | deepseek-v3.2 (measured) | Qwen3-4B-GGUF 62% · deepseek-v3.2 38% | 62% | 120 | valid |