Home / Benchmarks

Open-model AI benchmarks (September 2026)

Published evaluation results for DeepSeek V4.1, GLM-5.3, Kimi K3, Qwen3.8, and MiniMax M3 — agentic coding, software engineering, science reasoning, web research, and security — with charts, sources, and independent checks.

How to read these numbers: Scores are vendor-reported unless marked independent, as of September 2026. Labs use different agent harnesses, reasoning settings, and benchmark versions, so treat cross-lab gaps of a few points as roughly equal and test on your own workload.

Terminal-Bench 2.1 score by model release date (2026)

Open models closed most of the gap on agentic terminal work in a single summer: from 66.0 in June to 90.6 in September.

Terminal-Bench 2.1 score by model release date (2026)Terminal-Bench 2.1 (%) — MiniMax M3 (Jun 1): 66.0; GLM-5.2 (Jun 13): 81.0; Kimi K3 (Jul 16): 88.3; Qwen3.8-Max (Aug 3): 86.6; GLM-5.3 (Aug 14): 88.2; DeepSeek V4.1 Flash (Sep 10): 90.65060708090100Terminal-Bench 2.1 (%)66.0MiniMax M3Jun 181.0GLM-5.2Jun 1388.3Kimi K3Jul 1686.6Qwen3.8-MaxAug 388.2GLM-5.3Aug 1490.6DeepSeek V4.1 FlashSep 10
Terminal-Bench 2.1 score by model release date (2026). Sources: MiniMax M3 launch coverage (VentureBeat) (opens in new tab); Z.ai GLM-5 series repository (opens in new tab); Moonshot AI Kimi K3 model card (opens in new tab); Qwen 3.8 benchmarks (Apidog, citing Alibaba) (opens in new tab); GLM-5.3 benchmarks (MindStudio, citing Z.ai) (opens in new tab); DeepSeek-V4.1-Flash model card (NVIDIA) (opens in new tab).
View chart data as a table
ModelReleasedTerminal-Bench 2.1 (%)
MiniMax M3Jun 166.0
GLM-5.2Jun 1381.0
Kimi K3Jul 1688.3
Qwen3.8-MaxAug 386.6
GLM-5.3Aug 1488.2
DeepSeek V4.1 FlashSep 1090.6

Results by benchmark

Each table is sorted by score. Models without a published result are omitted rather than guessed.

Terminal-Bench 2.1 — agentic coding in real terminal environments
ModelScore (%)
DeepSeek V4.1 FlashDeepSeek · live on kryyt
90.6
Kimi K3Moonshot AI
88.3
GLM-5.3Z.ai (Zhipu)
88.2
Qwen3.8-MaxAlibaba Cloud
86.6
GLM-5.3 FlashZ.ai (Zhipu)
84.3
MiniMax M3MiniMax
66.0

Sources: DeepSeek-V4.1-Flash model card (NVIDIA) (opens in new tab); Moonshot AI Kimi K3 model card (opens in new tab); GLM-5.3 benchmarks (MindStudio, citing Z.ai) (opens in new tab); Qwen 3.8 benchmarks (Apidog, citing Alibaba) (opens in new tab); GLM-5.3-Flash benchmarks (DataCamp, citing Z.ai) (opens in new tab); MiniMax M3 launch coverage (VentureBeat) (opens in new tab).

GPQA Diamond — graduate-level science reasoning
ModelScore (%)
Kimi K3Moonshot AI
93.5
Qwen3.8-MaxAlibaba Cloud
92.6
DeepSeek V4.1 FlashDeepSeek · live on kryyt
90.9

Sources: Moonshot AI Kimi K3 model card (opens in new tab); Qwen3.8-Max benchmarks (DataCamp, citing Alibaba) (opens in new tab); DeepSeek-V4.1-Flash model card (NVIDIA) (opens in new tab).

DeepSWE v1.1 — repairing fresh, real-world software issues
ModelScore (%)
DeepSeek V4.1 FlashDeepSeek · live on kryyt
74.2
Kimi K3Moonshot AI
67.5
GLM-5.3 FlashZ.ai (Zhipu)
63.4

Sources: DeepSeek-V4.1-Flash model card (NVIDIA) (opens in new tab); Moonshot AI Kimi K3 model card (opens in new tab); GLM-5.3-Flash benchmarks (DataCamp, citing Z.ai) (opens in new tab).

SWE-bench Pro — resolving issues in professional codebases
ModelScore (%)
Qwen3.8-MaxAlibaba Cloud
67.7
GLM-5.2Z.ai (Zhipu)
62.1
MiniMax M3MiniMax
59.0

Sources: Qwen3.8-Max benchmarks (DataCamp, citing Alibaba) (opens in new tab); Z.ai GLM-5 series repository (opens in new tab); MiniMax M3 launch coverage (VentureBeat) (opens in new tab).

Humanity's Last Exam — frontier knowledge and reasoning (no tools)
ModelScore (%)
Qwen3.8-MaxAlibaba Cloud
43.6
Kimi K3Moonshot AI
43.5

Sources: Qwen 3.8 benchmarks (Apidog, citing Alibaba) (opens in new tab); Moonshot AI Kimi K3 model card (opens in new tab).

BrowseComp — autonomous web research
ModelScore (%)
Kimi K3Moonshot AI
91.2
MiniMax M3MiniMax
83.5

Sources: Moonshot AI Kimi K3 model card (opens in new tab); MiniMax M3 launch coverage (VentureBeat) (opens in new tab).

CyberGym — security vulnerability discovery
ModelScore (%)
DeepSeek V4.1 FlashDeepSeek · live on kryyt
88.1
GLM-5.3Z.ai (Zhipu)
84.5

Sources: DeepSeek-V4.1-Flash model card (NVIDIA) (opens in new tab); GLM-5.3 benchmarks (Emergent, citing Z.ai) (opens in new tab).

Vendor-reported vs independent results

Scores can shift by several points depending on who runs the eval and with which agent harness.

Terminal-Bench 2.1 — vendor-reported vs independently measured
BenchmarkScores (%)
Terminal-Bench 2.1 — Kimi K3Vendor vs Artificial Analysis
Vendor-reported88.3
Artificial Analysis (independent)85.0
Terminal-Bench 2.1 — MiniMax M3Vendor vs Vals AI
Vendor-reported66.0
Vals AI (independent)53.6

Sources: Kimi K3 benchmark caveats (Emergent, citing Artificial Analysis) (opens in new tab); Vals AI — independent MiniMax M3 evaluation (opens in new tab); Moonshot AI Kimi K3 model card (opens in new tab); MiniMax M3 launch coverage (VentureBeat) (opens in new tab).

DeepSeek on GPQA Diamond, generation by generation

DeepSeek's science-reasoning score climbed more than 30 points across five releases.

DeepSeek on GPQA Diamond, generation by generationGPQA Diamond (%) — V3 (Dec 2024): 59.1; R1 (Jan 2025): 71.5; R1-0528 (May 2025): 81.0; V4 Flash (Apr 2026): 89.9; V4.1 Flash (Sep 2026): 90.95060708090100GPQA Diamond (%)59.1V3Dec 202471.5R1Jan 202581.0R1-0528May 202589.9V4 FlashApr 202690.9V4.1 FlashSep 2026
DeepSeek on GPQA Diamond, generation by generation. Sources: DeepSeek-V3 technical report (opens in new tab); DeepSeek API change log (opens in new tab); DeepSeek-V4.1-Flash technical report (opens in new tab).
View chart data as a table
ModelReleasedGPQA Diamond (%)
V3Dec 202459.1
R1Jan 202571.5
R1-0528May 202581.0
V4 FlashApr 202689.9
V4.1 FlashSep 202690.9

What each benchmark measures

Terminal-Bench 2.1
Agentic coding in real terminal environments.
GPQA Diamond
Graduate-level science reasoning.
DeepSWE v1.1
Repairing fresh, real-world software issues.
SWE-bench Pro
Resolving issues in professional codebases.
Humanity's Last Exam
Frontier knowledge and reasoning (no tools).
BrowseComp
Autonomous web research.
CyberGym
Security vulnerability discovery.
NL2Repo-Bench
Generating whole repositories from a spec.
AutomationBench
Workflow-automation agents.
Agents' Last Exam
Long-horizon agentic tasks.
Terminal-Bench 3.0
Harder next-generation terminal tasks.
FrontierSWE
Frontier-difficulty software engineering.
MMMU-Pro
Multimodal, college-level reasoning.
PaperBench
Reproducing ML research papers.
OSWorld-Verified
Operating a real desktop computer.
IFBench
Precise instruction following.
MCP Atlas
Tool use through MCP servers.

Benchmark FAQ

Which open model scores best on agentic coding?

On Terminal-Bench 2.1 — the benchmark every lab in our roster reports — DeepSeek V4.1 Flash leads at 90.6, followed by Kimi K3 (88.3), GLM-5.3 (88.2), and Qwen3.8-Max (86.6), all vendor-reported. On DeepSWE v1.1, DeepSeek V4.1 Flash (74.2) also leads Kimi K3 (67.5).

Which model is strongest on science reasoning?

On GPQA Diamond, Kimi K3 reports 93.5, Qwen3.8-Max 92.6, and DeepSeek V4.1 Flash 90.9. GLM-5.3 and MiniMax M3 have not published GPQA Diamond results.

Are these benchmark scores independent?

Most are vendor-reported. Where independent measurements exist we show them: Artificial Analysis measured Kimi K3 at 85 on Terminal-Bench 2.1 (vs 88.3 reported), and Vals AI measured MiniMax M3 at 53.6 (vs 66.0 reported). Harness and settings differences explain much of the gap.

Which of these models can I use on kryyt today?

Kryyt AI, powered by DeepSeek V4.1 Flash, is live now in browser chat, the VS Code extension, and the OpenAI-compatible API. GLM, Kimi, Qwen, and MiniMax models are coming soon.

How often is this page updated?

We update results when labs publish new models or revised evaluations. Figures on this page are current as of September 2026.

Follow the models as they move: for ongoing AI news, new model releases, and benchmark analysis, read our sister publication ai-ai.ooo.

Sources

Ship with one endpoint

Get a key, change your base URL, and call every model on this page. No new SDKs.

Get your API key