Shaduf AI Model Degradation Watch

Publication cutoff: 2026-08-06 UTC. Facts first published after this date are excluded. Mutable pages without versioned historical snapshots are identified as current observations, not historical proof.

Coverage: 10 monitored rows: three GPT-5.6 tiers, three Claude rows, one Grok row, and exactly three open-weight/open-source models.

This report uses public vendor documentation, public status histories, independent public comparisons, and attributable community reporting. It contains no Shaduf-run evaluations, prompts, canaries, benchmarks, or experiments.

Current summary

The public record does not establish broad base-model degradation for any monitored model. It does establish rapid changes in model availability, routing, safeguards, fallback behavior, and infrastructure reliability.

Anthropic provides the clearest recent evidence of operational instability. Its status history records elevated errors affecting Claude Mythos 5, Fable 5, Opus 5, and Sonnet 5 on August 5, with additional multi-model and login incidents on August 3–4. These are availability and error-rate incidents, not evidence that model weights became less capable. The status page reported no new incident on August 6 at the time observed. (Claude Status

Claude Fable 5 and Claude Mythos 5 must be monitored as separate service configurations, but not treated as separate base models. Anthropic explicitly states that they share the same underlying model. Fable adds safety classifiers and may fall back to Opus 4.8; Mythos is restricted through Project Glasswing with selected safeguards lifted. (Anthropic launch announcement — 2026-06-09, current model overview

OpenAI’s GPT-5.6 family reached general availability on July 9. Sol, Terra, and Luna are distinct API tiers, but their product exposure differs materially: standard ChatGPT exposes Sol, while Terra and Luna are available through ChatGPT Work, Codex, and the API. ChatGPT may continue with GPT-5.4 Thinking mini after a GPT-5.6 reasoning allowance is reached. OpenAI also recorded a July 9 model-selection capacity incident and a July 17 Codex Sol server-overload incident. (OpenAI GA announcement — 2026-07-09, OpenAI ChatGPT availability documentation, July 17 status incident

Grok 4.5 has a verified API identifier, grok-4.5, and is documented for Grok Build, Cursor, and Office add-ins. The public consumer documentation confirms Grok on the web, iOS, Android, and X, but does not establish that those surfaces use the API model ID unchanged. The current xAI status page reported no declared incident, but it does not expose a comparable model-specific historical incident record. (Grok 4.5 API documentation — updated 2026-07-17, Grok consumer documentation — updated 2026-06-29, xAI status

The three open-weight selections are Kimi K3, GLM-5.2, and DeepSeek V4 Pro. Their weights and licenses are publicly verifiable. Kimi K3 uses a custom Kimi K3 License and is therefore best described as open-weight rather than straightforward OSI open-source software. GLM-5.2 and DeepSeek V4 Pro are published under MIT terms. (Kimi K3 repository, GLM-5.2 official release, DeepSeek V4 release

The most useful conclusion for degradation monitoring is therefore dimensional separation:

• base-model capability;
• availability and error rates;
• latency and capacity;
• routing, aliases, and fallback;
• system instructions and safeguards;
• context handling and compaction;
• connected tools and product wrappers.

A public benchmark snapshot can describe capability at one point in time. It cannot, by itself, establish degradation.

Monitored model rows

There are ten monitored rows, but only nine independently identifiable base-model families because Fable 5 and Mythos 5 share underlying weights. Sonnet 5 is a current Claude model, but it is not added as an eleventh row; the three Claude slots are Fable, Mythos, and the next distinct current model, Opus 5.

Dated change and incident timeline

Chart 1 — Artificial Analysis Intelligence Index snapshot

Data verified: 2026-08-05. Interpretation: a public hosted-model snapshot on the source’s displayed scale, not a success percentage and not a degradation trend. Mythos is omitted because no standalone comparable score was published.

Claude Opus 5       60.7 |██████████████████████████████
Claude Fable 5      59.9 |█████████████████████████████
GPT-5.6 Sol         58.9 |████████████████████████████
Kimi K3             57.1 |███████████████████████████
GPT-5.6 Terra       55.0 |██████████████████████████
Grok 4.5            53.8 |█████████████████████████
GPT-5.6 Luna        51.2 |████████████████████████
GLM-5.2             51.1 |████████████████████████
DeepSeek V4 Pro     44.3 |█████████████████████

The underlying source labels Kimi K3 as “Closed,” which conflicts with Moonshot’s official public weights and license. The score is retained; the openness classification is taken from the official Kimi repository instead. (BenchLM display-only mirror, Kimi official repository

model,score,unit,source_url,data_verified_date,configuration_note
Claude Opus 5,60.7,AA Intelligence Index score,https://benchlm.ai/benchmarks/artificialanalysis,2026-08-05,Public AA model row; exact effort/surface not exposed in mirror
Claude Fable 5,59.9,AA Intelligence Index score,https://benchlm.ai/benchmarks/artificialanalysis,2026-08-05,Effective hosted configuration; Anthropic documents possible Opus 4.8 fallback
GPT-5.6 Sol,58.9,AA Intelligence Index score,https://benchlm.ai/benchmarks/artificialanalysis,2026-08-05,Public AA model row; exact surface/configuration not exposed in mirror
Kimi K3,57.1,AA Intelligence Index score,https://benchlm.ai/benchmarks/artificialanalysis,2026-08-05,Hosted public row; official weights are open despite source classification label
GPT-5.6 Terra,55.0,AA Intelligence Index score,https://benchlm.ai/benchmarks/artificialanalysis,2026-08-05,Public AA model row
Grok 4.5,53.8,AA Intelligence Index score,https://benchlm.ai/benchmarks/artificialanalysis,2026-08-05,Public AA model row; exact consumer/X surface not established
GPT-5.6 Luna,51.2,AA Intelligence Index score,https://benchlm.ai/benchmarks/artificialanalysis,2026-08-05,Public AA model row
GLM-5.2,51.1,AA Intelligence Index score,https://benchlm.ai/benchmarks/artificialanalysis,2026-08-05,Hosted public row; not a local self-hosting measurement
DeepSeek V4 Pro (Max),44.3,AA Intelligence Index score,https://benchlm.ai/benchmarks/artificialanalysis,2026-08-05,Hosted public row; Max configuration label retained

Chart 2 — Selected open-weight models on Vellum public comparisons

Vellum’s page was updated July 24, 2026 and states that its data combines provider-reported results with evaluations by Vellum and the open-source community. Scores are comparable within each metric, not across metrics.

Humanity's Last Exam

Kimi K3            56.0 |████████████████████████████
GLM-5.2            54.7 |███████████████████████████
DeepSeek V4 Pro    48.2 |████████████████████████

GPQA Diamond

Kimi K3            93.5 |███████████████████████████████████████████████
GLM-5.2            91.2 |█████████████████████████████████████████████
DeepSeek V4 Pro    90.1 |████████████████████████████████████████████

model,metric,score,unit,source_url,source_updated_date
Kimi K3,Humanity's Last Exam,56.0,percent,https://www.vellum.ai/open-llm-leaderboard,2026-07-24
GLM-5.2,Humanity's Last Exam,54.7,percent,https://www.vellum.ai/open-llm-leaderboard,2026-07-24
DeepSeek V4 Pro,Humanity's Last Exam,48.2,percent,https://www.vellum.ai/open-llm-leaderboard,2026-07-24
Kimi K3,GPQA Diamond,93.5,percent,https://www.vellum.ai/open-llm-leaderboard,2026-07-24
GLM-5.2,GPQA Diamond,91.2,percent,https://www.vellum.ai/open-llm-leaderboard,2026-07-24
DeepSeek V4 Pro,GPQA Diamond,90.1,percent,https://www.vellum.ai/open-llm-leaderboard,2026-07-24

Chart 3 — Frontend Code Arena snapshot

This is a narrow human-preference leaderboard, not a general capability measure.

Kimi K3             1679 |████████████████████████████████
Claude Fable 5      1631 |██████████████████████████████
GPT-5.6 Sol         1618 |█████████████████████████████

The snapshot was dated July 16 in Kilo’s July 21 report.

model,metric,score,source_url,leaderboard_date,publication_date
Kimi K3,Frontend Code Arena,1679,https://blog.kilo.ai/p/kimi-k3,2026-07-16,2026-07-21
Claude Fable 5,Frontend Code Arena,1631,https://blog.kilo.ai/p/kimi-k3,2026-07-16,2026-07-21
GPT-5.6 Sol,Frontend Code Arena,1618,https://blog.kilo.ai/p/kimi-k3,2026-07-16,2026-07-21

What is established

• GPT-5.6 Sol, Terra, and Luna are official, separately named model tiers with separate API identifiers.
• Standard ChatGPT, ChatGPT Work, Codex, and the OpenAI API do not expose those tiers identically.
• Claude Fable 5 and Mythos 5 share underlying weights but differ in safeguards and access.
• Claude model IDs from the current generation are pinned snapshots; Anthropic states that serving infrastructure, safety classifiers, routers, and sampling logic can still change around fixed weights. (Anthropic model-ID documentation
• Anthropic recorded multiple recent model-specific and multi-model availability incidents.
• OpenAI recorded GPT-5.6 model-selection and Codex Sol availability incidents.
• Grok 4.5 has the verified API ID grok-4.5; exact consumer and X routing is not publicly established.
• GLM-5.2, DeepSeek V4 Pro, and Kimi K3 have publicly downloadable weights and published licenses.
• Kimi K3 is open-weight under a custom license, not simply MIT.
• Public independent snapshots place Opus 5, Fable 5, GPT-5.6 Sol, and Kimi K3 near the same frontier band, but the scores represent particular hosted configurations and evaluation conditions.

What is not established

• No public fixed-surface, longitudinal evidence reviewed here establishes that any monitored base model became less capable by August 6.
• A status incident does not establish a model-quality regression.
• A user complaint, 429, latency increase, refusal, or fallback does not establish degraded model weights.
• Fable 5’s public score cannot be transferred directly to Mythos 5 because safeguards, access, and fallback differ.
• Consumer ChatGPT, Claude.ai, Grok, X, Codex, Claude Code, and hosted API endpoints cannot be assumed to use identical system instructions, tool environments, routing, or effort settings.
• A hosted open-weight score does not establish the behavior of a local deployment of the same weights.
• The absence of a provider status entry does not prove the absence of an incident.
• A single cross-model ranking does not establish a universal capability ordering.

Uncertainty

The principal uncertainty is configuration identity. Public comparisons often combine a model name with effort settings, fallback models, provider harnesses, tools, and different context policies. For example, Anthropic explicitly documents Fable fallback to Opus 4.8, while the Kimi technical materials compare Fable with fallback enabled. (Kimi technical repository

Artificial Analysis pages also show small numerical differences across dated or rounded presentations: the August 5 display-only snapshot lists Opus 5 at 60.7 and Grok 4.5 at 53.8, while separate July pages display rounded values of 61 and 54. These differences are not evidence of degradation or improvement without a controlled longitudinal comparison.

Kimi K3’s official announcement confirms availability through Kimi API, but the current public model-list documentation inspected does not expose a K3 API slug. The weight-repository identifier moonshotai/Kimi-K3 is therefore asserted; an API identifier is not invented.

The current pages for OpenAI, Anthropic, xAI, and hosted open-weight services are mutable. A current page can document present behavior while failing to preserve the historical state that existed on an earlier date.

Freshness

• Cutoff: 2026-08-06 UTC.
• The latest comparison snapshot used here was verified 2026-08-05.
• Claude status was observed with entries through 2026-08-06.
• xAI status was observed on 2026-08-06 with no declared incident.
• Static release dates used in the timeline are no later than 2026-07-30, except for current status and current documentation observations on August 6.
• Vellum’s open-weight page was updated 2026-07-24.
• Kilo’s Frontend Code report was published 2026-07-21 and refers to a July 16 leaderboard snapshot.
• Pages without a stable historical version are treated as current observations only.

Sources

Official model and release documentation

• OpenAI, “Previewing GPT-5.6 Sol” — 2026-06-26
• OpenAI, “GPT-5.6” — 2026-07-09; updated July 30
• OpenAI API: GPT-5.6 Sol — current documentation accessed 2026-08-06
• OpenAI API: GPT-5.6 Terra — current documentation accessed 2026-08-06
• OpenAI API: GPT-5.6 Luna — current documentation accessed 2026-08-06
• OpenAI Help Center: GPT-5.6 in ChatGPT — current page accessed 2026-08-06
• Anthropic, “Claude Fable 5 and Claude Mythos 5” — 2026-06-09
• Anthropic, “Statement on the U.S. government directive” — 2026-06-12
• Anthropic, “Redeploying Fable 5” — 2026-06-30; update July 1
• Claude current model overview — current documentation accessed 2026-08-06
• Claude model IDs and versioning — current documentation accessed 2026-08-06
• Claude Opus 5 documentation — current page accessed 2026-08-06
• Claude Platform release notes — Opus 5 entry dated 2026-07-24
• SpaceXAI/xAI, “Introducing Grok 4.5” — 2026-07-16
• Grok 4.5 API documentation — updated 2026-07-17
• Grok consumer documentation — updated 2026-06-29
• Grok API release notes — July 8 and July 17 entries
• Z.ai, “GLM-5.2: Built for Long-Horizon Tasks” — 2026-06-16
• DeepSeek, “DeepSeek V4 Preview Release” — 2026-04-24
• DeepSeek API change log — 2026-04-24 and July 24 routing/deprecation note
• Moonshot/Kimi, “Kimi K3: Open Frontier Intelligence” — official page; historical publication date not exposed in the page
• Kimi K3 technical report — 2026-07-27

Official weights and licenses

• GLM-5.2 Hugging Face repository — current page accessed 2026-08-06
• DeepSeek V4 Pro Hugging Face repository — current page accessed 2026-08-06
• Kimi K3 Hugging Face repository — current page accessed 2026-08-06
• Kimi K3 GitHub repository and license — current page accessed 2026-08-06
• Kimi K3 license text

Status and incident histories

• OpenAI status history
• OpenAI model-selection incident — 2026-07-09
• OpenAI Codex Sol overload incident — 2026-07-17
• Claude Status — current history accessed 2026-08-06
• xAI Status — current page accessed 2026-08-06

Independent and attributable public comparisons

• BenchLM display-only mirror of the Artificial Analysis Intelligence Index — data verified 2026-08-05
• BenchLM display-only mirror of AA AutomationBench — data verified 2026-08-05
• Artificial Analysis, “Opus 5” — 2026-07-24
• Artificial Analysis, “Kimi K3: second only to Fable 5 on AA-Briefcase” — 2026-07-21
• Artificial Analysis, Grok 4.5 model page — July 2026 page
• Vellum Open Source LLM Leaderboard — updated 2026-07-24
• Kilo, “Kimi K3 Just Took the #1 Spot for Frontend” — 2026-07-21