Shaduf AI Model Degradation Watch

Cutoff: 2026-08-03 23:59 UTC Scope: Public internet sources only. No Shaduf evaluations, prompts, canaries, benchmarks, or experiments were run.

Current-summary article

The admissible public record does not establish generalized degradation of the underlying model weights across OpenAI, Anthropic, xAI, or the selected open-weight models.

What it does establish is rapid change in the surrounding systems: new model releases, altered pricing, product-specific routing, safety classifiers, fallback models, context/tool changes, and service outages. Those factors can change a user’s observed experience without proving that the base model became less capable.

OpenAI’s GPT-5.6 family became generally available on July 9 as three separately named models: GPT-5.6 Sol, GPT-5.6 Terra, and GPT-5.6 Luna. Their API identifiers are separately documented as gpt-5.6-sol, gpt-5.6-terra, and gpt-5.6-luna. OpenAI explicitly discusses API, ChatGPT, and Codex as distinct surfaces. Its July 30 price-performance update changed Terra and Luna pricing and described improvements spanning models, inference systems, routing, and context management—not only model weights. OpenAI preview, June 26, OpenAI launch, July 9, OpenAI price update, July 30.

OpenAI’s official status record documents a Codex Sol overload incident on July 17, a broader ChatGPT/Codex incident on July 19, and elevated API, ChatGPT, and Codex error rates on July 23–24. These are availability and infrastructure events. OpenAI’s status page itself warns that aggregate availability does not represent every model, subscription, or feature equally. None is evidence that GPT-5.6 Sol, Terra, or Luna’s underlying weights degraded. Codex Sol incident, July 17, July 19 write-up, July 23–24 incident, OpenAI status history.

Anthropic exposes an important counterexample to treating access tiers as independent models. Claude Fable 5 and Claude Mythos 5 share the same underlying model, according to Anthropic, but differ in safeguards and access. Fable adds stronger safety classifiers and may fall back to Claude Opus 4.8 for certain flagged requests; Mythos is a restricted Project Glasswing tier with fewer safeguards. A change in fallback frequency or classifier behavior can therefore look like a model-quality change while the underlying weights remain unchanged. Anthropic launch, June 9, Anthropic safeguards, July 2.

The third monitored Claude row is Claude Opus 5, released July 24 with the API identifier claude-opus-5. Anthropic’s current model documentation positions it as the model for complex agentic coding and enterprise work, below Fable in capability but above Sonnet in the current lineup. Sonnet 5 is current and appears in the August 3 status incident, but it is not included in the three-row Claude monitoring allocation. Anthropic model overview, Anthropic release notes, July 24 entry.

xAI’s current leading model is Grok 4.5. The API identifier is grok-4.5, but the public record does not establish a fixed model ID or snapshot for Grok in X, the consumer Grok website/apps, or the Grok Build coding product. xAI documents Grok 4.5 as the default model in Grok Build, while the consumer and X surfaces remain product-level access paths. This distinction prevents an API result from being silently treated as a result for every Grok surface. xAI launch, July 16, xAI Grok 4.5 API documentation, xAI model catalog.

The three open-weight selections are Kimi K3, GLM-5.2, and DeepSeek V4 Pro. All have public downloadable weights and an identified license. Kimi K3 is open-weight under a custom Kimi K3 license rather than an unqualified OSI open-source license. GLM-5.2 and DeepSeek V4 Pro are released under MIT. The selection is based on current relevance and fresh independent public comparisons available before the cutoff; it is not a claim that these are universally the three best open models. Kimi K3 release record, July 27, GLM-5.2 model card, DeepSeek V4 announcement, April 24.

The strongest independent comparison evidence comes from Artificial Analysis, but its scores are snapshots of complete systems under specified effort, fallback, tool, and harness configurations. A July 24 snapshot placed Opus 5 at 61, Fable 5 at 60, GPT-5.6 Sol at 59, Kimi K3 at 57, and Opus 4.8 at 56 on its Intelligence Index. These figures are comparable within that article, but they are not pure measurements of base weights: Fable and Opus 5 used fallback configurations. Artificial Analysis, July 24.

Likewise, DeepSeek V4 Pro was reported at 52 in an April 24 Artificial Analysis article, while a later public Artificial Analysis page showed 44. Without a frozen methodology and raw historical record, that difference is not proof of degradation. It may reflect endpoint, configuration, benchmark, or index changes. DeepSeek V4 comparison, April 24, current DeepSeek V4 Pro page.

Monitored model comparison

The table contains exactly ten monitored rows. Fable 5 and Mythos 5 are deliberately separate access-tier rows, but they are not claimed to be independent underlying base models.

Dated change and incident timeline

Comparable public charts

These charts use only dated public data from comparable sources. They are not Shaduf measurements.

Chart 1 — Artificial Analysis Intelligence Index snapshot

Publication date: July 24, 2026 Scale: 0–65; higher is better. Important configuration note: Fable 5 and Opus 5 include fallback configurations reported by Artificial Analysis.

Claude Opus 5       61 |███████████████████████████████
Claude Fable 5      60 |██████████████████████████████
GPT-5.6 Sol         59 |█████████████████████████████▌
Kimi K3             57 |████████████████████████████▌
Claude Opus 4.8     56 |████████████████████████████

Source: Artificial Analysis, “Opus 5” — July 24, 2026.

Underlying CSV:

model,provider,metric,score,effort_or_configuration,publication_date,source_url
Claude Opus 5,Anthropic,Artificial Analysis Intelligence Index,61,"max; Opus 4.8 fallback enabled",2026-07-24,https://artificialanalysis.ai/articles/opus-5
Claude Fable 5,Anthropic,Artificial Analysis Intelligence Index,60,"max; Opus 4.8 fallback enabled",2026-07-24,https://artificialanalysis.ai/articles/opus-5
GPT-5.6 Sol,OpenAI,Artificial Analysis Intelligence Index,59,"max",2026-07-24,https://artificialanalysis.ai/articles/opus-5
Kimi K3,Moonshot,Artificial Analysis Intelligence Index,57,"as reported by Artificial Analysis",2026-07-24,https://artificialanalysis.ai/articles/opus-5
Claude Opus 4.8,Anthropic,Artificial Analysis Intelligence Index,56,"max",2026-07-24,https://artificialanalysis.ai/articles/opus-5

Chart 2 — GPT-5.6 tier comparison

Publication date: July 9, 2026 Two separate scales: Intelligence Index is higher-is-better; cost per task is lower-is-better.

Artificial Analysis Intelligence Index — higher is better

GPT-5.6 Sol       59 |█████████████████████████████▌
GPT-5.6 Terra     55 |███████████████████████████▌
GPT-5.6 Luna      51 |████████████████████████████▌


Cost per Intelligence Index task, USD — lower is better

GPT-5.6 Sol      1.04 |████████████████████
GPT-5.6 Terra    0.55 |███████████
GPT-5.6 Luna     0.21 |████

Artificial Analysis states that it supported OpenAI’s prerelease evaluation for this launch. These are external evaluator results, not Shaduf tests. Artificial Analysis, July 9, 2026.

Underlying CSV:

model,metric,value,unit,configuration,publication_date,source_url
GPT-5.6 Sol,Artificial Analysis Intelligence Index,59,index,max,2026-07-09,https://artificialanalysis.ai/articles/gpt-5-6-has-landed
GPT-5.6 Sol,Cost per Artificial Analysis Intelligence Index task,1.04,USD,max,2026-07-09,https://artificialanalysis.ai/articles/gpt-5-6-has-landed
GPT-5.6 Terra,Artificial Analysis Intelligence Index,55,index,max,2026-07-09,https://artificialanalysis.ai/articles/gpt-5-6-has-landed
GPT-5.6 Terra,Cost per Artificial Analysis Intelligence Index task,0.55,USD,max,2026-07-09,https://artificialanalysis.ai/articles/gpt-5-6-has-landed
GPT-5.6 Luna,Artificial Analysis Intelligence Index,51,index,max,2026-07-09,https://artificialanalysis.ai/articles/gpt-5-6-has-landed
GPT-5.6 Luna,Cost per Artificial Analysis Intelligence Index task,0.21,USD,max,2026-07-09,https://artificialanalysis.ai/articles/gpt-5-6-has-landed

Chart 3 — Public API list prices at the cutoff

Units: USD per 1M input/output tokens. Scope: API list pricing only; excludes consumer subscriptions, cached-input discounts, provider routing, tool costs, and local open-weight deployment.

Input price — scale: █ ≈ $1

Claude Fable 5       $10.00 |██████████
Claude Mythos 5      $10.00 |██████████
GPT-5.6 Sol           $5.00 |█████
Claude Opus 5        $ 5.00 |█████
GPT-5.6 Terra         $2.00 |██
Grok 4.5              $2.00 |██
GPT-5.6 Luna          $0.20 |▏


Output price — scale: █ ≈ $5

Claude Fable 5       $50.00 |██████████
Claude Mythos 5      $50.00 |██████████
Claude Opus 5        $25.00 |█████
GPT-5.6 Sol          $30.00 |██████
GPT-5.6 Terra         $12.00 |██▍
Grok 4.5               $6.00 |█▏
GPT-5.6 Luna           $1.20 |▏

Sources: OpenAI pricing update, July 30, Anthropic Fable/Mythos launch, June 9, Anthropic Opus 5 release notes, July 24, xAI API release notes, July 8.

Underlying CSV:

surface_model,provider,input_usd_per_1m,output_usd_per_1m,pricing_date,surface,source_url
GPT-5.6 Sol,OpenAI,5,30,2026-07-30,API,https://openai.com/index/advancing-the-price-performance-frontier-with-gpt-5-6/
GPT-5.6 Terra,OpenAI,2,12,2026-07-30,API,https://openai.com/index/advancing-the-price-performance-frontier-with-gpt-5-6/
GPT-5.6 Luna,OpenAI,0.20,1.20,2026-07-30,API,https://openai.com/index/advancing-the-price-performance-frontier-with-gpt-5-6/
Claude Fable 5,Anthropic,10,50,2026-06-09,API,https://www.anthropic.com/news/claude-fable-5-mythos-5?type=company
Claude Mythos 5,Anthropic,10,50,2026-06-09,Project Glasswing API/access tier,https://www.anthropic.com/news/claude-fable-5-mythos-5?type=company
Claude Opus 5,Anthropic,5,25,2026-07-24,API,https://platform.claude.com/docs/en/release-notes/overview
Grok 4.5,xAI,2,6,2026-07-08,API,https://docs.x.ai/developers/release-notes

What is established

• GPT-5.6 Sol, Terra, and Luna are separately named models with separately documented OpenAI API identifiers.
• OpenAI API, ChatGPT, and Codex are distinct surfaces. Service incidents affecting one or more surfaces do not establish base-model degradation.
• Fable 5 and Mythos 5 share underlying weights but intentionally differ in safeguards, fallback behavior, and access.
• Claude Opus 5 is a separate current Claude model with identifier claude-opus-5.
• Grok 4.5 is the current documented leading xAI model, with API identifier grok-4.5; the API, Grok Build, consumer Grok, and Grok in X are not proven to be identical deployments.
• Kimi K3, GLM-5.2, and DeepSeek V4 Pro satisfy the open-weight selection requirement through public downloadable weights and identified licenses.
• GLM-5.2 and DeepSeek V4 Pro are MIT-licensed according to their official model records. Kimi K3 uses a custom license with additional commercial conditions.
• Provider-documented availability, routing, safeguard, and evaluation-harness incidents occurred before the cutoff.
• Artificial Analysis published dated cross-model comparisons before the cutoff.
• Fable’s and Opus 5’s published comparison results include fallback/configuration effects; they are not clean measurements of isolated base weights.
• DeepSeek’s April 24 score and later public score differ, but the evidence does not establish that the difference is degradation.

What is not established

• No public evidence in this run proves generalized degradation across the monitored set.
• No frozen longitudinal dataset compares all ten monitored rows on the same tasks, same surfaces, same settings, and same model snapshots.
• No base-weight quality decline is established for GPT-5.6 Sol, Terra, or Luna.
• No base-weight quality decline is established for Fable/Mythos, Opus 5, or Grok 4.5.
• No public evidence establishes that Grok in X, Grok consumer, and Grok Build use the exact grok-4.5 API snapshot.
• No ranking of all ten monitored rows is justified from the available data.
• OpenAI and Anthropic cybersecurity-evaluation disclosures do not establish ordinary production model degradation.
• Outages, elevated error rates, fallback events, or routing changes cannot be converted into quality claims without comparable output evidence.
• Community complaints are not treated as proof because they generally do not identify the exact model snapshot, surface, routing path, system instructions, or fallback state.

Uncertainty

The major uncertainty is not simply statistical noise. It is identity and configuration uncertainty.

A public result may depend on:

• the underlying model snapshot;
• provider routing or alias resolution;
• system instructions;
• safety classifiers and refusal policies;
• fallback models;
• effort or reasoning settings;
• context length and truncation;
• tool availability and tool harness;
• rate limits, overload, or regional capacity;
• API versus consumer-product deployment.

Artificial Analysis’ Fable 5 results illustrate this directly: Fable can fall back to Opus 4.8, so a benchmark result can measure a composite system. The same applies to Opus 5 when fallback is enabled.

Mutable documentation also limits historical reconstruction. Current model pages and model repositories were used only where they expose stable identifiers, licenses, or dated release records. They were not treated as proof that every current field or file-tree state existed unchanged at the cutoff.

The DeepSeek V4 Pro score difference—52 in the April 24 launch comparison versus 44 on a later public page—is a useful warning. Without a frozen methodology, endpoint, configuration, and raw data, it is impossible to distinguish real performance change from measurement drift.

The exact timezone of the displayed timestamps on the OpenAI status pages was not established from the page itself. The Anthropic August 3 incident provides explicit UTC timestamps through its status API.

Freshness

The cutoff rule used here is:

1. A claim is admitted when the source was published, updated, or recorded no later than 2026-08-03 23:59 UTC.
2. A current page viewed after the cutoff is not used as a historical snapshot unless it exposes a dated release or incident record whose historical state is clear.
3. Post-cutoff status entries visible in current provider histories are excluded.
4. Current mutable pages are labeled as current or non-frozen where relevant.
5. No undocumented model ID, incident, ranking, measurement, or date is inferred.
6. All quality numbers are attributed to external provider or evaluator publications; Shaduf did not run tests.

Sources

OpenAI

• GPT-5.6 preview — June 26, 2026
• GPT-5.6 general availability — July 9, 2026
• GPT-5.6 price-performance update — July 30, 2026
• OpenAI API model catalog — current mutable documentation
• GPT-5.6 Sol API page
• GPT-5.6 Terra API page
• GPT-5.6 Luna API page
• OpenAI status history
• Codex Sol overload incident — July 17, 2026
• ChatGPT/Codex infrastructure incident write-up — July 19, 2026
• Elevated error rates — July 23–24, 2026
• Hugging Face model-evaluation security incident — July 21, 2026

Anthropic

• Claude Fable 5 and Mythos 5 — June 9, 2026
• Redeploying Fable 5 — June 30 / July 1, 2026
• Fable safeguards and jailbreak framework — July 2, 2026
• Anthropic model overview — current mutable documentation
• Anthropic release notes
• Anthropic status incident API
• Cybersecurity-evaluation incident disclosure — July 30, updated August 3, 2026

xAI

• Grok 4.5 launch — July 16, 2026
• Grok 4.5 API documentation — updated July 17, 2026
• xAI model catalog
• xAI API release notes
• Grok consumer overview
• Grok consumer FAQ
• xAI/Grok service status
• Grok Web status history
• Grok Build changelog

Open-weight model records

• Kimi K3 official GitHub
• Kimi K3 model card and weights
• Kimi K3 license
• Kimi K3 technical report — July 27, 2026
• GLM-5.2 official release — June 16, 2026
• GLM-5.2 model card
• GLM-5.2 downloadable weights
• DeepSeek V4 launch — April 24, 2026
• DeepSeek API updates
• DeepSeek transparency page
• DeepSeek V4 Pro model card
• DeepSeek V4 Pro downloadable weights
• DeepSeek V4 technical report

Independent comparisons

• Artificial Analysis methodology
• GPT-5.6 Sol/Terra/Luna — July 9, 2026
• Claude Fable 5/Mythos 5 — June 9, 2026
• Grok 4.5 — July 8, 2026
• Kimi K3 — July 17, 2026
• GLM-5.2 — June 16, 2026
• DeepSeek V4 Pro/V4 Flash — April 24, 2026
• Opus 5 cross-model comparison — July 24, 2026