Shaduf.Research preview
AI Model Degradation Watch/Opus 5.5 rebounded on one benchmark; a Windows Codex user still reports late failures
Checked 17 October 2026 · Public-source review

Opus 5.5 rebounded on one benchmark. A Windows Codex user still reports late failures.

Current call: broad AI-model decline is not established. BridgeBench reports Claude Opus 5.5 at 105.8% of its first tracked session on 16 October, up from 95.7% on 9 October and inside its stated 90–110% band. A separate issue describes an October 1 Opus 5.5 behavior shift in one user's sessions. OpenAI marked its GPT-6.1 Sol Codex/Work incident resolved on 13 October; one Windows user now reports additional stream and remote-compaction failures on 16–17 October. These are different tests, tasks and failure lanes.

Use the right test for the symptom. If no answer arrives, check status and capture the model, surface, client, time and error; keep the original work intact. If an answer completes but seems worse, replay the same task with the same surface and settings. Score correctness, correction work, time and allowance separately.

What changed in the evidence

Evidence laneWhat is reportedWhat it establishes
BridgeBench NerfBench, 16 OctOpus 5.5 is 105.8% of the first tracked session; the 9 Oct point was 95.7%. The 16 Oct result is 10.1 points above the prior test and 5.8 points above the reference.A publisher-run task suite did not show a sustained drop outside its own band. It is not a general Opus result.
Opus 5.5 issue #98679, 1 Oct comparisonOne author reports median thinking tokens per request rising from 67 to 126, median output tokens from 447 to 711, and worse task judgment. The comparison is 29 sessions across 23–30 Sep versus 8 sessions on 1 Oct.A detailed user signal worth checking. The first reported shift coincides with a Claude Code update from 2.1.285 to 2.1.286, the tasks are not fixed, and the reported numbers do not show a population effect.
OpenAI Status, 13 OctOpenAI marked the GPT-6.1 Sol Codex/Work degraded-performance event resolved at 16:36 UTC.The provider's aggregate status state; it does not establish an individual user's recovery.
Codex issue #53648, 16–17 Oct follow-upOne Windows user describes long-running stream/capacity failures. A follow-up lists two waits of 86 and 33 minutes without a saved assistant reply or tool action, followed by a 95-minute remote-compaction failure with HTTP 408/systemError.A serious account-level reliability report after the status closure. It is not a completed-answer quality test, and a shared cause is unknown.

How much weight should NerfBench carry?

BridgeBench says each session has 50 private coding and reasoning tasks, each run three times, for 150 attempts. The first tracked session becomes the frozen 100% reference. Its power score combines correctness at 60%, output tokens at 20%, and cost at 20%. If cost is missing, the remaining weights are reallocated; Codex is scored 75% correctness and 25% tokens. This is a measure of how a served system performs and what it costs on one suite, not a pure answer-quality score.

The latest Opus 5.5 result is within BridgeBench's own 90–110% band. The publisher says it chose that band because sampling variation can reach about nine points for a model passing 70% of attempts; it also requires two comparable consecutive sessions outside the band before publishing a confirmed change. That is the publisher's rule, not a universal threshold for every model or task mix. The board lists seven Opus 5.5 sessions since its first tracked session, and keeps the tasks/tests private.

The methodology says OpenRouter uses default routing without pinning the upstream provider, and that a changed serving configuration can move the measurement. The benchmark can flag a change in the model as served; it cannot assign that change to model weights alone. BridgeBench's current score argues against a sustained decline on its suite. It cannot cancel a user-specific report on a different surface.

Do not call a request failure a wrong answer. The newest Codex follow-up describes interruptions and a remote-compaction error. It does not show that a completed GPT-6.1 Sol answer failed a fixed quality test.

Next move

  1. If the task returns an error, check the current status and record product surface, client/version, selected model, effort, time and exact error. Preserve the original task and project changes.
  2. If a completed answer seems worse, use a safe saved task and repeat it under the same model label, surface, effort, tools and context. Record the answer, correction time and accepted result; keep speed and quota as separate measures.
  3. Do not infer a provider-wide change from one account or a composite board. Look for a dated before/after comparison with matching tasks and serving details.

Sources

Return to the current verdict · Read how evidence is assessed

Search published pools, pages, reports, and evidence.