CLI documentary diagnostic · 7 October 2026 · saved release rehearsal · 1 October 2026

Test and debug a plugin

Installed/enabled but missing a skill or tool? Inspect the CLI boundary below. Offline package checks, supported-host behavior and public review still need separate proof.

Installed/enabled, but the skill or MCP tool is missing

Codex CLI · documented checks 7 October 2026 · no host run

For a plugin from a configured marketplace, inspect source/ref → installed/enabled → selection/session capability → actual output. This is our diagnostic derived from official documentation, not an official universal troubleshooter or tested fix. These are future reader checks; none ran here. Do not transfer them to desktop, web, Cloud or every CLI version.

1. Preserve the original failure.

   Before toggling, updating, reinstalling or opening another session, save client/version, project/current directory, redacted account/workspace, plugin/marketplace identity, expected skill/tool and exact error or absence. An enable-toggle failure differs from an enabled package with a missing capability. Keep the original state even if another session behaves differently.

2. Inspect configured sources and installed state.

   From the failing project, these documented shell commands inspect marketplace and plugin state:

   codex plugin marketplace list --json
   codex plugin list --json

   Marketplace JSON has a marketplaces array with name, root and optional marketplaceSource. Plugin JSON has installed and available arrays; entries include pluginId, name, marketplaceName, version, installed, enabled, source, installPolicy and authPolicy, with marketplaceSource when available. installedPath is documented for add output, not list output. Match the intended entry/source and any recorded Git ref/SHA; missing evidence stays UNKNOWN. Available is not installed. Source/root and installed labels do not prove loaded bytes, authorization or successful work. If a handle is unsupported, retain its error/version. Developer commands · codex plugin / codex plugin marketplace.

3. Check only the capability you need.

   After preserving the failure, meet the documented prerequisite of a new CLI session after installation, in the same project. It is not a guaranteed repair; /clear starts a fresh chat in the same CLI session. Plugins · Overview / permissions; Developer commands · /clear.

   Skill: enter /skills and select the expected skill, or type $ to mention it. Record observed selection/context: selection inserts skill context for the next request. A fluent answer or ordinary text naming it is insufficient. Large skill sets may omit entries from the initial model-facing list; that note establishes neither uninstallation nor a picker limit. allow_implicit_invocation: false disables implicit matching, not explicit invocation. These distinctions are not diagnoses; local standalone-skill controls are not plugin-state workarounds. Developer commands · /skills; Build skills · initial-list note / invocation / Optional metadata.

   MCP tool: enter /mcp; use /mcp verbose for server diagnostics. Compare the exact expected server/tool with tools callable in this session, not merely configured servers. Separate setup/authentication, host policy and intended provider identity/access still matter. Presence is not task success. Developer commands · /mcp; Plugins · permissions.

4. Fix an observed boundary—or stop.

   Resolve only an evidenced mismatch through supported owning controls. Local-marketplace project settings and workspace-import enabled state have different owners; do not overwrite configuration or bypass trust/policy. No universal CLI cache path is established here. Packaging · Enable or disable a plugin for a repo / How local marketplaces work.

   Still absent? Give the responsible maintainer/admin the redacted versioned failure and inspection evidence, without guessing a cure. Present but failing? Retain actual authorized invocation/output/error separately and compare with the useful task. Reuse the blank host record or next-evidence worksheet; do not publish raw diagnostics/secrets. Loaded-byte provenance remains UNKNOWN without independent evidence; our ten actual-host cases remain NOT RUN.

Submission metadata and icons. Checked 1 Oct 2026.

Build skills, Submission error reference. Checked 1 Oct 2026.

Run the offline release-rehearsal kit

Download the 19-file authoring kit

Python 3 standard library only: no dependencies, login, API key, network or model call. Extract the ZIP, then run from its release-rehearsal/ root:

cd release-rehearsal
python3 rehearse.py --source source --runtime-zip runtime/meeting-evidence-0.1.0.zip
python3 selftest.py

The download contains the checker, self-tests, unchanged source/runtime ZIP, synthetic host fixtures, ten manual specifications, rubric, blank record, provenance and final engineering evidence. Do not submit this whole authoring kit as a plugin runtime ZIP.

Actual final observations, Python 3.11.2: 32 subprocess commands plus one deterministic invariant—all 33 TOOLING assertions passed. Six positive commands, 19 broken/out-of-scope fixtures, one nonexistent-path I/O check and six CLI/output guards. A negative assertion passed only when the exact expected diagnostic set and exit matched; a failing checker command is not a failed plugin run.

Read exact commands, stdout/stderr and exits · Inspect the machine-readable final summary. These local observations—not the documentation—support the result count. The independent Runner download reproduction verified the 19-file allowlist and 18 provenance hashes and reran both documented commands successfully. This is not a Runtime delivery receipt.

A finite profile—not a universal plugin validator

Profile meeting-evidence-two-file-skills-only-v1 allowlists only plugin.json and skills/meeting-evidence/SKILL.md. It checks strict UTF-8/JSON, the four-field local manifest profile, schema name rule, fixed sample identity, numeric release version, nonempty description, plain one-line skill front matter and nonempty instructions.

The external schema requires only $schema and name; our required version/description, exact identity and numeric version are narrower profile choices. We do not implement every schema rule or arbitrary YAML. Optional metadata, extensions, icons/assets, other skills, resources or connected/compatibility layouts need another profile. Recognized layouts receive OUT_OF_SCOPE, not a claim that every such plugin is invalid.

Agent Plugins 1.0.0 schema, Accepted layouts and components. Checked 1 Oct 2026.

Source only

Checks selected authoring files. Does not establish the bytes of any ZIP or installed copy. Root README/tests can remain authoring aids; they are ignored and never shipped.

Runtime ZIP only

Checks the allowlisted package members and their CRC reads. Does not establish equivalence to an external source folder.

Source + runtime ZIP

Compares each runtime member byte-for-byte with the explicit source. It still does not show which cached version a host loaded or what a model will do.

Archive reads never extract members. Strict paths, selected-symlink rejection, entry collision checks, unexpected-artifact rejection and 1 MiB file / 4 MiB archive / 16-entry limits are our conservative packaging policy. This is not a secret detector, safety scanner or portal emulator. Extra resources inside the skill are out of scope, not silently omitted.

0 · scoped pass

Checked inputs passed this finite profile.

1 · checked error

A profile or packaging check failed.

2 · CLI / I/O

Arguments, paths or output safeguards failed.

3 · out of scope

A recognized layout/profile needs different checks.

With multiple conditions, I/O takes priority, then checked errors, then out-of-scope. Read all diagnostics. Optional --json ../fresh-evidence.json records inputs, profile, version, hashes, diagnostics and pending gates; outputs must be new and outside source, with existing parent directories.

An observed failure → allowlisted repair

Failure: the synthetic unexpected_artifact archive contained both correct runtime files plus an authoring README.md. Our checker returned exit 1 · ZIP_UNEXPECTED [README.md]. This is our diagnostic, not an observed OpenAI rejection.

ERROR ZIP_UNEXPECTED [README.md]
# actual exit: 1

Repair: keep authoring aids outside the runtime ZIP. Replace recursive “zip everything” behavior with the checker’s explicit allowlist; from the extracted kit:

python3 rehearse.py --source source --write-zip ../repaired-runtime.zip
python3 rehearse.py --source source --runtime-zip ../repaired-runtime.zip

Actual repair_allowlisted_packaging and repaired_source_zip commands returned 0; the archive contained exactly the original two runtime payloads. Inspect the failure and repaired records. No source/version change or host test was required for this packaging repair.

mkdir -p ../scratch
python3 - <<'PY'
from zipfile import ZipFile
with ZipFile('runtime/meeting-evidence-0.1.0.zip') as original, ZipFile('../scratch/accidental.zip', 'w') as bad:
    for item in original.infolist():
        bad.writestr(item.filename, original.read(item.filename))
    bad.writestr('README.md', 'Authoring aid, not runtime')
PY
python3 rehearse.py --source source --runtime-zip ../scratch/accidental.zip
# Expected ZIP_UNEXPECTED, exit 1
python3 rehearse.py --source source --write-zip ../scratch/repaired.zip
python3 rehearse.py --source source --runtime-zip ../scratch/repaired.zip
# Expected exit 0 for both repair commands

Two newly written stored ZIPs were byte-identical to each other. They have the original payloads but a different compression encoding from the unchanged old ZIP. Determinism is an engineering property, not behavior or safety proof.

Ten separate host specifications—all actuals NOT RUN

Before any future run, establish lawful client/account/workspace access and an actually accepted package. Preserve this teaching source; record any adapted release’s version, bytes and compatibility. Install from a trusted separate repository, merge rather than replace the local catalog, record restart/new-session steps and confirm the loaded version. Source edits alone do not update an installed cached copy.

Host metadata compatibility, Local source and cached installation, Supported host procedure. Checked 1 Oct 2026.

Our original manual cases use synthetic input and exact prompts. Expand a case to see its contract, then download all specifications or use the fixtures in the kit. No synthetic expected response is presented as model output.

H09’s applicability is not established. Invalid-UTF-8 binary data does not prove any host could not read an attachment. Record the actual attach/read outcome; if a lawful replacement is needed, document it. Missing access or media must not become a fabricated model failure.

Record selection separately from output quality

Download the blank per-case evidence record · Download the manual source-to-output rubric

Selection

Record picker/command and reliable invocation evidence for the loaded release. User text naming the skill or fluent formatted prose is not enough. If a host exposes no reliable activation trace, use UNKNOWN, even when the answer is good.

Quality

Compare every decision/action with its cited passage. Preserve wording strength, unknown owners/deadlines, proposals versus commitments, contradictions, relative-date anchors and the explicit requested format.

Boundary

Inspect actual outside-action trace where available. Transcript code/URL/send requests are data, not authorization. No connector in this source is not proof of host retention policy or other enabled tools’ safety.

Retest

Save the failure, changed source/version and independent fresh-session result. Never calculate a model success rate from ten blank cases or 33 checker assertions.

The blank record covers UTC date, client/version, surface/account/workspace prerequisites (redacted), source/version/hashes and loaded-copy evidence, catalog merge/install/restart steps, enabled tools/permissions, fixture/prompt, observed selection, actual output, per-assertion source comparison and repair/retest. Keep credentials and sensitive meeting content out of saved evidence.

Activation and quality guidance, Security engineering principles. Checked 1 Oct 2026.

Design before a connected-action observation · 4 October: inspect the proposed human approval boundary and its evidence prerequisites. Entire scenario/three contracts are hypothetical / NOT RUN; three implementation traps are lessons, not additional tests. No prior case, 33-checker count or 49-recorder count changed.

Debug the actual boundary

Package rejected / profile out of scope

Record the actual host or checker finding. Icons, extensions and connected layouts may require an appropriate broader validator; do not rename manifests or treat exit 3 as universal invalidity.

Wrong or stale loaded copy

Record source/version/hash, installation source and fresh-session evidence. An authoring folder is not the installed cache.

Missing input or provider access

Ask for genuinely absent notes. In a connected workflow, verify the right identity, scopes and entitlement; installation does not supply them.

Unsupported outside action

This source can draft, not independently fetch/send. Record attempted calls as well as output text; other enabled tools retain separate consent boundaries.

Faithfulness failure

Save the actual incorrect assertion and passage comparison. Repair instructions or fixtures, then repeat independently—do not invent a representative model answer.

Optional helper evidence · 5 October: loaded configuration, resources, qualifying orchestration, trust/enable state, event/context and actual policy use need separate future observations. Every field remains UNKNOWN / NOT RUN; this is no new suite or addition to the ten retained host cases.

Keep the September check as historical evidence

The unchanged original validate.py, 29 September static log and original authoring download remain available. That earlier narrow check is not the new checker and never was a host result. Inspect the unchanged 0.1.0 runtime source.

Next: engineer the trust boundary → · Separate private QA from public reviewer evidence · Read the dated research

Updating rather than first packaging? Create a before/after change record before the future host regression. The 2 October recorder is not this 1 October checker or a completed host update; all ten actual host records remain NOT RUN.