Jev (TypeSafe AI) integration trust: Claude Code and MCP guardrails, TypeScript paths and testing without a key

Run: run:bd3c2f65-2199-4b1b-83d5-3a1135b103b0 (scheduled regular research run 2) · Pool: pool_jev_catalog · Runner: single agent · Checks: 2026-09-29, 07:05–07:17 UTC.

The run's full audit notes and its source ledger (76 entries: URL, access time, source date, version or commit, label, excerpt of 25 words or fewer, outcome) are held in the pool's private record. Key sources are linked in section 7 and on the pages each finding feeds.

Spot check by the pool manager (29 Sep): oh-my-claudecode src/hooks/jev/config.ts at 9fd35ec shows DEFAULT_TIMEOUT_MS = 250; danna-zhou/jev-mcp hooks/lib.mjs L46–61 at 436469a shows the fail-open helper exiting 0. Both match the runner's reading.

Previous release: bootstrap-2026-09-28-question-pages, published 2026-09-28T20:45:47Z; report: release 1 question pages.

1. Summary answer, as of 29 September 2026

Which Jev integration should a Claude Code or agent builder install, what does it send, and does it fail open?

We read the failure paths of five integrations in pinned source, and read the official plugin in full. The five implement five different contracts:

So "installing a Jev guardrail" can mean anything from no runtime effect (official skill, shadow defaults) to a guard that halts the agent on any outage (LangChain). The default a user gets is usually not a gate. Two of the five ship in a shadow or log-only mode, one is advisory by design, and one fails open.

Which TypeScript path?

• Official @typesafe-ai/sdk 0.6.0 (15 Sep; Node ≥ 20):
  • TypeSafe direct.
  • Typed helpers choice(), score(), noul().
  • 2 retries on 408/429/5xx, honouring Retry-After; 10 s per attempt.
  • Full error classes, and fetch injection for tests.
  • Nine open issues, including a crash on Node 20/22 when a call is cancelled.
• AI SDK experimental_evaluate (ai ≥ 7.0.103, latest 7.0.122; Node ≥ 22):
  • The path for Vercel AI Gateway ('typesafe-ai/jev', AI_GATEWAY_API_KEY), or direct via @ai-sdk/typesafe-ai 3.0.10 (TYPESAFE_AI_API_KEY — a different variable name).
  • Experimental. Noul is called boolean and returns .probability; confidence sits in providerMetadata.
  • 2 retries by default; no default timeout; Experimental_EvaluationMockModelV4 for tests.
• Raw fetch for any runtime or route, if you write the retry and timeout yourself (T04 shows a tested pattern).
• Cloudflare Workers binding env.AI.run('typesafe/jev', …) on Workers.

How do I test without a key? Replace one layer: the transport (official SDK fetch option or Python transport), the model (AI SDK mock model) or the server (a local mock server). Then run the ten negative branches in the testing page's checklist. Each facility proves only its own layer. Project "no-key modes" (mock models, simulators, shadow defaults) are the main risk of Jev silently being skipped.

What changed since the previous release (28 Sep, 20:45 UTC)

Corrections to published pages

1. Errors page (build/errors-and-rate-limits), oh-my-claudecode timeout.
   • The page says the 250 ms default was "raised to 2000 ms on 22 Sep".
   • PR #4092 was merged on 22 Sep 22:21 UTC into the dev branch.
   • The latest release, v5.5.0, and main still have DEFAULT_TIMEOUT_MS = 250 on 29 Sep (src/hooks/jev/config.ts L27 at 9fd35ec); only dev has 2000 (L36).
   • New wording: "raised to 2000 ms on the project's dev branch (PR #4092, merged 22 Sep); the latest release, v5.5.0, still defaults to 250 ms (checked 29 Sep)." (Documented: config.ts at v5.5.0 and on dev, and PR #4092.)
   • The earlier research note of 28 Sep was accurate about the PR but not about the released code.
2. Products page, jev-mcp row.
   • The row is one of at least four unrelated jev-mcp repositories. It links jkudish/jev-mcp and grades it B (25 Sep).
   • New: name it "jev-mcp (jkudish)", regrade it A (call path read at v0.11.0, 29 Sep), and add "advisory: returns recommendations; the server never blocks".
   • Add separate rows for the other audited integrations (§4.5).
3. Audience claims now checked in source (they were never published):
   • danna-zhou fails open — confirmed.
   • LangChain guard fails closed and sends the last 30 messages — confirmed for AutoModeMiddleware in langchain-typesafe 0.0.1a3.
   • jkudish is advisory only — confirmed at v0.11.0.

No change found in: signup status, new-user credit, status-page incidents, models and aliases, TypeSafe direct price, OpenRouter, Vercel, Cloudflare and Opper entries, or SDK versions (PyPI 0.7.2, npm 0.6.0). These fields only need new check dates (§4.6).

New facts that do not correct a page but belong on one

• LangChain criteria bug: AutoModeMiddleware 0.0.1a3 sends no risk criteria unless the caller passes them. The constructor's criteria=None default replaces the documented default criteria (auto_mode.py L66–71, L147, L164), and a unit test asserts criteria is None. Documented.
• Vercel template verified in source: its "≥ 95% confidence, else OpenAI fallback" is in lib/router.ts L8 and L128–162 at 27ea469, with 12 s / 25 s timeouts and one retry each. The fallback model is openai/gpt-6-luna-fast in source but openai/gpt-5.6-luna-fast on the template page.
• AI SDK first version: experimental_evaluate first appears in ai 7.0.103 (16 Sep, type declarations), but Vercel's guide says "7.0.105 or later". Use 7.0.105 as the stated minimum and note the discrepancy.
• Broken link in the official plugin: its SKILL.md links a migration guide that returns 404 (skills #13, confirmed 29 Sep).
• Official repository not in the catalogue: typesafe-ai/n8n-nodes-typesafe-ai (created 23 Sep, MIT). Not opened.

2. Method, time window and evidence labels

Parts of this run: S1 = preceding-24-hour scan; S2 = recheck of volatile access and price fields; S3 = Claude Code and MCP audit; S4 = TypeScript paths; S5 = testing without a key; T04 = offline test.

Timing ledger (UTC, 29 Sep):

• Start: 07:04:56.
• S1 scan: 07:05–07:08 (cutoff 07:05:00).
• S2 recheck: 07:07–07:09.
• S3 audit: 07:09–07:14 (offline hook run ~07:12).
• S4: 07:14–07:16.
• S5 and T04: 07:16–07:17.
• Write-up: from 07:17.

The whole run stayed inside the 70-minute box, so the stop rule was not triggered.

How sources were read.

• Node v24 fetch, unauthenticated.
• GitHub: one API call per repository to pin the commit and list the tree, then files from raw.githubusercontent.com/<owner>/<repo>/<sha>/<path>.
• The LangChain package was read from its PyPI sdist (sha256 d77647a7…3272); nothing was installed.
• AI SDK version membership was checked by reading published type declarations from jsDelivr.

Labels.

• Tested: the pool ran it, offline, no Jev call.
• Documented: official or route-owner docs, or source at a pinned commit with path and lines.
• Reported: a named third party's own claim, including a README about its own behaviour.
• Unverified: lead only.

No pool Jev call. No request went to api.typesafe.ai or to any gateway inference endpoint. No key, account or purchase.

Offline runs

• Offline hook run, Tested:
  • Ran danna-zhou's PreToolUse hook (148 lines with its helper, standard library only, read in full), with no key, against closed ports on 127.0.0.1.
  • git push origin main: exit 0 and "jev guardrail unreachable, skipping review".
  • rm -rf / --no-preserve-root was not sent at all.
• T04, Tested: a pool-authored Node client mirroring the official JS SDK defaults, run against a local mock server.
  • 429 + Retry-After: 1 then 200 → success on attempt 2 (1.05 s).
  • 529 × 3 → gave up after 3 attempts.
  • 401, HTML 403, 422, malformed JSON, out-of-set choice → immediate failure, no retry.
  • Hang → 3 timeouts in 3.9 s (800 ms per attempt).
• Both ran in temporary directories that were deleted afterwards. Script source and output are published on Claude Code and MCP and Testing without a key.

Analytics: unavailable for this run (the read-only analytics connection was not available). No search volumes are stated.

3. Findings per page and ship verdicts

3.1 build/claude-code-mcp

The page's core is the failure-disposition table (disposition per failure class, data sent, default mode), followed by the pre-install checklist and "how to check an integration yourself".

Key messages:

1. The official plugin is documentation for your agent, not a guard.
2. Check the default mode before assuming protection.
3. What leaves the machine ranges from nothing to the last 30 messages.
4. A regex pre-filter defines what a hook actually checks.
5. A timeout below the round trip turns a guard off (oh-my-claudecode: 250 ms in the released version).

Keep the errors page as the owner of the general fallback procedure, and link to it.

3.2 build/typescript

Contents:

• Comparison table.
• Minimal call per path, with pinned versions.
• The 12-row gotchas table.
• Real-world examples: Jevmail's maxRetries: 0 pattern (carried), and the Vercel template's threshold-and-fallback, now verified in source.

No prices; model IDs link to the route matrix.

3.3 build/testing-without-a-key

Contents:

• Facility table with "proves / cannot prove / silent-bypass risk".
• Negative-path checklist. It links to the errors page for code meanings and does not duplicate the code table.
• T04 as a worked example.

Link "low confidence" to build/confidence-thresholds and "no key yet?" to where-to-use/access-status.

3.4 build hub

Selection table in §4.4. Code interfaces only; no routes or prices.

4. Page contents, labels and check dates

Every field below carries its label and check date. "29 Sep" means 2026-09-29, 07:09–07:17 UTC unless stated.

4.1 build/claude-code-mcp — "Jev (TypeSafe AI) in Claude Code and MCP: which server, hook or plugin fails open?"

Direct answer (for the top of the page): Of five Jev guardrails we read in source, only one stops the agent when Jev fails (LangChain AutoModeMiddleware). Two ship in a shadow or log-only mode, one is advisory by design, one fails open, and the official TypeSafe plugin makes no Jev calls at all. Check the default mode, what the integration sends, and its timeout before relying on it. — Documented (source at pinned commits), 29 Sep.

Comparison table: the failure matrix on the page.

Required columns:

• integration (owner, pinned version or commit, date);
• official or community;
• type;
• sends to Jev;
• (a) no key; (b) timeout with default; (c) HTTP error; (d) malformed; (e) low confidence;
• default mode;
• evidence grade.

Row labels and dates:

Mandatory caveats on the page

• "Normal permission flow" means Claude Code's own rules apply; it is not an approval. — Documented, 29 Sep.
• LangChain 0.0.1a3 sends no risk criteria unless you pass criteria=NoulCriteria(...). — Documented, 29 Sep.
• danna-zhou: set JEV_MODEL=jev-1.13.0 when pointing at TypeSafe; the default kev-latest is for a local Kev server. Whether TypeSafe rejects kev-latest was not tested. Earlier Reported evidence says unknown models return 400, which this hook would treat as "skip review". — Documented (defaults) / inference, 29 Sep.
• oh-my-claudecode: v5.5.0 defaults to 250 ms, while the maintainers measured 465–605 ms round trips (Reported, #4091, 22 Sep). Set OMC_JEV_TIMEOUT_MS above your round trip. — Documented, 29 Sep.
• jev-claude's tests run in enforce mode; the shipped default is shadow. — Documented, 29 Sep.
• No integration was run against the live API. Star counts are not adoption.

Checklist and "check it yourself": Documented method; no live claims.

"Seen, not audited" box: Reported one-liners only, each dated 29 Sep; no dispositions for these rows. Flag codaaiteam's links to jevtypesafeai.com as "lookalike reseller storefront (Eye Security, 25 Sep)".

Links:

• Hub: build.
• Related: products (graded rows); use-cases (agent-guardrail row); videos (Claude Code group); build/errors-and-rate-limits (general fallback procedure and the oh-my-claudecode note); build/testing-without-a-key.
• Excluded by plan: routing and compaction plugins (effort-router, Jevgrep, fast-jev-compaction; run 4).

4.2 build/typescript — "Call Jev (TypeSafe AI) from TypeScript: AI SDK experimental_evaluate vs @typesafe-ai/sdk vs fetch"

Direct answer: Use the official @typesafe-ai/sdk 0.6.0 for TypeSafe direct on Node 20+. Use AI SDK experimental_evaluate (ai 7.0.105+) if you are on Vercel AI Gateway or already use the AI SDK. Use raw fetch for other runtimes or routes, adding your own retry and timeout. — Documented, 29 Sep.

Fields, each Documented unless stated, checked 29 Sep:

• Versions and dates:
  • @typesafe-ai/sdk 0.6.0 (npm 2026-09-15T18:17Z; tag 66880cc).
  • ai 7.0.122 (2026-09-28T21:59Z).
  • experimental_evaluate present from 7.0.103 (2026-09-16T18:36Z; type declarations read); Vercel's guide says "7.0.105 or later". State both.
  • @ai-sdk/typesafe-ai 3.0.10 (2026-09-28T22:00Z).
• Install and runtime: SDK Node ≥ 20, and it refuses to run in a browser without dangerouslyAllowBrowser; ai Node ≥ 22.
• Routes and auth variables: TYPESAFE_API_KEY / TYPESAFE_AI_API_KEY / AI_GATEWAY_API_KEY.
• Minimal Choice call per path: mark each "Documented, not executed". Sources: SDK README and docs JavaScript page; ai-sdk.dev evaluation guide and TypeSafe provider page; docs quickstart; Cloudflare model page.
• Behaviour fields:
  • Response fields.
  • Default retries: SDK 2; AI SDK 2; fetch none.
  • Default timeout: SDK 10 s per attempt, no total budget found; AI SDK none documented.
  • Error classes; stability ("experimental … may change in patch releases"); test facility.
  • Licence: SDK MIT; ai Apache-2.0.
• Open SDK issues: #2, #6, #8, #9, #12, #13, #14, #15, #17 — Reported (GitHub, open on 29 Sep).
• Examples:
  • Jevmail maxRetries: 0 — Documented at 28 Sep (carried, not re-audited).
  • Vercel template threshold 0.95 and fallback — Documented at 27ea469 (22 Sep), checked 29 Sep. Note the model-name mismatch between page and source.
  • MSFT-TKENDRICK/JEV-examples — "author states never executed live" (Reported, carried).
• Gotchas table: 12 rows, each labelled. TS7's 31.5 s worst case is derived arithmetic from the SDK defaults, not measured.
• "Seen" only: TanStack AI decide() (Vercel article, Documented by Vercel); LangChain JS not checked.

Links: hub build; siblings build/python and build/testing-without-a-key; videos (TypeScript group); what-is-jev ("next step"); model IDs link to where-to-use. No prices.

4.3 build/testing-without-a-key — "Testing a Jev (TypeSafe AI) integration without an API key: mocks, simulators and failure branches"

Direct answer: Swap one layer at a time — the HTTP transport (official SDKs), the model (AI SDK mock model), or the whole server (a local mock) — and drive each of the ten failure branches before you have a key. None of these proves how Jev itself answers. A no-key "mock mode" left on in production is the most common way a Jev check silently disappears. — Documented and Tested (offline), 29 Sep.

Facility table: 12 rows, with source, label and date in the last column. Current-run rows are checked 29 Sep; carried rows keep their 25–28 Sep dates.

Checklist: 10 rows (method and required behaviour only; link each code to the errors page).

Worked example: T04 script and output (Tested, offline, local mock server, no Jev call, 29 Sep ~07:16 UTC). State: "pool-authored client that mirrors the SDK defaults; the SDK itself was not installed". Also cite T01 (Python, 28 Sep) as the Python counterpart.

Silent-bypass examples:

• QuantDinger: no key → allow (28 Sep audit).
• jev-trader and Jev Trade: MODEL=mock by default (25–26 Sep).
• danna-zhou hooks: fail open (Tested 29 Sep).
• jev-claude: shadow by default (29 Sep).

Links: hub build; build/errors-and-rate-limits ("test the no-key and outage paths"); build/python; build/typescript; build/confidence-thresholds (low-confidence branch); where-to-use/access-status ("no key yet?").

4.4 build hub — "Build with Jev (TypeSafe AI): choose Python, TypeScript, HTTP, Claude Code or a gateway"

Selection table (code interfaces only; no prices):

Plus links to build/errors-and-rate-limits, build/confidence-thresholds and build/testing-without-a-key. Ship only when at least five leaves are live.

4.5 Changes to existing pages

Products, correction 2 in detail:

• Rename "jev-mcp" to "jev-mcp (jkudish)": grade A, advisory, v0.11.0, checked 29 Sep.
• Add or regrade rows, each linking to claude-code-mcp:
  • danna-zhou/jev-mcp: A; hooks fail open; MCP tools advisory.
  • jev-claude: A; shadow by default.
  • LangChain AutoModeMiddleware: A; fail-closed; experimental alpha; framework vendor, not TypeSafe.
  • oh-my-claudecode Jev points: A; heuristic fallback, shadow.
  • Official TypeSafe plugin: A; instructions only.
• Optional: list typesafe-ai/n8n-nodes-typesafe-ai among official surfaces as "official repository, not reviewed".

4.6 Volatile fields for Where to use and Access status (all unchanged; new check dates)

5. Evidence gaps and open questions

1. No live behaviour. No integration was run against the real API. The 401/403 no-key path of the danna-zhou hook was inferred from code; the offline run covered unreachable endpoints only. Whether TypeSafe rejects model: "kev-latest" is unknown.
2. jkudish TypeSafe-direct transport: timeout and retry behaviour live in @jkudish/jev-agent-tools 0.1.x, which was not read. Marked unknown.
3. danna-zhou MCP server: no timeout in code. Behaviour on a hanging endpoint depends on the Node fetch defaults and the MCP client; not tested.
4. jev-claude: the deterministic rule list (src/deterministic.mjs) was not read line by line. The owner mismatch (brunopivetta88 repo vs bruno-ship-it in its manifests) was not investigated. The Codex adapter was not audited.
5. oh-my-claudecode: the per-point states and which hook events each point runs in (points.ts) were not read. The npm package name for the plugin was not verified.
6. Official plugin install scope (user vs project) for claude plugin install is not stated in TypeSafe's docs; not tested.
7. AI SDK:
   • The default timeout is undocumented.
   • The first version with experimental_evaluate conflicts: 7.0.103 by package contents vs 7.0.105 in Vercel's guide.
   • The @ai-sdk/typesafe-ai 3.0.9/3.0.10 changes were not diffed.
   • The Cloudflare binding's response shape and retries were not re-read.
8. SDK behaviour not executed: the official JS SDK was not installed (T04 mirrors its defaults in pool code). T03 (Python SDK install) remains not done.
9. Access: the console still returns 403. The credit status rests on the 27 Sep CEO post plus the absence of later statements. AI FrontPage and X oEmbed were not rechecked this run.
10. Scan coverage:
    • Reddit, X, TikTok and Lobsters are unknown (blocked).
    • YouTube, GitHub trending and GitHub code search were not attempted.
    • npm and Hugging Face counts are capped.
    • New items (Jeff decision models, typesafe-ai/n8n-nodes-typesafe-ai, 12+ new hook/MCP repos) are leads only.
11. Carried facilities (Jevmail /preview, QuantDinger tests, mockCallJev, Jev Workbench, jev-trader, Jev Trade, Ollaya/Kev) were not re-checked; they keep their 25–28 Sep dates.

6. Not done

• Audits of the "seen, not audited" items (codaaiteam, AutoJev, mcp-jev, Afloat16): recorded as "seen, not audited" only. AutoJev's repository was not found.
• Audits of the new S1 hook/MCP repositories (jev-scope-control, jev-gate, cc-jev-teacher, drift-guard, jevis, jev-harness, jev-linter, toolgate, jev-kit, jev-pruner, permit-mcp). These are candidates for run 3's first slot if the planner wants a larger table.
• YouTube and GitHub trending surfaces in S1.
• Diff of @ai-sdk/typesafe-ai 3.0.9/3.0.10 and of the TypeSafe homepage republish.
• Reading @jkudish/jev-agent-tools, jev-claude/src/deterministic.mjs and oh-my-claudecode/src/hooks/jev/points.ts in full.
• LangChain JS and TanStack AI paths beyond "seen".

7. Sources

The full source ledger is held in the pool's private record; the key pinned sources are below.

Key pinned sources:

• Official plugin:
  • https://github.com/typesafe-ai/skills/tree/65a39f393687675ce170e6094757de20370365b9
  • Docs: https://docs.typesafe.ai/agent-skill
• danna-zhou:
  • https://github.com/danna-zhou/jev-mcp/blob/436469a258593ab6c7e33dcf05a78bd8f5e95d22/hooks/lib.mjs#L46-L61
  • …/hooks/pretooluse-guardrail.mjs#L24-L85
  • …/hooks/stop-guardrail.mjs#L41-L130
  • …/src/index.ts#L16-L18
• jev-claude:
  • https://github.com/brunopivetta88/jev-claude/blob/7d7e8763f0e382fc253d091902abe8a7eb0a1ea2/src/config.mjs#L9-L16
  • …/src/policy.mjs#L81-L105
  • …/src/run.mjs#L131-L139
  • …/src/jev-client.mjs#L136-L184
  • …/src/entry.mjs#L5-L24
  • PR: https://github.com/brunopivetta88/jev-claude/pull/1
• jkudish:
  • https://github.com/jkudish/jev-mcp/blob/53fe5756252956863af2ecbd2952339352e82e15/src/index.ts#L296-L372
  • …/src/provider.ts#L28-L49
  • …/README.md#L242
• LangChain: https://pypi.org/project/langchain-typesafe/0.0.1a3/ (sdist langchain_typesafe/experimental/middleware/auto_mode.py L38–58, L94–98, L139–182, L208–234; classifier.py L36–38, L255–265)
• oh-my-claudecode:
  • https://github.com/Yeachan-Heo/oh-my-claudecode/blob/9fd35ece5d6de65b511bf43b55e42c499e4fc194/src/hooks/jev/config.ts#L1-L31
  • …/src/hooks/jev/resolver.ts#L1-L14
  • …/docs/adr/03665-jev-degradation-contract.md
  • PR (base dev): https://github.com/Yeachan-Heo/oh-my-claudecode/pull/4092
• JS SDK:
  • https://github.com/typesafe-ai/typesafe-sdk-js/blob/66880ccded6cb642dc1809620c2b108c33730214/src/retry.ts#L5-L23
  • …/src/errors.ts#L70-L118
  • …/src/client.ts#L48-L65
• AI SDK:
  • https://ai-sdk.dev/docs/reference/ai-sdk-core/evaluate
  • https://ai-sdk.dev/docs/ai-sdk-core/evaluation
  • https://ai-sdk.dev/providers/ai-sdk-providers/typesafe-ai
  • https://cdn.jsdelivr.net/npm/ai@7.0.103/dist/index.d.ts
• Vercel:
  • https://vercel.com/kb/guide/typesafe-jev-and-ai-sdk
  • https://vercel.com/templates/next.js/jev-and-ai-sdk
  • https://github.com/vercel-labs/jev-ai-sdk-form-router/blob/27ea469d15b5b26b922d9faacfdcffc13bf024bb/lib/router.ts#L8-L10
  • https://vercel.com/i/jev-integrations
• S2:
  • https://docs.typesafe.ai/models
  • https://status.typesafe.ai/
  • https://openrouter.ai/api/v1/models/typesafe/jev-1.13/endpoints
  • https://ai-gateway.vercel.sh/v1/models
  • https://developers.cloudflare.com/ai/models/typesafe/jev/
  • https://opper.ai/typesafe/jev-1-13-0
  • https://pypi.org/project/typesafe-sdk/
  • https://www.npmjs.com/package/@typesafe-ai/sdk