Shaduf.Research preview
Build64 implementations in 11 domains, recomputed

When Jev fails (TypeSafe AI)

What do real Jev implementations do when Jev returns an error, a malformed answer or an unsure answer, when no API key is set, or when a code path skips Jev? This page counts the answers across 64 implementations in eleven use cases, already published row by row on their domain pages, and turns them into an eight-question checklist for your own code. The title is an internal label: six failure phrases ("jev fallback", "jev timeout", "jev error handling", "jev fail open", "jev retry", "typesafe api error") got no relevant autocomplete suggestion on Google, YouTube, Bing or DuckDuckGo when the pool checked on .

Key finding

Of 64 Jev implementations read in source, on a Jev error 35 stop or hold the action, 13 fall back to another model, rule or default, 8 are advisory or return no decision, and 8 let it through: 4 moderation bots, 2 agent hooks, a passage filter and an eval tool.

Documentedrecomputed from 11 domain tables, checked

The eval tool is openlayer jevals: its report counts an errored eval as passed and its gate allows by default. Counting 5 fallbacks that can still end in the action as well, the action goes ahead on a Jev error in 13 of 64 (8 fail-open plus 5 fallbacks; details). The "8 are advisory or return no decision" are 3 advisory and 5 no-decision rows.

The 2 agent hooks are danna-zhou/jev-mcp for Claude Code and jev-judge for Hermes Agent, in enforce mode (its default is shadow).

Implementations were picked per domain page as stated there, not as a survey. The counts describe these 64 implementations at their pinned commits; they are not a rate for Jev integrations in general. "Not recorded" means the domain page and its research note do not record the cell; it is never inferred. n = 64 in every count. The official TypeSafe Claude Code plugin makes no runtime Jev call, so it is excluded from every denominator; the trycua/cua jev-use example calls a mock provider by default and is not counted either. Judge tools count from 7 Oct 2026: a judge whose own code turns Jev's answer into a score, pass/fail or gate is an action (an error that becomes a pass is fail-open; a dropped item is no-decision). The four judge tools on Jev as a judge form the LLM as a judge domain. The Jevals harness on Jev as a judge is not counted: its code is not public, so its failure cells are Reported from its methodology page, not read at a pinned commit. n8n nodes are counted with Continue On Fail (COF) off, the default; the COF-on result is given beside them. The pool ran nothing for this page: no bot, no browser agent, no eval, no Jev call.

What the 64 do, condition by condition

64 implementations per condition: each bar is split by what the code does (default settings)

  • Jev error or timeoutHTTP error, rate limit, timeout; n8n with Continue On Fail off · n = 64 · not recorded 0/64
    • 35/64 fail-closed
    • 13/64 fallback
    • 3/64 advisory
    • 5/64 no-decision
    • 8/64 fail-open
    • 0/64 not recorded

    action goes ahead 13/64 on a Jev error: the 8 fail-open rows (4 content-moderation bots, the danna-zhou hook, jev-judge in enforce mode, the Spring AI passage filter and openlayer jevals, whose report counts an errored eval as passed) plus 5 fallbacks that can still end in the action (QuantDinger after its LLM fallback; btc-hft-jev through rules-only; jev-permission-gate, where Claude Code's built-in classifier can still allow; rh-guard, where its lexical detectors can still allow; opencode-tool-gate, where OpenCode's default rules allow every tool, checked at v1.18.35). Not counted here: the 4 routing fallbacks and the LlamaIndex reranker, which return another model or the retriever's order by design; osuki, whose fallback routes to the default tier (the same routing-fallback convention; on the coding-agents page its tool call still runs); second-look, which shows a recommendation to a person and takes no action; jev-approvals and jev-curator, which hold; DeepEval, which aborts the run; and did-they-answer and competitor-hunter, which drop the item.

  • Malformed answerunknown option, missing field, bad probability · n = 64 · not recorded 3/64
    • 29/64 fail-closed
    • 16/64 fallback
    • 3/64 advisory
    • 3/64 no-decision
    • 10/64 fail-open
    • 3/64 not recorded
  • Below thresholdJev answers but is unsure; separate terms, never added to the error totals · n = 64 · not recorded 6/64
    • 13/64 held
    • 7/64 fallback
    • 4/64 advisory
    • 4/64 no-decision
    • 13/64 acts-anyway
    • 17/64 no-threshold
    • 6/64 not recorded
  • No API key configuredn = 64 · not recorded 19/64
    • 29/64 fail-closed
    • 9/64 fallback
    • 3/64 advisory
    • 0/64 no-decision
    • 4/64 fail-open
    • 19/64 not recorded
  • Bypassa code path that acts without asking Jev · n = 64 · not recorded 33/64
    • 31/64 recorded
    • 33/64 not recorded
Every number is in the text beside its bar; the bars only show the same numbers as lengths. Colours: green stops, refuses or holds for a person (fail-closed, held); blue is a named substitute (fallback-…); amber means Jev never decides (advisory); gray returns nothing and leaves it to the caller (no-decision); red acts anyway (fail-open, acts-anyway); dashed is no-threshold (no confidence cut in the code); white is not recorded. Documented recomputed from the domain tables, 7 Oct 2026.

The same numbers as tables:

Error-type conditions: each row sums to 64
Conditionnfail-closedfail-openadvisoryfallbackno-decisionnot recorded
Jev error or timeout6435831350
Malformed answer64291031633
No API key configured6429439019
Below threshold: its own terms, counted separately; sums to 64
Conditionnheldacts-anywayfallbackno-decisionadvisoryno-thresholdnot recorded
Below threshold641313744176
Bypass: sums to 64. "Not recorded" means the domain page had no bypass column and its note names none; it does not mean "none"
Conditionnrecordednot recorded
Bypass643133

Counting judgements in the 5, 6 and 7 Oct recounts

  • second-look is counted as a browser row although it never acts: it recommends, and a person buys.
  • The JS autoModeMiddleware (@langchain/typesafe 0.0.2) is counted under frameworks, separately from the Python AutoModeMiddleware already counted on Claude Code / MCP guards.
  • Pydantic AI TypeSafeModel is not a decision row: it raises Jev errors to the agent the builder writes. Its behaviour is on framework integrations.
  • ironbee-express with no key is counted fail-open (conditional: a cached recording replays unjudged), following the 4 Oct convention for conditional cells.
  • 6 Oct: gulbaki/jev-llm-guard is not counted: it returns a verdict; it is not a decision. Its jev-guard CLI exits 0 whatever the verdict (allow, review or block) and exits 1 only on errors.
  • 6 Oct: jev-judge (Hermes Agent) is counted although its default is shadow (logs only); its cells describe enforce mode.
  • 6 Oct: osuki (OpenCode) is not counted in the "action goes ahead" line: its fallback routes to a default tier, under the same routing-fallback convention as the model routers and LlamaIndex.
  • 7 Oct: the four judge tools on Jev as a judge are counted as the "LLM as a judge" domain (4 rows), with the same cells and colours as on that page. DeepEval counts although it is a library: its own test runner in the same repository aborts the run or fails the test case. The Jevals harness is not counted: its code is not public.
  • 7 Oct: conditional judge cells follow the 4 Oct convention (the default configuration counts; the condition is in the note). openlayer: a Jev error is fail-open (the report says passed; the gate allows by default), and its no-key cell is not recorded because neither the page nor its research note says which backend branch is the default. did-they-answer: malformed is fail-closed (missing answer keys stop the run); the missing-usage case alone gives no-decision. competitor-hunter: a Jev error is no-decision; above 5% errors the run aborts.
  • 7 Oct: the Hermes plugins jev-approvals and jev-curator are now counted (they were "not settled" on 6 Oct). Hermes core v2026.9.24 runs an APPROVE without a person and sends errors to a person. Both do nothing until switched on: jev-approvals is off after install (core default manual), and jev-curator observes by default. Their cells describe the switched-on modes.
  • 7 Oct: settled without a count change: rh-guard's default mode is enforce; opencode-tool-gate's "action goes ahead" holds under OpenCode's default rules (v1.18.35: every tool allowed); osuki checked at 0.2.8, failure path unchanged.

The same counts by domain

Open a condition to see its counts per domain. Opening one closes the others. Each row sums to its n; "not recorded" is a column of its own. The domain name links to the page with the per-project rows.

Jev error or timeoutfail-closed 35, fail-open 8, advisory 3, fallback 13, no-decision 5 (of 64)
Domainnfail-closedfail-openadvisoryfallbackno-decisionnot recorded
Ticket triage5400100
Model routing6100410
Content moderation6140010
Claude Code / MCP guards6113100
Email and lead5400010
n8n nodes (COF off)5500000
Trading and fraud7500200
Browser and computer use8700100
Frameworks4210100
Coding agents8410300
LLM as a judge4110020
All6435831350

n8n with COF on: fallback 4, no-decision 1 (of 5). The action goes ahead on a Jev error in 13 of 64: the 8 fail-open rows plus 5 fallbacks (2 trading, jev-permission-gate, rh-guard, opencode-tool-gate). osuki's default-tier fallback is not counted there (routing-fallback convention).

Malformed answerfail-closed 29, fail-open 10, advisory 3, fallback 16, no-decision 3, not recorded 3 (of 64)
Domainnfail-closedfail-openadvisoryfallbackno-decisionnot recorded
Ticket triage5400100
Model routing6000411
Content moderation6050010
Claude Code / MCP guards6113100
Email and lead5300110
n8n nodes (COF off)5500000
Trading and fraud7320200
Browser and computer use8600200
Frameworks4210100
Coding agents8310301
LLM as a judge4200101
All64291031633

The 2 trading fail-open cells (jev-trader, fraud-jev) depend on the shape of the bad answer. jev-agent-browser is counted as a fallback (semantic target resolver), but the target it picks by word overlap is then clicked.

Below thresholdheld 13, acts-anyway 13, fallback 7, no-decision 4, advisory 4, no-threshold 17, not recorded 6 (of 64)
Domainnheldacts-anywayfallbackno-decisionadvisoryno-thresholdnot recorded
Ticket triage54000010
Model routing61021020
Content moderation61300110
Claude Code / MCP guards61110300
Email and lead51121000
n8n nodes (COF off)50000005
Trading and fraud72210020
Browser and computer use80002060
Frameworks40200020
Coding agents83310001
LLM as a judge40100030
All641313744176

Counted with the below-threshold terms and never added to the error totals. n8n: the low-confidence outputs are optional, so all 5 are not recorded. jev-android (below 0.65) and computer-use-jev (mutating actions below 0.5) stop with BLOCKED or an error for the caller: no-decision, not held, because no person or review queue receives the item.

No API keyfail-closed 29, fail-open 4, advisory 3, fallback 9, not recorded 19 (of 64)
Domainnfail-closedfail-openadvisoryfallbackno-decisionnot recorded
Ticket triage5500000
Model routing6000105
Content moderation6010005
Claude Code / MCP guards6113100
Email and lead5400001
n8n nodes (COF off)5000005
Trading and fraud7300301
Browser and computer use8610100
Frameworks4400000
Coding agents8310301
LLM as a judge4300001
All6429439019
Bypassrecorded 31, not recorded 33 (of 64)

Pages published before 4 Oct had no bypass column, so "not recorded" here does not mean "none".

Domains with 3 or more implementations: 11 of 11 (4 to 8 each). How the pages picked their implementations is stated on each domain page.

Eight questions to ask of your own Jev integration

Apply each question to your own code. The counts show how the 64 answered it; the chips link to example rows on the domain pages. Chip colours: green holds or stops, red acts anyway, blue falls back, amber is advisory.

  1. When Jev errors or times out, does the action still happen?

    35 of 64 stop or hold it. 8 let it happen unchecked: 4 content-moderation bots, 2 agent hooks (danna-zhou for Claude Code, jev-judge for Hermes), the Spring AI passage filter and openlayer jevals, whose report counts an errored eval as passed. None of the 7 browser agents clicks or types on a Jev error. 13 of 64 fall back, 3 of 64 are advisory, 5 of 64 return no decision; not recorded 0 of 64.

    jev-judge: fail-open in enforce modeWordPress Jev Comment Triage: holds, retries, then a persondanna-zhou/jev-mcp: the command runsSpring AI JevDocumentFilter: unscreened passages keptopenlayer jevals: errored eval counted as passedjev-ultrafast: "no action executed"
  2. What is your timeout, and what happens when it fires?

    Timeouts set in code range from 0.40 s (btc-hft-jev) to 180 s per task (DeepEval JevEval). Some set none: jev-for-chrome and Ulka have no request timeout. Coding-agent hooks and providers use 1.2 s (rh-guard) to 25 s (the two Hermes plugins; Hermes core's own default is 30 s). Timeouts were not tallied across all 64. Status codes and retries: errors and rate limits.

    btc-hft-jev: 0.40 s, then rules-onlyjkudish/jev-mcp: 60 s, error to the agent
  3. If Jev answers in the wrong shape, do you treat it as "no" or as "yes"?

    10 of 64 let the action go ahead on a malformed answer. 1 more (jev-agent-browser) guesses a target by word overlap and clicks it. 29 of 64 fail closed; 16 of 64 fall back (openlayer reads a non-numeric answer as 0.5); 3 of 64 are advisory; 3 of 64 return no decision; not recorded 3 of 64. Validate before acting.

    fraud-jev: a non-numeric probability → allowjev-trader: any answer other than "sell" → buygbesse/zammad-jev-triage: checks model, option and probability
  4. What happens when Jev is unsure?

    13 of 64 send unsure items to a person or a review queue. 13 act anyway below their cut. 17 have no threshold at all, including 5 of the 7 browser agents and 3 of the 4 judge tools. 7 of 64 fall back, 4 of 64 are advisory, 4 of 64 return no decision; not recorded 6 of 64. Choosing a cut: confidence thresholds.

    vynnlee/jev-mail: Review labelgbesse: review groupbackmeupplz/jev_antispam_bot: below 0.81, kept
  5. What happens with no API key?

    29 of 64 refuse to start, error or hold the call. 4 proceed as if allowed: WordPress triage, the danna-zhou hook, ironbee-express when it replays a cached recording, and jev-judge. 9 fall back to an LLM, a mock, a heuristic or the host's own permission rules. 19 not recorded. 3 of 64 are advisory. Test it: testing without a key.

    WordPress triage: WordPress's own decision standsdanna-zhou hook: proceedsjev-fraud-shield: mock
  6. Which code paths never ask Jev?

    31 of 64 have a recorded bypass, including passthrough and replay modes in 4 browser agents, read-only or permission-request-only paths in 2 coding-agent gates (opencode-tool-gate, jev-kit), and the off or observe modes of the 2 Hermes plugins. Not recorded 33 of 64. List yours: exits, allow-lists, admin users, kill switches, "mock" modes.

    jev-trader: the mock model trades without Jevbackmeupplz: group admins' messages skippedJevmail: /preview simulator, no mailbox actioncomputer-use-jev: passthrough mode clicks with no Jev and no key
  7. Is your fallback cheaper or riskier than failing?

    13 of 64 fall back to another model, scorer, rule set or default on a Jev error. second-look shows "Good deal — buy it" from a heuristic when Jev is unavailable. Not recorded 0 of 64.

    0xNatoshi/jev-codex-router: bills the frontier model on every Jev failureQuantDinger: asks an LLM, then allows
  8. Is the failure written down, and does a failed log write block the action?

    Not counted across the 64: the domain tables have no logging column. Record every failure with its reason.

    QuantDinger: logs "audit persistence skipped" and continuesjev-trader: writes its log after the order is sentBillionsBobby/JevRouter: attaches provider_errorSpring AI JevDocumentFilter: logs a WARN with the number of unscreened passages

The questions are a design reading of the 64 rows, not a measured result. No count on this page comes from anywhere but the map below.

Checklist to paste

Select the block and copy it into your review.

- [ ] When Jev errors or times out, does the action still happen? (35 of 64 stop or hold it. 8 let it happen unchecked: 4 content-moderation bots, 2 agent hooks (danna-zhou for Claude Code, jev-judge for Hermes), the Spring AI passage filter and openlayer jevals, whose report counts an errored eval as passed. None of the 7 browser agents clicks or types on a Jev error. 13 of 64 fall back, 3 of 64 are advisory, 5 of 64 return no decision; not recorded 0 of 64.)
- [ ] What is your timeout, and what happens when it fires? (Timeouts set in code range from 0.40 s (btc-hft-jev) to 180 s per task (DeepEval JevEval). Some set none: jev-for-chrome and Ulka have no request timeout. Coding-agent hooks and providers use 1.2 s (rh-guard) to 25 s (the two Hermes plugins; Hermes core's own default is 30 s). Timeouts were not tallied across all 64.)
- [ ] If Jev answers in the wrong shape, do you treat it as "no" or as "yes"? (10 of 64 let the action go ahead on a malformed answer. 1 more (jev-agent-browser) guesses a target by word overlap and clicks it. 29 of 64 fail closed; 16 of 64 fall back (openlayer reads a non-numeric answer as 0.5); 3 of 64 are advisory; 3 of 64 return no decision; not recorded 3 of 64.)
- [ ] What happens when Jev is unsure? (13 of 64 send unsure items to a person or a review queue. 13 act anyway below their cut. 17 have no threshold at all, including 5 of the 7 browser agents and 3 of the 4 judge tools. 7 of 64 fall back, 4 of 64 are advisory, 4 of 64 return no decision; not recorded 6 of 64.)
- [ ] What happens with no API key? (29 of 64 refuse to start, error or hold the call. 4 proceed as if allowed: WordPress triage, the danna-zhou hook, ironbee-express when it replays a cached recording, and jev-judge. 9 fall back to an LLM, a mock, a heuristic or the host's own permission rules. 19 not recorded. 3 of 64 are advisory.)
- [ ] Which code paths never ask Jev? (31 of 64 have a recorded bypass, including passthrough and replay modes in 4 browser agents, read-only or permission-request-only paths in 2 coding-agent gates (opencode-tool-gate, jev-kit), and the off or observe modes of the 2 Hermes plugins. Not recorded 33 of 64.)
- [ ] Is your fallback cheaper or riskier than failing? (13 of 64 fall back to another model, scorer, rule set or default on a Jev error. second-look shows "Good deal — buy it" from a heuristic when Jev is unavailable. Not recorded 0 of 64.)
- [ ] Is the failure written down, and does a failed log write block the action? (Not counted across the 64.)
Counts: 64 Jev implementations read in source, 11 domains, 7 Oct 2026. Source: /p/jev-catalog/build/when-jev-fails/

All 64 rows

Show the full map: 64 implementations × 5 conditions, plus the 2 excluded entries

Each implementation links to its row on the domain page (the older Claude Code / MCP rows link to that page's failure matrix and jev-permission-gate to its own row; browser and framework rows link to their rows on the domain pages published 5 Oct; coding-agent rows link to their rows on coding agents, 6 and 7 Oct; judge rows link to their rows on Jev as a judge, read 4 Oct and counted from 7 Oct). The commit link opens the source tree at the pinned commit; file paths and line ranges for each cell are on the domain page. Documented source at the pinned commit, checked on the date of the domain page (see sources).

DomainImplementation (commit)Jev error or timeoutMalformedBelow thresholdNo keyBypass
Ticket triageCyrilBaah/jev-triage-desk commit 63913177fallback-archive lanefallback-archive laneno-thresholdno confidence gatefail-closedHTTP 500not recorded
Ticket triagetuhinmitra888/ai-jev-ticket-triage commit 3e8b83a8fail-closed502fail-closed500held< 0.5 → human triagefail-closeddoes not startnot recorded
Ticket triageenderkus/zammad-jev-dispatcher commit f999eca7fail-closed502; ticket stays Unclassifiedfail-closed502held< 0.55 → Unclassified + notefail-closedfails at bootnot recorded
Ticket triagegbesse/zammad-jev-triage commit 5d2a0a78fail-closed503, retried; 20 sfail-closedvalidated → 503held< 0.9 → review groupfail-closed503not recorded
Ticket triageveermshah/jev-support-desk commit aef8fa0cfail-closedbatch aborts; 10 s / 30 sfail-closedheld< 0.6 → triage reviewfail-closedexit 2not recorded
Model routingLiteLLM complexity router commit 8d28e8dfallback-configured tier or local scorer3 sfallback-configured tier or local scorersame path as an errorno-thresholdconfidence logged onlynot recordednot recorded
Model routinggargpratyush/jev-router commit 38da6b8fallback-current model3 s deadlinefallback-current modelfallback-current model< 0.3not recordedrecordedprompt phrases ("use opus") override Jev
Model routingBillionsBobby/JevRouter commit e11084cno-decision15 s / 20 sno-decisionno-decision< 0.55not recordednot recorded
Model routing0xNatoshi/jev-codex-router commit 8701ef7fallback-frontier model4 sfallback-frontier modelno-thresholdconfidence logged onlyfallback-frontier modelrecordedkill-switch file; shadow mode
Model routingnidhi-singh02/agent-router commit fb24d06fail-closedexits; nothing launchednot recorded"not traced"held< 0.75 and consequential → you picknot recordednot recorded
Model routingthinkany-ai/autojev commit d5d5731fallback-local selection2 s / 8 sfallback-local selectionfallback-local selection< 0.5not recordedrecordedexplicit model request kept
Content moderationWordPress Jev Comment Triage commit d848a5bfail-closedcomment held, retried, then left for a personfail-openread as 0.0; WordPress approval can standheldmiddle bandfail-openno provider → WordPress decidesnot recorded
Content moderationbrainstormity/Jev-Moderation-Bot commit 325ea7ffail-openmessage left up action goes aheadfail-open→ legitimateacts-anyway< 0.70 left upnot recordednot recorded
Content moderationfrolleks/soter commit 756579afail-open"never blocks chat" action goes aheadfail-openadvisory0.5–0.8 → mod-log review, no actionnot recordednot recorded
Content moderationbackmeupplz/jev_antispam_bot commit 0b07e88fail-openmessage kept action goes aheadfail-openacts-anyway< 0.81 keptnot recordedrecordedgroup admins skipped
Content moderationCodeAlive-AI/mastra-jev-moderation commit 8735ea8fail-open5 s action goes aheadfail-openacts-anyway< 0.7 passednot recordednot recorded
Content moderationZafer-Liu/jev-demo-moderator commit cc47b25no-decisionerror field; batch 502no-decisionno-thresholdJev's Choice decides; code only tightensnot recordednot recorded
Claude Code / MCP guardsdanna-zhou/jev-mcp PreToolUse hook commit 436469afail-open8,000 ms; the command runs action goes aheadfail-openheld"dangerous" < 0.6 → askfail-openthe hook proceeds Tested offline no Jev call, 29 Sep 2026, as published on the Claude Code pagerecordednon-matching commands never sent
Claude Code / MCP guardsjev-claude 0.1.0, shadow mode as installed commit 7d7e876advisoryadvisoryadvisoryadvisoryflags onlyrecordedread-only tools with a clean rule check not sent
Claude Code / MCP guardsjkudish/jev-mcp (audited 0.11.0; checked 0.13.0) commit 53fe575advisoryerror to the agent; 60 sadvisory"review"advisoryadvisoryerror to the agentnot recorded
Claude Code / MCP guardsLangChain AutoModeMiddleware (Python) 0.0.1a3 commit 4af7ab8fail-closedrun ends; 30 sfail-closedacts-anywayis_risky < 0.5 → tool runs; fixed cutfail-closedmiddleware cannot be constructedrecordedtools not listed
Claude Code / MCP guardsoh-my-claudecode (v5.5.0; checked v5.6.0) commit 9fd35ecadvisoryshadow by default; the heuristic decides; 2,000 msadvisoryadvisoryno gate; shadowadvisorypoint off; heuristicrecordedpoints not listed; cap; open circuit
Claude Code / MCP guardsmadisonrickert/jev-permission-gate commit 7349bc9fallback-built-in classifier1.5 s action goes aheadfallback-built-in classifierfallback-built-in classifierdefersfallback-built-in classifierrecordedengine already decided or not auto mode; tools other than Bash, WebFetch, WebSearch; secrets or too-long input; measure mode; cache
Email and leadJevmail commit f6f20afno-decisionemail stays unclassifiedfallback-promotionalacts-anywayflagged; lane still assignedfail-closedrecorded/preview simulator (no Jev call; no mailbox action)
Email and leadilyamk Gmail labeler commit ade42bffail-closedjob pausesno-decisionskippedfallback-full-body callthen its label is applied uncheckedfail-closednot recorded
Email and leadparth-kp IMAP classifier commit db96194fail-closedrun stops; 15 sfail-closedno range checkno-decisionmarked processedfail-closednot recorded
Email and leadvynnlee/jev-mail commit 61e346ffail-closedretry laterfail-closedheldReview label, kept in the inboxfail-closednot recorded
Email and leadspam-leadgen-pipeline commit 78a7592fail-closedscript endsfail-closedfallback-not qualifiednot recorded"not determined": depends on the SDKnot recorded
n8n nodesOfficial node 0.9.0 commit c12537bfail-closedCOF on: fallback-first route (or Fallback/Uncertain if set)fail-closedRoute; Evaluate unvalidatednot recordedFallback/Uncertain outputs optionalnot recordednot recorded
n8n nodesDomMonte 0.1.1 commit ff2afcafail-closedCOF on: no-decisionfail-closednot recordednot recordednot recorded
n8n nodesvibe-with-me-tools 0.2.3 commit ddbfe85fail-closedCOF on: fallback-first route (or Low Confidence)fail-closednot recordedlow-confidence output optionalnot recordednot recorded
n8n nodestaifoon 2.0.5 commit cdcc0f3fail-closedCOF on: fallback-review outputfail-closed"Fail Closed" on by defaultnot recordednot recordedrecordedno routing configured → everything to pass
n8n nodeszampierid4p 0.3.3 commit 549a545fail-closedCOF on: fallback-question 1's branchfail-closedmissing answer → empty branchnot recordednot recordednot recorded
Trading and fraudQuantDinger commit a5a9f4cfallback-LLMthen fail-open; 8 s action goes aheadfallback-LLMthen fail-openfallback-LLMthen acts-anywayfallback-LLMthen fail-openrecordedexits; grid/DCA/martingale; credit denial
Trading and fraudjev-trader commit b587759fail-closedno new order; a resting order staysfail-openany other answer → buy; conditionalno-thresholdnot recordedrecordedmock model trades without Jev
Trading and fraudJev Trade commit a3f2f83fail-closedno new order; 4,000 msfallback-holdno-thresholdfail-closedrecordedmock model trades without Jev
Trading and fraudbtc-hft-jev commit 96f3541fallback-last judgment / rules-onlycan trade; 0.40 s action goes aheadfail-closedrun stopsacts-anywayhalf size; < 0.50fallback-mockforced dry runrecordedrules-only path
Trading and fraudjev-fraud-shield commit eeb4fc6fail-closedstep-upfail-closedheldstep-upfallback-mockrecorded0 or ≥ 3 flags decided without Jev
Trading and fraudfraud-jev commit aa33b97fail-closedreview, HTTP 500fail-opennon-numeric → allow; conditionalacts-anywayallow below the cutsfail-closed401recordedOFAC destination → review without Jev
Trading and fraudexpense-policy gate commit 161c33bfail-closedbatch aborts; 30 sfail-closedheldmanual reviewfail-closedrecordeddeterministic rejects skip Jev
Browser and computer usebrowser-use/jev-ultrafast commit 1231850fail-closed25 s; "no action executed"fail-closedno-thresholdconfidence recorded onlyfail-closednot recorded
Browser and computer useironbee-ai/ironbee-express commit 05079b4fail-closed25 s, 3 triesfail-closedno-thresholdfail-openconditional: a cached recording replays unjudged; fail-closed without onerecordedcached replay; LLM takeover
Browser and computer useforvela/jev-agent-browser commit eb35251fail-closed30 sfallback-semantic target resolverthen the action runsno-thresholdstop cuts onlyfail-closedstart URL opened firstrecordedclassify-mode regex overrides
Browser and computer usedougsong/jev-android commit be8364afail-closed25 sfail-closedno-decision< 0.65 → replan ×2 → BLOCKEDfail-closednot recorded
Browser and computer usepaulsmith/computer-use-jev commit ff0ad8bfail-closed60 sfail-closedno-decisionmutating < 0.5 → not executedfail-closedrecordedpassthrough mode (no Jev, no key)
Browser and computer usechy4pro/jev-for-chrome commit d5c24defail-closedno timeout; 3 retriesfail-closedno-thresholdfail-closednot recorded
Browser and computer userazaanstha/ulka commit bb04433fail-closedno timeoutfail-closedno-thresholdfail-closedrecordedcached replay; forced first action (both approval-gated)
Browser and computer useKaushikSiva/second-look commit 0c05d16fallback-heuristic verdict15 s; recommendation onlyfallback-heuristic verdictno-thresholdfallback-heuristic verdictnot recorded
FrameworksSpring AI TypeSafe JevDocumentFilter 0.4.0 commit 51993bbfail-openunscreened passages kept; 10 s × 3 attempts, 30 s budget action goes aheadfail-openacts-anywayinjection < 0.70 → passage keptfail-closedblank key: startup exceptionnot recorded
FrameworksLlamaIndex JevRerank (community) commit 421afdafallback-retriever ordertop_n; 2.5 sfallback-retriever orderno-thresholdoptional flag only, never dropsfail-closedValueError at constructionnot recorded
FrameworksLangChain ModelRouterMiddleware (Python) 0.0.1a3 commit 4af7ab8fail-closedrun ends; 30 sfail-closedno-thresholdalways answer.choicefail-closedconstructionnot recorded
FrameworksLangChain autoModeMiddleware (JS) 0.0.2 commit 417ddcffail-closedtool not run; 30 s × 3 attemptsfail-closedacts-anywayrisk < 0.5 → tool runs; fixed cutfail-closedconstructionrecordedunlisted tools never sent
Coding agentskeiffff/jev-kit jev-agent-review commit 3d74b05fail-closedheld for the host's prompt; SDK 10 s, no retry; 4 s hook timeout in the examplesfail-closedheldheldfail-closedheldrecordedsensitive input deferred unsent; only permission requests
Coding agentsbloudhood/codex-jev-approval-hook commit b6242c9fail-closedheld for Codex's approval; 5 sfail-closedheldheldfail-closedheldrecordedBash, apply_patch and mcp__ only; denial cap
Coding agents24601/rh-guard (default mode enforce, so a default install can block) commit c2e682efallback-lexical detectors1.2 s action goes aheadfallback-lexical detectorsacts-anywaysteer onlyfallback-lexical detectorsrecordedMCP tools not matched on Codex; shadow mode
Coding agentsjustmytwospence/opencode-tool-gate commit bdf1ef8fallback-opencode permission rules3 s; OpenCode v1.18.35 default rules allow every tool ("*": "allow") action goes aheadfallback-opencode permission rulesacts-anywayfallback-opencode permission rulesrecordedread-only tools; enabled: false
Coding agents@osuki-dev/opencode-osuki-agent 0.2.7, checked at 0.2.8 npm 0.2.7; tag commit c96ce03; 0.2.8 (tag 0ae9e88): failure path unchangedfallback-default tier10 s; routing fallback, not counted as going aheadfallback-default tierfallback-default tierfallback-default tierrecordedfixed roles; reuse; pruning limits
Coding agentsDoGMaTiiC/hermes-jev jev-judge (enforce mode; default shadow) commit 5dddbeafail-open3 s action goes aheadfail-openacts-anywayfail-openrecordedcache; circuit breaker; unlisted tools
Coding agentsanpicasso/hermes-jev-approvals with Hermes Agent core v2026.9.24 (off after install: core default approval mode manual) commit 28be98a; core f97608ffail-closedheld for a person (escalate); plugin 25 s, core default 30 sfail-closedempty or unknown answer escalates to a personheldAPPROVE below confidence 0.55 becomes ESCALATEfail-closedplugin raises; core escalates to a personrecordedapprovals off or manual (the default); allowlist; commands not flagged
Coding agentsanpicasso/hermes-jev-curator (guard or apply mode; default observe) commit 4e8626cfail-closedbackground skill delete or patch blocked; 25 snot recordednot recordednot recordedrecordedoff and observe modes inert; foreground operations; creates
LLM as a judgeDeepEval JevEval (deepeval 4.2.8) commit a200ecefail-closedrun aborts (default ignore_errors=False); with ignore_errors=True the test case is a FAIL; 180 s per taskfail-closeda missing answer raises, handled like an errorno-thresholdminimum confidence recorded, not acted onfail-closed"there is no LLM to fall back to"not recorded
LLM as a judgeopenlayer jevals 0.1.4 commit 0a8f895fail-openthe report's passed treats an errored eval as not failed; the runtime gate defaults to on_error="allow". Dataset summary: no score, left out of the denominator (no-decision). 15 s, 4 retries action goes aheadfallback-0.5a non-numeric Noul is read as 0.5; a missing answer is skippedno-thresholddefault; the gate holds only if you set escalate_below or escalate_above and an on_escalate handlernot recordedwhich branch is the default is not recorded: Jev backend named → error (fail-closed); auto-detect → other backends, then fallback-LLMnot recorded
LLM as a judgedid-they-answer commit e0ee60cno-decisionitem not written; retried on the next daily run; 60 s, 5 triesfail-closedmissing answer keys stop the run (traced, not executed); a missing usage field alone gives no-decisionno-thresholdtop option stored; "referred" split at 0.5fail-closedexits without a keynot recorded
LLM as a judgecompetitor-hunter (loading claude-x-jev c8662b0) commit 40613eano-decisionerrored items left out of the ranking; above 5% errors the run aborts (fail-closed); 60 s, 4 retriesnot recordeditem marked unsure; outcome not determinedacts-anywayunsure items flagged but still rankedfail-closedOpenRouter key requirednot recorded
Claude Code / MCP guardsOfficial TypeSafe plugin commit 65a39f3. Excluded from every count: no runtime Jev calln/aexcludedn/aexcludedn/aexcludedn/aexcludedn/aexcluded
LLM as a judgeJevals harness Excluded from every count: its code is not public, so its cells are Reported from a methodology page, not read at a pinned commitn/aexcludedn/aexcludedn/aexcludedn/aexcludedn/aexcluded
  • fail-closed or held
  • fallback-… (named)
  • advisory: Jev never decides
  • no-decision
  • fail-open or acts-anyway
  • no-threshold
  • not recorded

Where this map differs from earlier notes and pages

The starting point was a prep map from the 3 Oct run. Each difference was decided from the published domain page and its research note. None adds a new failure claim.

ItemEarlier map or pageUsed here
oh-my-claudecode, all cellsPrep map: fallback-heuristic twinadvisory at default settings (shadow; the heuristic always decides). fallback-heuristic only with OMC_JEV=<point>:active. On the 32-row tally: fallback 6 → 5, advisory 2 → 3
No key: jev-claude and jkudish/jev-mcpPrep map: not mappedadvisory for both, as the Claude Code / MCP page records ("Flags only"). Claude Code / MCP no-key cells: 5 of 5 recorded
Below threshold, relabelled to the 4 Oct termsPrep map: CyrilBaah fail-open; LiteLLM and 0xNatoshi "acts anyway"; Zafer-Liu "Jev's own Choice decides"; agent-router advisory; review-group and review-label rows (tuhinmitra888, enderkus, gbesse, veermshah, vynnlee) fallback; WordPress "held band"CyrilBaah, LiteLLM, 0xNatoshi and Zafer-Liu → no-threshold; agent-router, the review rows and WordPress → held. Vocabulary only; no fact changes
BypassPrep map: email re-processing guards (ilyamk, vynnlee, spam-leadgen) and agent-router's secret-pattern refusal listed as bypassesnot recorded: they skip or refuse work, they do not act without Jev. Jevmail's /preview simulator stays a bypass
Jevmail, unsure answerEmail page (3 Oct): advisoryacts-anyway (flagged): the lane is still assigned. Relabel only; applied on the email page on 4 Oct
spam-leadgen-pipeline, unsure answerEmail page (3 Oct): fail-closed, which is not a below-threshold termfallback-not qualified. Relabel only; applied on the email page on 4 Oct
vynnlee/jev-mail, unsure answerEmail page (3 Oct): "fallback: Review label"held (a review label kept in the inbox), under the same relabel rule as the other review rows; applied on the email page on 4 Oct
DenominatorPrep map: 32 (+1 n/a)32 for the six domains published before 4 Oct; 39 with trading (4 Oct); 51 with browser and computer use and frameworks (5 Oct); 58 with jev-permission-gate and the six coding-agent rows (6 Oct); 64 with the four judge tools and the two Hermes plugins (7 Oct). The official plugin, the trycua/cua mock example, jev-llm-guard and the Jevals harness stay out

No disagreement was found between a published page cell and its note for the error, malformed and no-key cells of triage, routing, moderation, email and n8n (page tables spot-read against the prep map).

What was not verified

  • No-key behaviour is not recorded for 19 of 64, and bypass for 33 of 64. The pages published before 4 Oct had no bypass column.
  • n8n: the below-threshold and no-key cells are not recorded for all 5 nodes, and n8n core's own error-output handling was not read.
  • Rows from the six earlier domains were recomputed from their published tables and notes, not re-read from source in this run. The trading rows were read from source on 4 Oct; the browser and framework rows on 5 Oct (helper reads spot-checked by the runner; the Spring AI auto-configuration lines were read by a helper and not spot-checked).
  • Browser and framework rows: SDK defaults were not read (for example the @langchain/core retry backoff); whether computer-use-jev sends [secure] field values and how second-look's card words its fallback verdict are not recorded.
  • Coding-agent rows (6 and 7 Oct): osuki was read from the npm 0.2.7 tarball and checked at 0.2.8 (tag 0ae9e88); the tarballs and tags were not matched file by file. jev-judge is counted in enforce mode (default shadow), jev-approvals is off after install and jev-curator observes by default; their cells describe the switched-on modes. jev-curator's malformed, below-threshold and no-key cells and its cache lifetime are not recorded; the step in which Hermes core hands the question to the plugin (call_llm) was not read. How Codex handles jev-kit's exit 2 was not read.
  • Judge rows (7 Oct): mapped from the published Jev as a judge table and its 4 Oct research note at the 4 Oct pins, not re-read from source; HEADs were not rechecked. openlayer's default no-key branch needs a one-file read. None of the four has a bypass column.
  • The 2 trading fail-open cells for malformed answers depend on the shape of the bad answer.
  • Logging (question 8) and timeouts (question 2) were not tallied across all 64.
  • How any of the 64 behaves in real use: none was run here (no agent, hook, plugin, eval or MCP server was installed or run; no Jev call), and how many people use them is unknown.

Sources and check times (UTC)

  • Recount of 7 Oct 2026, 05:23–05:30 UTC (run 10): the 58-row base as published here (R10-S30), plus 4 judge rows mapped from Jev as a judge (R10-S31) and its 4 Oct note (R10-S32) at DeepEval a200ece, openlayer jevals 0a8f895, did-they-answer e0ee60c and competitor-hunter 40613ea; plus 2 Hermes rows. Every condition row and every domain sums to its n (script). No existing row changed a cell.
  • Hermes rows, 7 Oct 2026, 05:19–05:26 UTC (helper reads; runner spot-checks 05:24–05:25 matched): jev-approvals 28be98a (plugin/jev_policy.py L13, L54–59; R10-S41) with Hermes Agent core v2026.9.24 f97608f (tools/approval.py L774–790, tools/approval_smart.py L115–131; R10-S101–R10-S105, R9-S61); jev-curator 4e8626c (plugin/engine.py L91–94, plugin/guard.py L57–74, L143–160, plugin.yaml L50–53; R10-S106–R10-S111). Settled without a count change: rh-guard default mode (src/lib/risk/kinds.ts L176–189 at c2e682e; R10-S117, R10-S118); OpenCode v1.18.35 default rules (packages/opencode/src/agent/agent.ts L119–136 at 53d1eab; R10-S114–R10-S116); osuki 0.2.8, tag 0ae9e884a97a5539fef3f0772318a628c1f75e00 (R10-S112, R10-S113).
  • Recount of 6 Oct 2026, 05:22–05:29 UTC (run 9): the 51-row base as published here, read 05:25:44, plus 7 new rows; every domain row and total sums to its n (script). No existing row changed.
  • jev-permission-gate: madisonrickert/jev-permission-gate 7349bc9 (hooks/policy.ts, hooks/register.ts, hooks/typesafe.ts), read 4 Oct 2026 (run 7); HEAD = that pin by git ls-remote, 6 Oct 2026, 05:20:19 (R9-S70). Not counted: gulbaki/jev-llm-guard 7c7b1b5 (src/cli.js, src/web-server.js; R9-S70, R9-S71).
  • Coding agents: six rows read at pinned commits, 6 Oct 2026, 05:14–05:25 (helper reads spot-checked by the runner at 05:22): jev-kit 3d74b05 (packages/hook-adapters/src/index.ts, packages/agent-review/src/index.ts, packages/decision-contract/src/index.ts, packages/cli/src/cli.ts; R9-S72, R9-S73); codex-jev-approval-hook b6242c9 (hook.ps1; R9-S81); rh-guard c2e682e (src/lib/risk/score.ts, jev.ts, hooks.ts, hosts.ts, hooks/run.ts; R9-S82); opencode-tool-gate bdf1ef8 (src/index.ts, src/jev.ts, src/judge.ts; R9-S74); @osuki-dev/opencode-osuki-agent 0.2.7, tag commit c96ce03c1c7dfc5c6fdf9d966691549101c68080 (dist/index.js; R9-S80); hermes-jev jev-judge 5dddbea (plugins/jev-judge/__init__.py, jev.py, gate.py, plugin.yaml; R9-S63).
  • Recount of 5 Oct 2026, 05:29 UTC (run 8): the 39-row base as published here, read 05:26, plus 12 new rows; every domain row and total sums to its n. No existing row changed.
  • Browser and computer use: eight projects read at pinned commits, 5 Oct 2026, 05:12–05:24 (runner spot-check of 15 key lines): jev-ultrafast 1231850 (jev_ultrafast/model.py, agent.py); ironbee-express 05079b4 (src/engine/systemone.ts, src/agent/policy.ts, agent.ts, run/runner.ts); jev-agent-browser eb35251 (src/decision.js, src/loop.js, src/classifier.js, cli.js); jev-android be8364a (JevProvider.kt, ProviderTransport.kt, JevAgent.kt, Models.kt); computer-use-jev ff0ad8b (decide/decide.go, typesafe/client.go, cmd/computer-use-jev/main.go); jev-for-chrome d5c24de (src/background/agent.ts, src/shared/providers/http.ts, typesafe.ts); Ulka bb04433 (apps/extension/src/agent/vercel-jev.ts, agent-runner.ts, approvals.ts, background.ts); second-look 0c05d16 (backend/app/jev.py, backend/app/research.py). Not counted: trycua/cua jev-use ba0033a (run.py: mock provider by default).
  • Framework integrations: four decision rows read at pinned tags or commits, 5 Oct 2026, 05:12–05:27 (runner spot-check of 26 key lines, plus its own read of the three decision paths): Spring AI TypeSafe 51993bb (JevDocumentFilter.java, RetryPolicy.java, TypeSafeConstants.java); LlamaIndex JevRerank 421afda (base.py, utils.py); LangChain ModelRouterMiddleware 4af7ab8 (model_router.py, classifier.py); LangChain autoModeMiddleware 417ddcf (src/middleware/autoMode.ts, classifier.ts).
  • Domain tables re-read on 4 Oct 2026 at 05:19; counts recomputed 05:19–05:30. Each cell's file path and line range is on its domain page.
  • Support-ticket triage: five projects read at pinned commits, 1 Oct 2026, 05:32–05:36.
  • Model routing: six routers read at pinned commits, 2 Oct 2026, 05:16–05:21.
  • Content moderation: six projects read at pinned commits, 2 Oct 2026, 05:23–05:27.
  • Claude Code and MCP guards: integrations read in pinned source, 29 Sep 2026, 07:09–07:14; versions rechecked 3 Oct 2026, 05:21 (oh-my-claudecode at v5.6.0, jkudish/jev-mcp at 0.13.0). The danna-zhou no-key cell was also Tested offline there (no Jev call), 29 Sep 2026.
  • Email and lead sorting: five projects read at pinned commits, 3 Oct 2026, 05:19–05:27.
  • Jev in n8n: five nodes read at pinned commits, 3 Oct 2026, 05:20–05:27.
  • Trading and fraud gates: seven projects read at pinned commits, 4 Oct 2026, 05:14–05:21: QuantDinger a5a9f4c (backend_api_python/app/services/ai_decision_filter.py, services/strategy_v2/live_execution.py, routes/quick_trade.py, routes/quick_trade_ai.py, routes/alpaca.py, services/trading_executor.py, services/pending_order_worker.py, tests/test_ai_decision_filter.py); jev-trader b587759 (src/config.ts, model.ts, trader.ts, market.ts, server.ts); Jev Trade a3f2f83 (src/config.ts, model.ts, plan.ts, trader.ts, index.ts, market.ts, sleeves.ts, server.ts); btc-hft-jev 96f3541 (jevloop/client.py, loop.py, policy.py, battery.py, ladder.py, state.py, strategy.py, limits.py, execution/alpaca.py); jev-fraud-shield eeb4fc6 (jev_client.py, decision_engine.py, demo.py); fraud-jev aa33b97 (src/lib/fraud.ts, src/app/api/analyze/jev/route.ts); expense-policy gate 161c33b (src/jev_expense_gate/cli.py, jev_client.py, policy.py, pipeline.py, config.py, config/expense-policy.json).
  • Query wording: Google suggest (client=firefox), YouTube (ds=yt), Bing (osjson) and DuckDuckGo (ac), 4 Oct 2026, 05:21:29, for "jev fallback", "jev timeout", "jev error handling", "jev fail open", "jev retry" and "typesafe api error": no relevant suggestion (the same result as 3 Oct). A suggestion is a demand signal, never a volume.

Search published pools, pages, reports, and evidence.