Shaduf.Research preview
Jev: Use Cases, Alternatives & Products/Jev vs OpenAI and Microsoft decision APIs
CompareNew page · rows checked and

Decisions API vs Jev (TypeSafe AI) vs Microsoft Decision 1, with Cloudflare Clef: graded tests, price, images and data terms

Key finding, 13 Oct 2026

Graded results depend on the task: OpenAI's Decisions API ahead on RPG move accuracy (B38); Jev on Portuguese exact judge score (B39); Jev and Microsoft-Decision-1 not separated on routing macro-F1, both ahead of OpenAI (B41).

Reportedresults, Documentedmaterials; B38 and B39 checked ; B41 checked .

New page, 13 Oct 2026. Since 12 Oct: Microsoft-Decision-1 is listed as Preview, not GA, in Foundry; Decision-1 rate limits, OpenAI refusal behaviour and Clef routes added; prices unchanged. A fourth graded comparison with the Decisions API, B42 (grade B), sits below the three above. All changes.

Comparison card

Four providers, each with its own models and terms. Each cell links its dated, sourced cell in the full table. Prices are as each vendor publishes them; the pool does no per-call cost arithmetic. OpenAI, Microsoft and Cloudflare are other providers: your data goes to them (on OpenRouter's non-Cloudflare hosts for Clef, PrimeIntellect and DeepInfra, it goes to that host).

FactJevTypeSafeOpenAI Decisions APIOpenAIanother provider; your data goes to OpenAIMicrosoft-Decision-1Microsoftanother provider; your data goes to MicrosoftCloudflare Clef familyCloudflareanother provider; your data goes to Cloudflare on Workers AI; on OpenRouter's non-Cloudflare hosts (PrimeIntellect, DeepInfra), to that host
Graded testsB38 behind on RPG move accuracy; B39 ahead on exact judge score, level on routing and injection; B41 not separated from Decision-1 on routing macro-F1, ahead of OpenAI; B42 (grade B) ahead on logic-question accuracy; answer key withheldB38 ahead on RPG move accuracy; B39 behind on exact judge score; B41 behind both on routing macro-F1; B42 (grade B) behind on logic-question accuracy; answer key withheldB41 only: not separated from Jev on routing macro-F1, ahead of OpenAIB32 security experiments only; not an accuracy comparison
Price as published$0.042 per 1M input tokens; output free$0.10 per 1M input tokens; no output charge (guide; not on pricing page)$0.042 per 1M input tokens; output freeClef $0.240, Clef-flash $0.038, Clef-omni $0.150 per 1M input; output free
ImagesText only; no image, audio or videoText and images (up to 128 images per request)Text onlyImages on all three; audio and video on Clef-omni
Request shape vs Jev's /v1/systemonePOST /v1/systemone: state plus typed questions; Bearer keyDifferent: POST /v1/decisions, input plus questions (predicate, choice, score)Same state-plus-questions shape on Foundry …/v1/systemone; Entra ID or api-keySame state-plus-questions shape via Workers AI /ai/run; media extensions
Your dataNot used to train models (Privacy Policy); no retention period publishedNot used for training; abuse-monitoring logs up to 30 daysNot used to train base models; "temporarily store" for abuse monitoring; no period foundNot used to train; no retention period on Cloudflare's data-usage page
Documented errors and limits401, 422, 429, 529; back off and retry; 80 requests/s, 100K tokens/s (errors checked 6 Oct 2026)Refusal answer type documented; no error codes or rate limits found for /v1/decisions400–429 table; 429: exponential backoff; 60–900 RPM by tierPlatform errors only (408, 429); no Clef-specific rate limit found

Checked 12–13 Oct 2026 (UTC). Different providers, models and terms; no overall ranking. Full table and sources below: https://shaduf.ai/p/jev-catalog/alternatives/jev-vs-decision-apis/

Graded tests, one task at a time

Wording, grades, operators and flags as on the benchmarks ledger (39 graded rows, 13 Oct 2026). Graded comparisons with OpenAI's Decisions API: 4 (B38, B39, B41, B42); with Microsoft-Decision-1: 1 (B41). Results are the operators' own; the pool ran none of these tests.

  • B38 A Reported On NPC move choices in a small role-playing game, the Decisions API was ahead (per-move accuracy 59.4% vs Jev 51.0%).
    Operator: komzweb. 4 flags: game taskoperator-designed taskone runfull logs not committed Checked .
  • B39 A Reported On Portuguese routing, injection detection and judging, Jev was ahead on the exact judge score (80.7% vs 72.0%) and level on routing and injection.
    Operator: mateusfonsek. 5 flags: single languageLLM-generated dataone runjev-latest aliasp-value source not located Checked .
  • B41 A Reported On synthetic bank-withdrawal routing, Jev (89.2% macro-F1) and Microsoft-Decision-1 (87.6%) were not separated and both were ahead of OpenAI's Decisions API (71.4%); Decision-1 kept the same route across all 6 option orders more often (97.2–97.6% vs Jev 91.6–91.8%, Decisions API 64.2%).
    Operator: kartikanand73. 4 flags: synthetic single taskthresholds set with Jev-shaped probabilitiesall three via OpenRoutercode version not pinned Checked .
  • B32 A Reported Clef column: security experiments; not an accuracy comparison. The 2,048-token truncation finding concerns Clef, not Jev.
    Operator: system-one-security. 4 flags: small-nno-baselinesealedsingle-task Graded .
  • Added 13 Oct 2026, below B38, B39 and B41; not in the key finding. B42 B Reported On 1,000 five-option logic questions (B42, grade B), Jev answered 817 correctly and OpenAI's Decisions API 612; Jev's answers come from a 22 Sep run, and the answer key is not published.
    Operator: GitHub account etsabary (etsabary/luna-decisions-benchmark at 7a5ea13). 7 flags: via OpenRouter's alpha Decisions endpoint, not OpenAI directJev not re-run (22 Sep vs 10 Oct; mixed Jev providers, 256 answers without a verified snapshot)answer key withheld (only its sha256 committed)17 persistent refusals (HTTP 502), cause unknownthe operator built both benchmarksyoung repository (2 commits; HEAD changed after the main publish commit)latency and cost not comparable across dates and routes Re-pin checked ; files opened in run 14 (R14-S300 to R14-S305; R15-S53).

Copy this comparison

This block repeats the card above word for word, as plain text with the links written out. Select it and paste it into a review.

Decisions API vs Jev (TypeSafe AI) vs Microsoft Decision 1, with Cloudflare Clef: comparison card
Checked 12–13 Oct 2026 (UTC). Different providers, models and terms; no overall ranking. Full table and sources below: https://shaduf.ai/p/jev-catalog/alternatives/jev-vs-decision-apis/
Columns: Jev (TypeSafe); OpenAI Decisions API (OpenAI, another provider; your data goes to OpenAI); Microsoft-Decision-1 (Microsoft, another provider; your data goes to Microsoft); Cloudflare Clef family (Cloudflare, another provider; your data goes to Cloudflare on Workers AI; on OpenRouter's non-Cloudflare hosts (PrimeIntellect, DeepInfra), to that host)
Graded tests: Jev: B38 behind on RPG move accuracy; B39 ahead on exact judge score, level on routing and injection; B41 not separated from Decision-1 on routing macro-F1, ahead of OpenAI; B42 (grade B) ahead on logic-question accuracy; answer key withheld | OpenAI Decisions API: B38 ahead on RPG move accuracy; B39 behind on exact judge score; B41 behind both on routing macro-F1; B42 (grade B) behind on logic-question accuracy; answer key withheld | Microsoft-Decision-1: B41 only: not separated from Jev on routing macro-F1, ahead of OpenAI | Cloudflare Clef family: B32 security experiments only; not an accuracy comparison
Price as published: Jev: $0.042 per 1M input tokens; output free | OpenAI Decisions API: $0.10 per 1M input tokens; no output charge (guide; not on pricing page) | Microsoft-Decision-1: $0.042 per 1M input tokens; output free | Cloudflare Clef family: Clef $0.240, Clef-flash $0.038, Clef-omni $0.150 per 1M input; output free
Images: Jev: Text only; no image, audio or video | OpenAI Decisions API: Text and images (up to 128 images per request) | Microsoft-Decision-1: Text only | Cloudflare Clef family: Images on all three; audio and video on Clef-omni
Request shape vs Jev's /v1/systemone: Jev: POST /v1/systemone: state plus typed questions; Bearer key | OpenAI Decisions API: Different: POST /v1/decisions, input plus questions (predicate, choice, score) | Microsoft-Decision-1: Same state-plus-questions shape on Foundry …/v1/systemone; Entra ID or api-key | Cloudflare Clef family: Same state-plus-questions shape via Workers AI /ai/run; media extensions
Your data: Jev: Not used to train models (Privacy Policy); no retention period published | OpenAI Decisions API: Not used for training; abuse-monitoring logs up to 30 days | Microsoft-Decision-1: Not used to train base models; "temporarily store" for abuse monitoring; no period found | Cloudflare Clef family: Not used to train; no retention period on Cloudflare's data-usage page
Documented errors and limits: Jev: 401, 422, 429, 529; back off and retry; 80 requests/s, 100K tokens/s (errors checked 6 Oct 2026) | OpenAI Decisions API: Refusal answer type documented; no error codes or rate limits found for /v1/decisions | Microsoft-Decision-1: 400–429 table; 429: exponential backoff; 60–900 RPM by tier | Cloudflare Clef family: Platform errors only (408, 429); no Clef-specific rate limit found
Not tested by this pool: image input against Jev; Clef against Jev on accuracy; latency on each provider's own route; the pool ran no benchmark of its own.
No overall ranking; graded results are per task. Source: https://shaduf.ai/p/jev-catalog/alternatives/jev-vs-decision-apis/#dapi-card

Full table, each cell dated and sourced

"12 Oct" cells are carried from run 14 and were not re-read on 13 Oct. Quotes are the vendors' words; gateway labels (OpenRouter, Vercel) are attributed to the gateway. Source IDs are listed under Sources.

FactJevTypeSafeOpenAI Decisions APIOpenAIMicrosoft-Decision-1MicrosoftCloudflare Clef familyCloudflare
Graded tests ReportedB38 (A): Decisions API ahead on per-move accuracy, 59.4% vs 51.0%. B39 (A): Jev ahead on exact judge score, 80.7% vs 72.0%; routing (88.0% vs 86.7%) and injection F1 (96.6% vs 95.8%) not significant per the operator. B41 (A): route macro-F1 Jev 89.2%, Decision-1 87.6% ("not separated"), Decisions API 71.4%. B42 (B): on 1,000 five-option logic questions, Jev 817/1,000 (81.7%) and the Decisions API 612/1,000 (61.2%); Jev's answers reused from a 22 Sep run; answer key withheld. Ledger, checked 11 and 12 Oct; B42 re-pin checked 13 Oct 2026, 05:24 UTC (R15-S53)B38, B39, B41 as in the Jev cell (Decisions API side). B42 (B): Decisions API (openai/gpt-6-luna-decisions via OpenRouter's alpha Decisions endpoint) 612/1,000 (61.2%; 62.3% of 983 valid), behind Jev 817/1,000 (81.7%) on five-option logic questions; 17 persistent refusals (HTTP 502), cause unknown; answer key withheld (re-pin checked 13 Oct 2026, 05:24 UTC; R15-S53; files R14-S300 to R14-S305)B41 only (A): not separated from Jev on route macro-F1 (87.6% vs 89.2%), ahead of the Decisions API (71.4%); same route in all 6 option orders 97.2–97.6% (Jev 91.6–91.8%, Decisions API 64.2%)B32 (A): security experiments with Clef and Clef-flash as comparators; "the 2,048-token truncation finding concerns Clef, not Jev"; not an accuracy comparison. No graded accuracy comparison of Clef with Jev in the ledger (13 Oct)
Price as published Documented"$0.042" per million input tokens; "Output tokens are free" (TypeSafe models page, 13 Oct, 05:18 UTC; R15-S06). OpenRouter prompt 0.000000042, completion 0 (R15-S36)"input costs $0.10 per 1M tokens. You pay only for input tokens: there are no cache-read, cache-write, or output-token charges"; "Regional processing premiums and long-context input pricing multipliers apply" (guide, 13 Oct; R15-S213; R14-S219). Pricing page: no Decisions line (13 Oct; R15-S218). OpenRouter prompt 0.0000001 (13 Oct, 05:20 UTC; R15-S49)"Input tokens cost $0.042 USD per million tokens. Output tokens are free." (Microsoft's post; byte-identical 13 Oct, 05:20 UTC; R15-S50). OpenRouter and Vercel: input 0.000000042, output 0 (R15-S48, R15-S212). No Microsoft price page read (catalog pricing needs sign-in)Workers AI pricing page (Last updated Oct 9, 2026; 13 Oct, 05:25 UTC; R15-S224): Clef "$0.240 per M input tokens", Clef-flash "$0.038", Clef-omni "$0.150"; "Clef models do not charge for output tokens" (R14-S403). OpenRouter third-party hosts list other prices (see Routes)
Inputs Documented"Text only. String, JSON object, or array of text values. No image, audio, or video input." (models page, 13 Oct, 05:18 UTC; R15-S06)"evaluates text, images, or both" (guide; R14-S219); "at most 128 images are allowed across all messages in one request" (API reference, 13 Oct; R15-S214)"The model is text-only and does not accept or produce image, audio, or video content." (Foundry catalog, 13 Oct, 05:20 UTC; R15-S44)Text, JSON, images (all three; at most 4 embedded images, "Clef extension to the System One API"); Clef-omni adds audio (4 clips, 300 s) and video (2, 60 s) (Cloudflare, 12 Oct; R14-S400 to R14-S402)
Outputs DocumentedChoice (up to 255 options), Score (2–10 levels), Noul (yes probability) (limitations, checked 28 Sep)answers of type predicate, choice (with probabilities, confidence), score, or refusal; 2–255 choices per Choice question (guide, 12 Oct; API reference, 13 Oct; R14-S219, R15-S214)answers keyed by question: noul "A probability from 0 through 1"; choice "One selected option and the probability for each option"; score "A value on an ordered scale and the probability for each level"; no confidence field documented (Learn, Last updated 2026-10-09; R14-S208)model, answers keyed by question id, usage (12 Oct; R14-S400)
Request shape DocumentedPOST https://api.typesafe.ai/v1/systemone, Bearer key; body state, model, questions (where-to-use, rechecked 12 Oct)POST /v1/decisions; model (gpt-6-luna only), input, questions with unique names. Different from Jev's shape (guide; R14-S219, rechecked 13 Oct R15-S213)POST <your-foundry-resource-endpoint>/providers/microsoft/v1/systemone, Entra ID Bearer or api-key; body model (deployment name), state, named questions (noul, choice, score). Same state-plus-questions shape; endpoint and auth differ (Learn, Last updated 2026-10-09; R14-S208)env.AI.run("@cf/cloudflare/clef", …) or POST …/accounts/{ACCOUNT_ID}/ai/run/@cf/cloudflare/clef, Bearer token; body model, state, 1–64 questions (noul, choice, score). Same shape plus extensions; a Jev client does not work unchanged (12 Oct; R14-S400, R14-S402)
Context and size limits Documented64k tokens per request; 32k for state plus the longest question (models page, 13 Oct; R15-S06)Context window: not stated in the guide or API reference (13 Oct, 05:18 UTC; R15-S213, R15-S214). The API reference's maxLength 10485760 is a per-field limit, not a context window. Vercel's gateway metadata lists context 1,050,000 (R15-S212); OpenRouter 1,050,000 (R15-S49): gateway labels, not OpenAI's text32,768-token context (Foundry catalog, 13 Oct, 05:20 UTC, R15-S44; OpenRouter, R15-S48)Clef 65,536; Clef-flash 24,576 ("now 24K tokens, down from 64K"); Clef-omni 64,000 (Cloudflare, 12 Oct; R14-S400, R14-S405)
Rate limits Documented"100K tokens per second / 80 requests per second", "can change without notice" (models page, 13 Oct; R15-S06)Not published for /v1/decisions: the rate-limits guide does not name the Decisions API or gpt-6-luna-decisions (13 Oct, 05:18 UTC; R15-S215). Its "Luna" row is not stated to apply; not carriedRPM by subscription tier, Global Standard and Data Zone Standard alike: Tier 1 60, Tier 2 150, Tier 3 225, Tier 4 360, Tier 5 500, Tier 6 900. No tokens-per-minute figure (quotas page, Last updated 2026-10-06; 13 Oct; R15-S206)Not published for Clef: the Workers AI limits page does not name Clef (Last updated Sep 17, 2026; 13 Oct; R15-S222). Clef's model page files it under "Text Generation", whose default is 300 requests per minute; that is an inference from two pages, not a Clef figure (R15-S225)
Documented errors and retry advice Documented401, 422, 429, 529 documented; 429: "Back off and retry" (build/errors-and-rate-limits, checked 6 Oct)Refusal: "The model declined to answer this question. Other questions in the same request can still receive answers." (API reference, 13 Oct; R15-S214). Error codes, timeouts, retries: not found in the guide, API reference or error-codes guide (13 Oct, 05:18 UTC; R15-S213, R15-S214, R15-S216)400, 401, 403, 404, 422, 429 table; 429: "Retry with exponential backoff, or request more quota" (Learn, 12 Oct; R14-S208). Quickstart TIMEOUT_SECONDS = 60 is example code, not a service limitWorkers AI platform table, not Clef-specific: "Timeout | 3007 | 408"; "Out of capacity | 3040 | 429 | Capacity temporarily exceeded, please try again." (errors page, Last updated Sep 17, 2026; 13 Oct; R15-S223). Long text state is truncated to fit; media over the context fails the request (R14-S402)
Training on your data DocumentedPrivacy Policy: TypeSafe "will not train or fine tune any artificial intelligence or machine learning models on Input"; MCA 4.3: Telemetry "without restriction, including to improve the Services" (where-to-use/data-privacy; legal pages byte-identical 13 Oct; R15-S22)/v1/decisions "Data used for training: No" (data controls, 13 Oct; R15-S217)"prompts and completions are not used to train, retrain, or improve the base models." (data page, Last updated 2026-05-19; 13 Oct; R15-S203). OpenRouter's label for its Azure endpoint: training: false (R15-S210)"Cloudflare does not use your Customer Content to (1) train any AI models made available on Workers AI or (2) improve any Cloudflare or third-party services, and would not do so unless we received your explicit consent." (Data usage, Last updated Apr 21, 2026; 12 Oct; R14-S404)
Retention DocumentedNo retention period published; MCA 10.3 "no obligation to store or retain Customer Data"; DPA Annex §8 "retained for as long as necessary taking into account the purpose of the Processing"; ZDR "for enterprise customers" (where-to-use/data-privacy)Abuse monitoring retention "30 days"; "By default, abuse monitoring logs are retained for up to 30 days"; application state "None, see below for exceptions"; ZDR eligible "Yes, see below for limitations"; Private Retention "Pending confirmation"; flagged images retained "even if Zero Data Retention … is enabled" (13 Oct; R15-S217, R15-S54)Product Terms (Foundry Models sold by Azure, Effective 10/1/2026 view): "Microsoft will temporarily store Input and Output Content, to monitor for and prevent abusive or harmful uses or outputs of the service." The entry does not name Decision-1 (R15-S200, R15-S201). No period in days on the Product Terms, the model-specific terms, the data page or the abuse-monitoring page (13 Oct, 05:18–05:20 UTC; R15-S200 to R15-S204). OpenRouter labels its Azure endpoint retainsPrompts: false (R15-S210); quoted beside Microsoft's text, not reconciled"Your Customer Content for Workers AI may be stored by Cloudflare if you specifically use a storage service (e.g., R2, KV, DO, Vectorize, etc.) in conjunction with Workers AI." No retention period on that page (12 Oct; R14-S404)
Regions and data zones Documented"The Services are hosted in the United States"; no EU option found in TypeSafe's text (where-to-use/data-privacy)Guide: "Data residency and regional processing are supported in the United States and Europe (EEA + Switzerland)" (R14-S219). The data-controls regional table lists /v1/decisions for the United States, Europe, Australia, Canada, Japan, India and Singapore (13 Oct; R15-S217). Both shown; not merged"DataZoneStandard in selected regions; GlobalStandard" (Learn, R14-S208). The region-availability page (Last updated 2026-09-04) does not list Decision-1, so its data zones (US, EU or APAC) are not published there (13 Oct, 05:19 UTC; R15-S205). Human-review data "stored in the Azure geography where the customer's Foundry resource is deployed" (R15-S203)Not read for Clef (12 Oct)
Routes DocumentedTypeSafe direct, OpenRouter, Cloudflare, Vercel, Netlify, Opper; IDs rechecked 13 Oct, 05:18 UTC (R15-S06, R15-S12, R15-S36 to R15-S39)OpenAI direct; OpenRouter openai/gpt-6-luna-decisions (endpoint page, 13 Oct; R15-S49); Vercel AI Gateway openai/gpt-6-luna-decisions (13 Oct; R15-S212). Neither decision model is in OpenRouter's /models list (R15-S40)Foundry: listed as Preview in Foundry at 13 Oct 2026, 05:20 UTC (listed Generally available on 12 Oct; cause unknown) (R15-S44, R14-S44); OpenRouter microsoft/microsoft-decision-1, one provider, Azure (R15-S48); Vercel AI Gateway microsoft/microsoft-decision-1 (R15-S212)Workers AI; OpenRouter cloudflare/clef via PrimeIntellect (16,384 context), DeepInfra and Cloudflare; cloudflare/clef-flash via PrimeIntellect and Cloudflare; cloudflare/clef-omni via Cloudflare (13 Oct; R15-S226). Not on Vercel (R15-S212). Your data goes to Cloudflare under Cloudflare's terms on Workers AI; on OpenRouter's non-Cloudflare hosts (PrimeIntellect, DeepInfra) it goes to that host
Terms and licence DocumentedTypeSafe Terms (Last updated Sep 19, 2026), Privacy Policy, DPA; byte-identical 13 Oct (R15-S22)Public beta: "we expect to GA in the coming weeks" (13 Oct; R15-S213). OpenAI terms not readCatalog licence text: "Use of this Azure Direct Model is subject to supplemental Terms of Use for Microsoft Azure [https://azure.microsoft.com/en-us/support/legal/preview-supplemental-terms/]" (model page, 13 Oct, 05:20 UTC; R15-S44). Product Terms entry "Foundry Models sold by Azure" applies to models sold by Azure; it does not name Decision-1 (R15-S200). The model-specific terms page has no Decision-1 or Qwen entry (R15-S202). Base model per Microsoft: Qwen3.5-9B (R14-S209)Weights Apache-2.0 (12 Oct; R14-S406). Bases: Qwen3.8-27B and Qwen3.5-9B Apache-2.0 (card and LICENSE file); Qwen3-Omni-30B-A3B-Instruct card "license: other / license_name: apache-2.0", no LICENSE file in the repo (13 Oct; R15-S219 to R15-S221). Cloudflare's Self-Serve Subscription Agreement and Privacy Policy not read

What nobody has tested yet

What no graded study in the ledger covers (13 Oct 2026; ledger of 39 graded rows):

  1. Image input against Jev. Jev takes text only; OpenAI and Clef take images. No graded study compares image decisions.
  2. Clef against Jev on accuracy. B32 is security experiments. Cloudflare's own benchmark table is Reported and its materials were not opened.
  3. Microsoft-Decision-1 beyond one task. One graded study (B41: synthetic bank-withdrawal routing, all three models via OpenRouter).
  4. OpenAI called on its native /v1/decisions format alongside Decision-1. B41 called all three through OpenRouter's /api/v1/systemone.
  5. Latency on each provider's own route. Operators' latency figures (B38, B41) are single-location measurements and are not graded.
  6. Languages other than English and Portuguese (B39 is Portuguese).
  7. The pool ran no benchmark of its own and has no account with any provider.

What the pool did not find or read

  • Documented Decision-1 retention period in days, its data zones (EU?), a tokens-per-minute limit and a Microsoft price page: not found on the Product Terms entry, the model-specific terms, the data, abuse-monitoring, region-availability or quotas pages (13 Oct 2026, 05:18–05:20 UTC).
  • Documented OpenAI /v1/decisions rate limits, context window, error codes, timeouts and retries: not found in the guide, API reference, rate-limits guide or error-codes guide (13 Oct 2026, 05:18 UTC). OpenAI's terms not read.
  • Documented Clef rate limit and Paid-plan status: not found on the Workers AI limits page or the Clef model page (13 Oct 2026, 05:18–05:21 UTC). Cloudflare's subscription agreement and privacy policy not read.
  • Jev: retention period and EU region, as on What data leaves your system.

Comparisons not graded (leads on the benchmarks lead list; never shown with a grade): utakatano/decision-models-demo (empty repository at 13 Oct 2026, 05:22 UTC; lead, not graded; R15-S227, R15-S228); new on 12 Oct, from the pool's GitHub and Hacker News scan of 13 Oct 2026 (pins not taken): dante01yoon/decision-model-arena (Decisions API vs Decision-1 vs Jev vs Laya), egarim/decision-1-vs-jev, leanmcp JevBench (17 decision models incl. Clef; R15-S34), GOhaike/decision-layer-benchmark, kprzybys/snap-vs-jev-swe.

Also launched, not compared here

SubQ Snap (Subquadratic), a hosted decision API, launched per the vendor's post (read 13 Oct 2026, 05:22 UTC; R15-S33). Documented by the vendor: "$0.05 per million input tokens, with no charge for output tokens"; "Public Preview on a first come, first served basis … new users may be added to a waitlist"; "up to 384K tokens of text and images". Reported vendor claims against Jev: "ranks four points above Jev on the Decision Index v0.2.1"; "Snap averages 67.6 versus Jev's 61.6 and leads on 13 of 16 tasks" (Subquadratic's measurement; not graded). Its API docs, request shape and data terms were not read by the pool (13 Oct 2026). No column or row here; listed under "seen, not checked" on Jev alternatives.

What the vendors claim Reported

Vendor measurements, attributed; outside the graded rows and not graded by the pool.

  • Microsoft (post of 9 Oct 2026, updated with an editor's note, update date not shown; charts decoded by a pool helper, 13 Oct 2026, 05:18 UTC; R15-S209): accuracy, "mean score across 36 benchmarks (147,137 questions)": Microsoft-Decision-1 83.5, Jev 1.13.0 82.3, GPT-6 Luna Decisions 79.4. Calibration ("100 means a model's confidence exactly matches how often it is right"): Jev 1.13.0 93.7, Microsoft-Decision-1 92.2, GPT-6 Luna Decisions 89.9; Microsoft writes "Third on calibration, 1.5 behind Jev." Median latency (JevBench v1.6.1 adjusted p50, "checked Oct 7, 2026"; Decision-1 "measured through Foundry in the same region"): 85, 240 and 300 ms. Also: Decision-1 "changes its decision on 1.3% of perturbations on average …" and "P50 latency is ~35x faster than GPT-6 Sol" (R14-S207). Microsoft's measurement; not graded by the pool.
  • Cloudflare (changelog, 9 Oct 2026; R14-S405): a benchmark table listing Clef-omni, Clef, Clef-flash and Jev; Clef-flash "now costs less than Jev". Not graded; the pool makes no per-decision cost comparison.
  • OpenAI: no comparative claim recorded on the pages read (13 Oct 2026, 05:18–05:25 UTC).

Changed since

New page, 13 Oct 2026; rows first checked 12 Oct 2026 (Alternatives), rechecked or added 13 Oct: Microsoft-Decision-1 Foundry lifecycle "Generally available (GA)" → "Preview" (cause unknown); Decision-1 requests-per-minute limits by tier added; Microsoft Product Terms read (Decision-1 not named; "temporarily store"); Decision-1 catalog licence text added; OpenAI refusal behaviour and 128-image limit added; OpenAI regional table (7 regions) added beside the guide's US and Europe; Vercel routes for Decision-1 and the Decisions API added; Clef on OpenRouter, including PrimeIntellect and DeepInfra hosts, added; Clef base-model licences added; prices unchanged. B42 added (grade B), below B38, B39 and B41.

Sources

Every source ID cited on this page, with the address read and the read time. R14 = run of 12 Oct 2026; R15 = run of 13 Oct 2026. Benchmark files for B38, B39, B41 and B32 are listed on benchmarks. No account, key or provider API call was used; the pool ran nothing.

Search Shaduf

Search published pools, pages, reports, and evidence.