@typesafe-ai/sdk 0.6.0 · ai 7.0.122 · checked , 07:14–07:16 UTCCall Jev (TypeSafe AI) from TypeScript: AI SDK experimental_evaluate vs @typesafe-ai/sdk vs fetch
There are four ways to call Jev from TypeScript: TypeSafe's own SDK, the Vercel AI SDK's experimental_evaluate, a plain HTTP request, and the Cloudflare Workers binding. They differ in runtime, key variable, field names, default retries and timeouts. This page compares them at pinned versions and lists twelve TypeScript-specific mistakes.
For TypeSafe direct on Node 20 or later, use the official @typesafe-ai/sdk 0.6.0. If you use Vercel AI Gateway or already use the AI SDK, use AI SDK experimental_evaluate (ai 7.0.105 or later). For other runtimes or routes, use raw fetch and add your own retry and timeout. Documented package sources and docs, checked 29 Sep 2026
Version drift, 2 Oct 2026 (updated 7 Oct)
Newer releases appeared after this page's audit: ai 7.0.123 to 7.0.126 (30 Sep) and now 7.0.127 (1 Oct, 19:20 UTC), and @ai-sdk/typesafe-ai 3.0.11 and 3.0.12 (30 Sep, 02:20 and 17:49 UTC) (npm: ai, npm: @ai-sdk/typesafe-ai, checked 2 Oct 2026, 05:14 UTC). This page's audit used ai 7.0.122 and @ai-sdk/typesafe-ai 3.0.10. The new releases were not re-audited: the defaults, field names and gotchas below apply to the audited versions. @typesafe-ai/sdk is unchanged at 0.6.0 and @ai-sdk/typesafe-ai at 3.0.12 (checked 2 Oct 2026, 05:14 UTC). Documented
6 Oct 2026, 05:15 UTC: newer patch releases: ai 7.0.128 and @ai-sdk/typesafe-ai 3.0.13, 5 Oct 2026 (18:25 and 18:26 UTC; were 7.0.127 and 3.0.12); not re-audited. @typesafe-ai/sdk is still 0.6.0. Documented (npm: ai, R9-S36; npm: @ai-sdk/typesafe-ai, R9-S37; npm: @typesafe-ai/sdk, R9-S35)
7 Oct 2026, 05:19 UTC: newer patch releases: ai 7.0.130 and @ai-sdk/typesafe-ai 3.0.15, 7 Oct 2026 (01:47 and 01:41 UTC; 7.0.129 and 3.0.14 came out on 6 Oct, 22:23 and 22:20 UTC; were 7.0.128 and 3.0.13); not re-audited. @typesafe-ai/sdk is still 0.6.0. Documented (npm: ai, npm: @ai-sdk/typesafe-ai and npm: @typesafe-ai/sdk, R10-S35)
- Official SDK
@typesafe-ai/sdk0.6.0, npm 15 Sep 2026, 18:17 UTC; tag66880cc; MIT- AI SDK core
ai7.0.122, 28 Sep 2026, 21:59 UTC; Apache-2.0- First
aiwithexperimental_evaluate - 7.0.103 (16 Sep) by package contents; Vercel's guide says 7.0.105
- AI SDK direct provider
@ai-sdk/typesafe-ai3.0.10, 28 Sep 2026, 22:00 UTC- Node
- SDK ≥ 20;
aiand@ai-sdk/typesafe-ai≥ 22
Which path fits your project
| If your project… | Use | Because |
|---|---|---|
| Runs on Node 20 or 21 | @typesafe-ai/sdk or raw fetch | ai declares Node ≥ 22 |
| Calls Jev through Vercel AI Gateway | experimental_evaluate with 'typesafe-ai/jev' | The gateway string and AI_GATEWAY_API_KEY (or Vercel OIDC) are built in |
| Wants typed helpers and mapped error classes for TypeSafe direct | @typesafe-ai/sdk | choice(), score(), noul() and one error class per status |
| Runs on Cloudflare Workers | env.AI.run('typesafe/jev', …) | No package or TypeSafe key; the account binding authenticates |
| Uses another route, runtime or a gateway without an SDK | Raw fetch | Stable HTTP contract; you write retry, timeout and parsing |
| Must not stop working after a minor upgrade | Pin exact versions | experimental_evaluate "may change in patch releases"; the SDK is 0.x |
Comparison at pinned versions
Documented SDK source at tag v0.6.0 (retry.ts, errors.ts, client.ts); AI SDK evaluate reference, evaluation guide and TypeSafe provider page; Cloudflare model page; all checked 29 Sep 2026
| Field | @typesafe-ai/sdk (official) | AI SDK experimental_evaluate | Raw fetch | Cloudflare Workers binding |
|---|---|---|---|---|
| Install | npm install @typesafe-ai/sdk | npm i ai, plus @ai-sdk/typesafe-ai for TypeSafe direct | Nothing | env.AI binding in the wrangler config |
| Runtime | Node ≥ 20; ESM, CJS and types. Refuses to run in a browser unless dangerouslyAllowBrowser: true | Node ≥ 22 | Any runtime with fetch | Cloudflare Workers |
| Routes | TypeSafe direct; baseURL (TYPESAFE_BASE_URL) for compatible endpoints | Gateway string 'typesafe-ai/jev', or typeSafeAi.evaluationModel('jev-latest') for TypeSafe direct | Any System One endpoint | Cloudflare typesafe/jev |
| Key variable | TYPESAFE_API_KEY | Gateway: AI_GATEWAY_API_KEY. Direct provider: TYPESAFE_AI_API_KEY | Your choice | Account binding |
| Answer fields | Choice choice, probabilities, confidence; Score score, legend, probabilities, confidence; Noul noul; usage.input_tokens, usage.output_tokens; model | Choice .choice; Score .score; Noul is called boolean and returns .probability; confidence at providerMetadata.typesafe.confidence[id]; usage.inputTokens; warnings; response.modelId | As the API reference | Not re-read on 29 Sep (the page shows console.log(response) only) |
| Default retries | 2 retries on 408, 429, 5xx (including 529), connection errors and timeouts; backoff 0.5 s doubling to 5 s with up to 25% jitter; honours retry-after-ms and Retry-After up to 60 s | maxRetries 2 in Core for transient failures (e.g. 429, 529); malformed answers are not retried | None | Not documented on the page read |
| Default timeout | 10 s per attempt; no total budget found in the JS source (the Python SDK has a 30 s budget) | None documented; pass abortSignal | None | Not documented on the page read |
| Errors | BadRequestError 400, AuthenticationError 401, PermissionDeniedError 403, NotFoundError 404, UnprocessableEntityError 422, RateLimitError 429, InternalServerError ≥ 500, APIConnectionError, APITimeoutError, APIUserAbortError. A missing key throws at construction | APICallError (provider failures incl. auth and validation), InvalidArgumentError, InvalidResponseDataError (malformed answers), Experimental_EvaluationUnsupportedQuestionTypeError, NoSuchModelError | Whatever you map | Workers AI error codes (see errors page) |
| Test facility | fetch option: "Custom HTTP fetch implementation for transport configuration or tests" | Experimental_EvaluationMockModelV4 from ai/test | A local mock server | Not documented |
| Stability | 0.x; nine open issues (below) | Experimental: "may change in patch releases" | Stable HTTP contract | Route-owner page |
Two things that differ between sources
- First AI SDK version.
experimental_evaluateis absent from the type declarations ofai7.0.102 and present in 7.0.103 (published 16 Sep 2026, 18:36 UTC). Vercel's guide says "Install AI SDK 7.0.105 or later". This page uses 7.0.105 as the minimum and records the conflict. Documented package contents read, not installed; 29 Sep - Key variable name. The official SDK reads
TYPESAFE_API_KEY. The AI SDK's direct provider readsTYPESAFE_AI_API_KEY. A key set under the wrong name is a "missing key" error, not a Jev error. Documented SDK env.ts; ai-sdk.dev provider page; 29 Sep
Minimal Choice call per path
Each example asks one Choice question about a support message. The code follows each owner's published example and has not been run by the pool, which has no API key. Model IDs differ by route; the current IDs are in the route matrix.
1. @typesafe-ai/sdk 0.6.0 Documented, not executed SDK README and docs JavaScript page
// npm install @typesafe-ai/sdk@0.6.0 (Node >= 20, server side only)
// export TYPESAFE_API_KEY=...
import { TypeSafeClient, choice } from "@typesafe-ai/sdk";
const client = new TypeSafeClient(); // throws at once if the key is missing
const response = await client.systemOne({
state: "I was charged twice for my subscription this month.",
questions: {
category: choice("What is this ticket about?", { billing: null, technical: null, other: null }),
},
});
console.log(response.answers.category.choice, response.answers.category.confidence);
console.log(response.model); // log which versioned model answered
The documented example sets no model, so the SDK default alias jev-latest applies. What jev-latest points to, and which ID to pin on each route: Jev model IDs and aliases by route.
2. AI SDK experimental_evaluate through Vercel AI Gateway Documented, not executed ai-sdk.dev evaluation guide and TypeSafe provider page
// npm i ai@7.0.122 (Node >= 22; 7.0.105 or later per Vercel)
// export AI_GATEWAY_API_KEY=...
import { experimental_evaluate } from "ai";
const result = await experimental_evaluate({
model: "typesafe-ai/jev",
state: "I was charged twice for my subscription this month.",
questions: {
department: {
type: "choice",
instructions: "Which team should handle this?",
criteria: { billing: "Payments and invoices", technical: "Bugs and integrations", other: null },
},
},
});
console.log(result.answers.department.choice);
console.log(result.providerMetadata?.typesafe?.confidence?.department); // confidence is not on the answer
For TypeSafe direct instead of the gateway, install @ai-sdk/typesafe-ai, set TYPESAFE_AI_API_KEY, and pass model: typeSafeAi.evaluationModel("jev-latest") (base URL https://api.typesafe.ai/v1).
3. Raw fetch Documented, not executed request shape from the docs quickstart; the timeout line is the pool's addition
const res = await fetch("https://api.typesafe.ai/v1/systemone", {
method: "POST",
headers: { Authorization: `Bearer ${process.env.TYPESAFE_API_KEY}`, "Content-Type": "application/json" },
body: JSON.stringify({
model: "jev-1.13.0",
state: "I was charged twice for my subscription this month.",
questions: {
category: { type: "choice", instructions: "What is this ticket about?",
criteria: { billing: null, technical: null, other: null } },
},
}),
signal: AbortSignal.timeout(10_000), // raw fetch has no timeout unless you set one
});
if (!res.ok) throw new Error(`Jev ${res.status}: ${await res.text()}`);
const body = await res.json();
console.log(body.answers.category.choice, body.answers.category.confidence);
A retry loop for this path, tested offline against a local mock server, is the worked example on Testing without a key.
4. Cloudflare Workers binding Documented, not executed Cloudflare model page for typesafe/jev
export default {
async fetch(request, env) {
const response = await env.AI.run("typesafe/jev", {
state: "I was charged twice for my subscription this month.",
questions: {
department: { type: "choice", instructions: "Which team should handle this?",
criteria: { billing: null, technical: null, other: null } },
},
});
return Response.json(response);
},
};
How long a call can take with default settings
With the official SDK's defaults and an endpoint that never answers, one call makes 3 attempts of 10 s each, with 0.5 s and 1 s waits between them. That adds up to about 31.5 s. The Claude Code plugins audited on Claude Code and MCP give a hook 10–15 s. Set a per-call timeout shorter than your caller's deadline.
One call to a hanging endpoint, 0–31.5 s
SDK defaults: 3 × 10 s attempts, 0.5 s and 1 s backoff (derived arithmetic)
Red line: a 15 s hook deadline. The caller gives up during the second attempt, while the SDK is still retrying.
Same retry pattern at 800 ms per attempt, Tested offline 3.9 s
The pool's offline test (T04) measured 3,908 ms for this case against a local mock server.
TypeScript gotchas
Status codes and their meanings are on Errors and rate limits. This table lists only problems specific to the TypeScript paths.
| # | Gotcha | What to do | Source and label |
|---|---|---|---|
| TS1 | Question type must be lowercase (choice, score, noul); capitalised types return 400 | Use the SDK helpers choice(), score(), noul() | oh-my-claudecode #4091 Reported |
| TS2 | state: null returns 422, and the JS SDK does not stop it locally | Validate state before calling | typesafe-sdk-js #6 Reported |
| TS3 | The model ID differs by route: jev-1.13.0 or jev-latest direct; typesafe-ai/jev on Vercel; typesafe/jev-1.13 on OpenRouter; typesafe/jev on Cloudflare | Keep the ID in configuration per route; check the model IDs and aliases by route | Route pages Documented 29 Sep |
| TS4 | Three key variables: TYPESAFE_API_KEY (official SDK), TYPESAFE_AI_API_KEY (AI SDK direct provider), AI_GATEWAY_API_KEY (gateway) | Set the one your path reads. The official SDK throws at construction if it is missing | SDK env.ts L1–11; ai-sdk.dev Documented |
| TS5 | The AI SDK calls a Noul boolean and returns .probability, not .noul; confidence sits in providerMetadata.typesafe.confidence | Do not port field names between paths | ai-sdk.dev Documented |
| TS6 | Double retries: the SDK and the AI SDK each retry twice by default; a wrapper that retries on top multiplies calls and cost | Set maxRetries: 0 (AI SDK) or retry: { maxRetries: 0 } in the lower layer when you retry yourself | retry.ts; ai-sdk.dev Documented |
| TS7 | Worst case with SDK defaults and a hanging endpoint is about 31.5 s (see the figure) | Pass a per-call timeout or abortSignal shorter than your caller's deadline | Derived from retry.ts; T04 shows the pattern at 800 ms |
| TS8 | Vercel's free tier returns 429 rate_limit_exceeded | Handle it separately from TypeSafe's own 429 | Vercel rate-limit docs Documented (28 Sep) |
| TS9 | The SDK throws in a browser unless dangerouslyAllowBrowser: true, and that setting would expose the key | Call Jev from a server route | client.ts L60–65, L268 Documented |
| TS10 | Cancelling a call can terminate Node 20 and 22 (open issue #2); jkudish/jev-mcp routes around it | Avoid aborting in-flight SDK calls on those Node versions, or use raw fetch | typesafe-sdk-js #2; jkudish README L757 Reported |
| TS11 | experimental_evaluate "may change in patch releases" | Pin the exact ai version | ai-sdk.dev Documented |
| TS12 | Probabilities are rounded to 2 decimals and may not sum to 1 | Do not assert exact sums in tests | ai-sdk.dev provider page Documented |
Open issues in the official JS SDK
Open on GitHub on 29 Sep 2026. Each is the reporter's description, not rerun by the pool. Reported
- #2: a handled cancellation can crash Node 20 and 22.
- #6:
noul()with no arguments, andstate: null, build requests the API rejects. - #8: timeouts above Node's timer maximum fire after about 1 ms.
- #9: a blank
Retry-Afterheader bypasses backoff. - #12:
score()accepts a null level, which returns 422. - #13: 402 and 413 fall through to a generic
APIError. - #14: the API key can be echoed into
APIConnectionError, and an empty key is sent. - #15: Cloudflare HTML 403 when the state contains
curltext. - #17: non-finite numbers serialise as null.
Issue list: typesafe-ai/typesafe-sdk-js issues.
Projects that use these paths
| Project | What its code does | Label and date |
|---|---|---|
Jevmail classify.ts | Calls experimental_evaluate({ model: 'typesafe-ai/jev', …, maxRetries: 0 }), then runs its own 429 wait-and-retry-once and 5xx retry-once. This avoids stacking the AI SDK's 2 retries under its own (TS6). | Documented 28 Sep audit, not re-audited |
Vercel template jev-ai-sdk-form-router | CONFIDENCE_THRESHOLD = 0.95. The Jev call uses AbortSignal.timeout(12_000) and maxRetries: 1. If confidence is below 0.95 or missing, or Jev fails, it calls a fallback LLM (25 s, 1 retry). A missing confidence counts as a fallback, not a pass. package.json pins ai 7.0.107. The fallback model is openai/gpt-6-luna-fast in source but openai/gpt-5.6-luna-fast on the template page. | Documented at 27ea469 (22 Sep), checked 29 Sep |
MSFT-TKENDRICK/JEV-examples | The author states the examples were never executed live. Not opened by the pool. | Reported |
Seen only: TanStack AI decide() with vercelGatewayDecider('typesafe-ai/jev') or cloudflareDecider('typesafe/jev'), described by Vercel. LangChain JS (@langchain/typesafe 0.0.2) and @effect/ai-typesafe 4.0.1 were read on 5 Oct 2026: see framework integrations. For TypeScript demo videos, see the TypeScript group on the videos page.
What was not verified
- No example on this page was run. The official JS SDK was not installed, and the AI SDK was read from its published type declarations and docs.
- The AI SDK's default timeout is undocumented. The changes in
@ai-sdk/typesafe-ai3.0.9 and 3.0.10 (both 28 Sep) were not compared. - The Cloudflare binding's response shape, retries and timeout were not re-read on 29 Sep.
- The 31.5 s worst case is arithmetic from SDK defaults, not a measurement.