Shaduf.Research preview
Build@typesafe-ai/sdk 0.6.0 · ai 7.0.122 · checked , 07:14–07:16 UTC

Call Jev (TypeSafe AI) from TypeScript: AI SDK experimental_evaluate vs @typesafe-ai/sdk vs fetch

There are four ways to call Jev from TypeScript: TypeSafe's own SDK, the Vercel AI SDK's experimental_evaluate, a plain HTTP request, and the Cloudflare Workers binding. They differ in runtime, key variable, field names, default retries and timeouts. This page compares them at pinned versions and lists twelve TypeScript-specific mistakes.

Short answer

For TypeSafe direct on Node 20 or later, use the official @typesafe-ai/sdk 0.6.0. If you use Vercel AI Gateway or already use the AI SDK, use AI SDK experimental_evaluate (ai 7.0.105 or later). For other runtimes or routes, use raw fetch and add your own retry and timeout. Documented package sources and docs, checked 29 Sep 2026

Version drift, 2 Oct 2026 (updated 7 Oct)

Newer releases appeared after this page's audit: ai 7.0.123 to 7.0.126 (30 Sep) and now 7.0.127 (1 Oct, 19:20 UTC), and @ai-sdk/typesafe-ai 3.0.11 and 3.0.12 (30 Sep, 02:20 and 17:49 UTC) (npm: ai, npm: @ai-sdk/typesafe-ai, checked 2 Oct 2026, 05:14 UTC). This page's audit used ai 7.0.122 and @ai-sdk/typesafe-ai 3.0.10. The new releases were not re-audited: the defaults, field names and gotchas below apply to the audited versions. @typesafe-ai/sdk is unchanged at 0.6.0 and @ai-sdk/typesafe-ai at 3.0.12 (checked 2 Oct 2026, 05:14 UTC). Documented

6 Oct 2026, 05:15 UTC: newer patch releases: ai 7.0.128 and @ai-sdk/typesafe-ai 3.0.13, 5 Oct 2026 (18:25 and 18:26 UTC; were 7.0.127 and 3.0.12); not re-audited. @typesafe-ai/sdk is still 0.6.0. Documented (npm: ai, R9-S36; npm: @ai-sdk/typesafe-ai, R9-S37; npm: @typesafe-ai/sdk, R9-S35)

7 Oct 2026, 05:19 UTC: newer patch releases: ai 7.0.130 and @ai-sdk/typesafe-ai 3.0.15, 7 Oct 2026 (01:47 and 01:41 UTC; 7.0.129 and 3.0.14 came out on 6 Oct, 22:23 and 22:20 UTC; were 7.0.128 and 3.0.13); not re-audited. @typesafe-ai/sdk is still 0.6.0. Documented (npm: ai, npm: @ai-sdk/typesafe-ai and npm: @typesafe-ai/sdk, R10-S35)

Official SDK
@typesafe-ai/sdk 0.6.0, npm 15 Sep 2026, 18:17 UTC; tag 66880cc; MIT
AI SDK core
ai 7.0.122, 28 Sep 2026, 21:59 UTC; Apache-2.0
First ai with experimental_evaluate
7.0.103 (16 Sep) by package contents; Vercel's guide says 7.0.105
AI SDK direct provider
@ai-sdk/typesafe-ai 3.0.10, 28 Sep 2026, 22:00 UTC
Node
SDK ≥ 20; ai and @ai-sdk/typesafe-ai ≥ 22

Which path fits your project

If your project…UseBecause
Runs on Node 20 or 21@typesafe-ai/sdk or raw fetchai declares Node ≥ 22
Calls Jev through Vercel AI Gatewayexperimental_evaluate with 'typesafe-ai/jev'The gateway string and AI_GATEWAY_API_KEY (or Vercel OIDC) are built in
Wants typed helpers and mapped error classes for TypeSafe direct@typesafe-ai/sdkchoice(), score(), noul() and one error class per status
Runs on Cloudflare Workersenv.AI.run('typesafe/jev', …)No package or TypeSafe key; the account binding authenticates
Uses another route, runtime or a gateway without an SDKRaw fetchStable HTTP contract; you write retry, timeout and parsing
Must not stop working after a minor upgradePin exact versionsexperimental_evaluate "may change in patch releases"; the SDK is 0.x

Comparison at pinned versions

Documented SDK source at tag v0.6.0 (retry.ts, errors.ts, client.ts); AI SDK evaluate reference, evaluation guide and TypeSafe provider page; Cloudflare model page; all checked 29 Sep 2026

Field@typesafe-ai/sdk (official)AI SDK experimental_evaluateRaw fetchCloudflare Workers binding
Installnpm install @typesafe-ai/sdknpm i ai, plus @ai-sdk/typesafe-ai for TypeSafe directNothingenv.AI binding in the wrangler config
RuntimeNode ≥ 20; ESM, CJS and types. Refuses to run in a browser unless dangerouslyAllowBrowser: trueNode ≥ 22Any runtime with fetchCloudflare Workers
RoutesTypeSafe direct; baseURL (TYPESAFE_BASE_URL) for compatible endpointsGateway string 'typesafe-ai/jev', or typeSafeAi.evaluationModel('jev-latest') for TypeSafe directAny System One endpointCloudflare typesafe/jev
Key variableTYPESAFE_API_KEYGateway: AI_GATEWAY_API_KEY. Direct provider: TYPESAFE_AI_API_KEYYour choiceAccount binding
Answer fieldsChoice choice, probabilities, confidence; Score score, legend, probabilities, confidence; Noul noul; usage.input_tokens, usage.output_tokens; modelChoice .choice; Score .score; Noul is called boolean and returns .probability; confidence at providerMetadata.typesafe.confidence[id]; usage.inputTokens; warnings; response.modelIdAs the API referenceNot re-read on 29 Sep (the page shows console.log(response) only)
Default retries2 retries on 408, 429, 5xx (including 529), connection errors and timeouts; backoff 0.5 s doubling to 5 s with up to 25% jitter; honours retry-after-ms and Retry-After up to 60 smaxRetries 2 in Core for transient failures (e.g. 429, 529); malformed answers are not retriedNoneNot documented on the page read
Default timeout10 s per attempt; no total budget found in the JS source (the Python SDK has a 30 s budget)None documented; pass abortSignalNoneNot documented on the page read
ErrorsBadRequestError 400, AuthenticationError 401, PermissionDeniedError 403, NotFoundError 404, UnprocessableEntityError 422, RateLimitError 429, InternalServerError ≥ 500, APIConnectionError, APITimeoutError, APIUserAbortError. A missing key throws at constructionAPICallError (provider failures incl. auth and validation), InvalidArgumentError, InvalidResponseDataError (malformed answers), Experimental_EvaluationUnsupportedQuestionTypeError, NoSuchModelErrorWhatever you mapWorkers AI error codes (see errors page)
Test facilityfetch option: "Custom HTTP fetch implementation for transport configuration or tests"Experimental_EvaluationMockModelV4 from ai/testA local mock serverNot documented
Stability0.x; nine open issues (below)Experimental: "may change in patch releases"Stable HTTP contractRoute-owner page

Two things that differ between sources

  • First AI SDK version. experimental_evaluate is absent from the type declarations of ai 7.0.102 and present in 7.0.103 (published 16 Sep 2026, 18:36 UTC). Vercel's guide says "Install AI SDK 7.0.105 or later". This page uses 7.0.105 as the minimum and records the conflict. Documented package contents read, not installed; 29 Sep
  • Key variable name. The official SDK reads TYPESAFE_API_KEY. The AI SDK's direct provider reads TYPESAFE_AI_API_KEY. A key set under the wrong name is a "missing key" error, not a Jev error. Documented SDK env.ts; ai-sdk.dev provider page; 29 Sep

Minimal Choice call per path

Each example asks one Choice question about a support message. The code follows each owner's published example and has not been run by the pool, which has no API key. Model IDs differ by route; the current IDs are in the route matrix.

1. @typesafe-ai/sdk 0.6.0 Documented, not executed SDK README and docs JavaScript page

// npm install @typesafe-ai/sdk@0.6.0        (Node >= 20, server side only)
// export TYPESAFE_API_KEY=...
import { TypeSafeClient, choice } from "@typesafe-ai/sdk";

const client = new TypeSafeClient();          // throws at once if the key is missing
const response = await client.systemOne({
  state: "I was charged twice for my subscription this month.",
  questions: {
    category: choice("What is this ticket about?", { billing: null, technical: null, other: null }),
  },
});

console.log(response.answers.category.choice, response.answers.category.confidence);
console.log(response.model);                  // log which versioned model answered

The documented example sets no model, so the SDK default alias jev-latest applies. What jev-latest points to, and which ID to pin on each route: Jev model IDs and aliases by route.

2. AI SDK experimental_evaluate through Vercel AI Gateway Documented, not executed ai-sdk.dev evaluation guide and TypeSafe provider page

// npm i ai@7.0.122                          (Node >= 22; 7.0.105 or later per Vercel)
// export AI_GATEWAY_API_KEY=...
import { experimental_evaluate } from "ai";

const result = await experimental_evaluate({
  model: "typesafe-ai/jev",
  state: "I was charged twice for my subscription this month.",
  questions: {
    department: {
      type: "choice",
      instructions: "Which team should handle this?",
      criteria: { billing: "Payments and invoices", technical: "Bugs and integrations", other: null },
    },
  },
});

console.log(result.answers.department.choice);
console.log(result.providerMetadata?.typesafe?.confidence?.department);  // confidence is not on the answer

For TypeSafe direct instead of the gateway, install @ai-sdk/typesafe-ai, set TYPESAFE_AI_API_KEY, and pass model: typeSafeAi.evaluationModel("jev-latest") (base URL https://api.typesafe.ai/v1).

3. Raw fetch Documented, not executed request shape from the docs quickstart; the timeout line is the pool's addition

const res = await fetch("https://api.typesafe.ai/v1/systemone", {
  method: "POST",
  headers: { Authorization: `Bearer ${process.env.TYPESAFE_API_KEY}`, "Content-Type": "application/json" },
  body: JSON.stringify({
    model: "jev-1.13.0",
    state: "I was charged twice for my subscription this month.",
    questions: {
      category: { type: "choice", instructions: "What is this ticket about?",
                  criteria: { billing: null, technical: null, other: null } },
    },
  }),
  signal: AbortSignal.timeout(10_000),        // raw fetch has no timeout unless you set one
});
if (!res.ok) throw new Error(`Jev ${res.status}: ${await res.text()}`);
const body = await res.json();
console.log(body.answers.category.choice, body.answers.category.confidence);

A retry loop for this path, tested offline against a local mock server, is the worked example on Testing without a key.

4. Cloudflare Workers binding Documented, not executed Cloudflare model page for typesafe/jev

export default {
  async fetch(request, env) {
    const response = await env.AI.run("typesafe/jev", {
      state: "I was charged twice for my subscription this month.",
      questions: {
        department: { type: "choice", instructions: "Which team should handle this?",
                      criteria: { billing: null, technical: null, other: null } },
      },
    });
    return Response.json(response);
  },
};

How long a call can take with default settings

With the official SDK's defaults and an endpoint that never answers, one call makes 3 attempts of 10 s each, with 0.5 s and 1 s waits between them. That adds up to about 31.5 s. The Claude Code plugins audited on Claude Code and MCP give a hook 10–15 s. Set a per-call timeout shorter than your caller's deadline.

One call to a hanging endpoint, 0–31.5 s

SDK defaults: 3 × 10 s attempts, 0.5 s and 1 s backoff (derived arithmetic)

Red line: a 15 s hook deadline. The caller gives up during the second attempt, while the SDK is still retrying.

Same retry pattern at 800 ms per attempt, Tested offline 3.9 s

The pool's offline test (T04) measured 3,908 ms for this case against a local mock server.

Blue: attempts; gray: backoff waits. The 31.5 s figure is derived from retry.ts defaults, not measured. Documented defaults, 29 Sep 2026.

TypeScript gotchas

Status codes and their meanings are on Errors and rate limits. This table lists only problems specific to the TypeScript paths.

#GotchaWhat to doSource and label
TS1Question type must be lowercase (choice, score, noul); capitalised types return 400Use the SDK helpers choice(), score(), noul()oh-my-claudecode #4091 Reported
TS2state: null returns 422, and the JS SDK does not stop it locallyValidate state before callingtypesafe-sdk-js #6 Reported
TS3The model ID differs by route: jev-1.13.0 or jev-latest direct; typesafe-ai/jev on Vercel; typesafe/jev-1.13 on OpenRouter; typesafe/jev on CloudflareKeep the ID in configuration per route; check the model IDs and aliases by routeRoute pages Documented 29 Sep
TS4Three key variables: TYPESAFE_API_KEY (official SDK), TYPESAFE_AI_API_KEY (AI SDK direct provider), AI_GATEWAY_API_KEY (gateway)Set the one your path reads. The official SDK throws at construction if it is missingSDK env.ts L1–11; ai-sdk.dev Documented
TS5The AI SDK calls a Noul boolean and returns .probability, not .noul; confidence sits in providerMetadata.typesafe.confidenceDo not port field names between pathsai-sdk.dev Documented
TS6Double retries: the SDK and the AI SDK each retry twice by default; a wrapper that retries on top multiplies calls and costSet maxRetries: 0 (AI SDK) or retry: { maxRetries: 0 } in the lower layer when you retry yourselfretry.ts; ai-sdk.dev Documented
TS7Worst case with SDK defaults and a hanging endpoint is about 31.5 s (see the figure)Pass a per-call timeout or abortSignal shorter than your caller's deadlineDerived from retry.ts; T04 shows the pattern at 800 ms
TS8Vercel's free tier returns 429 rate_limit_exceededHandle it separately from TypeSafe's own 429Vercel rate-limit docs Documented (28 Sep)
TS9The SDK throws in a browser unless dangerouslyAllowBrowser: true, and that setting would expose the keyCall Jev from a server routeclient.ts L60–65, L268 Documented
TS10Cancelling a call can terminate Node 20 and 22 (open issue #2); jkudish/jev-mcp routes around itAvoid aborting in-flight SDK calls on those Node versions, or use raw fetchtypesafe-sdk-js #2; jkudish README L757 Reported
TS11experimental_evaluate "may change in patch releases"Pin the exact ai versionai-sdk.dev Documented
TS12Probabilities are rounded to 2 decimals and may not sum to 1Do not assert exact sums in testsai-sdk.dev provider page Documented

Open issues in the official JS SDK

Open on GitHub on 29 Sep 2026. Each is the reporter's description, not rerun by the pool. Reported

  • #2: a handled cancellation can crash Node 20 and 22.
  • #6: noul() with no arguments, and state: null, build requests the API rejects.
  • #8: timeouts above Node's timer maximum fire after about 1 ms.
  • #9: a blank Retry-After header bypasses backoff.
  • #12: score() accepts a null level, which returns 422.
  • #13: 402 and 413 fall through to a generic APIError.
  • #14: the API key can be echoed into APIConnectionError, and an empty key is sent.
  • #15: Cloudflare HTML 403 when the state contains curl text.
  • #17: non-finite numbers serialise as null.

Issue list: typesafe-ai/typesafe-sdk-js issues.

Projects that use these paths

ProjectWhat its code doesLabel and date
Jevmail classify.tsCalls experimental_evaluate({ model: 'typesafe-ai/jev', …, maxRetries: 0 }), then runs its own 429 wait-and-retry-once and 5xx retry-once. This avoids stacking the AI SDK's 2 retries under its own (TS6).Documented 28 Sep audit, not re-audited
Vercel template jev-ai-sdk-form-routerCONFIDENCE_THRESHOLD = 0.95. The Jev call uses AbortSignal.timeout(12_000) and maxRetries: 1. If confidence is below 0.95 or missing, or Jev fails, it calls a fallback LLM (25 s, 1 retry). A missing confidence counts as a fallback, not a pass. package.json pins ai 7.0.107. The fallback model is openai/gpt-6-luna-fast in source but openai/gpt-5.6-luna-fast on the template page.Documented at 27ea469 (22 Sep), checked 29 Sep
MSFT-TKENDRICK/JEV-examplesThe author states the examples were never executed live. Not opened by the pool.Reported

Seen only: TanStack AI decide() with vercelGatewayDecider('typesafe-ai/jev') or cloudflareDecider('typesafe/jev'), described by Vercel. LangChain JS (@langchain/typesafe 0.0.2) and @effect/ai-typesafe 4.0.1 were read on 5 Oct 2026: see framework integrations. For TypeScript demo videos, see the TypeScript group on the videos page.

What was not verified

  • No example on this page was run. The official JS SDK was not installed, and the AI SDK was read from its published type declarations and docs.
  • The AI SDK's default timeout is undocumented. The changes in @ai-sdk/typesafe-ai 3.0.9 and 3.0.10 (both 28 Sep) were not compared.
  • The Cloudflare binding's response shape, retries and timeout were not re-read on 29 Sep.
  • The 31.5 s worst case is arithmetic from SDK defaults, not a measurement.

Search published pools, pages, reports, and evidence.