AI

AI features run through the Vercel AI SDK against one of three providers.

Nothing is bundled — a store with no provider key configured hides the AI surfaces rather than failing.

That is the whole posture of this area. AI is an optional capability of a self-hosted store, so every surface that uses it has to be able to not exist.

The three providers

export type Provider = "google" | "anthropic" | "openai"

const DEFAULT_PROVIDER: Provider = "google"

// Use GA / stable, low-latency models — NOT `-exp` previews, which Google shuts down
// on a rolling basis (that's how "AI Connected" ended up pointing at a dead endpoint).
export const DEFAULT_MODELS: Record<Provider, string> = {
  google: "gemini-2.0-flash",
  anthropic: "claude-sonnet-4-20250514",
  openai: "gpt-4o",
}

The comment about preview models is a recorded incident, not a style note: an -exp model was retired by the provider and left the store pointing at a dead endpoint. The defaults are GA models for that reason.

A provider counts as configured when its key is present and at least 20 characters long. A shorter value is treated as a placeholder or a truncated paste, and AIConfigError says so rather than letting the request fail at the provider with something unhelpful.

Configuration

AI_PROVIDER selects the provider.

AI_MODEL overrides the model, and applies only when the provider it belongs to is the active one. A model name is provider-specific, so applying it to a fallback provider would send a Gemini model id to Anthropic.

AI_FALLBACK_ENABLED defaults to on unless set to exactly "false", and is read from process.env rather than the validated env object.

An unrecognised AI_PROVIDER value is not an error. It falls back to the default provider, so a typo degrades to Google rather than taking the store down.

The failover chain

Each provider has exactly one successor:

google → anthropic → openai → google

Fallback is a single hop, not a walk through every provider. If the successor has no key — or a key under 20 characters — there is no fallback model at all, and the original error is rethrown.

Only the primary provider is validated up front. A missing key on the successor is a missing fallback, not a startup failure.

Whether an error falls through at all is decided by shouldFallback, which prefers the typed signal:

1. If the error is an SDK APICallError, use its status code.
2. 401 or 403 — do not fall back.
3. 429 or any 5xx — fall back.
4. Otherwise, trust the SDK's own isRetryable classification.
5. For statusless errors, such as network-level throws, match the message and error name against a pattern list.

A 401 or 403 deliberately does not fall through — a rejected credential is a configuration error, and quietly spending money at the next provider would hide it.

Reading the status first also survives SDK upgrades. Substring-matching a message means a provider rewording "rate limit" silently breaks the fallback path, which is exactly the kind of failure nobody notices until the bill arrives.

The pattern list is the fallback for errors that carry no status at all:

rate limit · rate_limit · too many requests · 429 · 503 · 502 · 500
timeout · timed out · ECONNRESET · ENOTFOUND · socket hang up
network error · service unavailable · overloaded · capacity

Streaming cannot fall back

generateObject and generateText are awaited, so their errors are catchable at the call site and can be retried against the fallback provider.

streamText is not. It returns synchronously, and its API errors surface while the stream is being consumed downstream — long after the wrapper has returned.

A try/catch around streamText therefore never fired for the very conditions it named. Those errors sailed past it and the SDK reported a generic "An error occurred" in the stream.

The streaming path does what the SDK can actually do:

• maxRetries: 3 retries the same provider with exponential backoff before the stream produces, which is real recovery for a blip.
• onError logs streaming errors, including whether the error was one that would have justified a fallback, instead of swallowing them.

Every path records usage to PostHog on completion — provider, resolved model, token counts, and whether it fell back. Telemetry failures are logged and swallowed, so generation stays the source of truth for the caller.

When nothing is configured

withAdminAI returns 501 with "AI features are not configured".

The dashboard's Ask AI action, the URL import option, the inbox draft and assistant, and the Settings → AI panels all check aiEnabled/isConfigured and stay hidden.

withAdminAI is the shared wrapper for every route under app/ai/v1/, and the 501 is only its first gate. In order, a request must clear:

1. Admin authentication and the ai:view permission.
2. The configured-provider check, or 501.
3. Zod validation of the request body, or 400 with the issue list.
4. An input safety check on the last user message, or 400 with blocked.

Errors from the SDK are translated rather than leaked. A network failure becomes 503, an unknown model becomes 404, a content-filter block becomes 400, and anything else becomes 500 with the provider message truncated to 200 characters.

Adding an AI feature

Put it behind withAdminAI if it is a route. That is where the configured check, the permission gate, input validation, the safety check and error translation already live, and re-implementing any of them is how a surface ends up with different behaviour from the rest.

Call generateObjectWithFallback or generateTextWithFallback rather than generateObject or generateText directly. The wrappers own model selection, the failover hop and usage telemetry.

Use streamTextWithFallback for anything streaming, and expect same-provider retries rather than a provider switch.

Check isAIEnabled() in the UI as well as the route. A 501 is the correct answer to a request, but an operator with no key should not be looking at a button that can only fail.

The question to ask when wiring a new AI surface is not "does it work with my key?" It is:

What does this surface look like on a store that has no AI configured at all?

Related