AI

Litestore's AI features call one of three providers through the Vercel AI SDK. Read this page when you configure a provider key or build a new AI feature.

Litestore doesn't bundle any AI. When a store has no provider key, it hides the AI features instead of failing.

That idea runs through this whole area. AI is optional in a self-hosted store, so every feature that uses it must be able to disappear cleanly.

The three providers

export type Provider = "google" | "anthropic" | "openai"

const DEFAULT_PROVIDER: Provider = "google"

// Use GA / stable, low-latency models — NOT `-exp` previews, which Google shuts down
// on a rolling basis (that's how "AI Connected" ended up pointing at a dead endpoint).
export const DEFAULT_MODELS: Record<Provider, string> = {
  google: "gemini-2.0-flash",
  anthropic: "claude-sonnet-4-20250514",
  openai: "gpt-4o",
}

The comment about preview models records a real incident. The provider retired an -exp model, and the store was left pointing at a dead endpoint. That is why the defaults are GA models.

Litestore treats a provider as configured when its key is present and at least 20 characters long. It treats a shorter value as a placeholder or a cut-off paste. AIConfigError then tells you so, instead of letting the provider fail the request with an unhelpful message.

Configuration

AI_PROVIDER chooses the provider.

AI_MODEL overrides the model, but only when its provider is the active one. Model names belong to one provider, so applying it to a fallback provider would send a Gemini model id to Anthropic.

AI_FALLBACK_ENABLED is on unless you set it to exactly "false". Litestore reads it from process.env, not from the validated env object.

An unknown AI_PROVIDER value isn't an error. Litestore falls back to the default provider, so a typo switches you to Google instead of taking the store down.

The failover chain

Each provider has exactly one successor:

google → anthropic → openai → google

Fallback goes one step, not around every provider. If the successor has no key, or its key is shorter than 20 characters, there is no fallback model, and Litestore rethrows the original error.

Litestore only checks the primary provider's key up front. A missing successor key just means there is no fallback, not a startup failure.

shouldFallback decides whether an error moves on to the successor. It looks at the typed signal first:

1. If the error is an SDK APICallError, it uses the status code.
2. On 401 or 403, it doesn't fall back.
3. On 429 or any 5xx, it falls back.
4. Otherwise, it trusts the SDK's own isRetryable flag.
5. For errors without a status, such as network failures, it matches the message and error name against a list of patterns.

A 401 or 403 deliberately does not fall back. A rejected credential is a configuration mistake, and quietly spending money at the next provider would hide it.

Checking the status first also keeps working after SDK upgrades. If Litestore matched message text, a provider rewording "rate limit" would silently break fallback, and nobody would notice until the bill arrived.

Litestore only uses the pattern list for errors that have no status at all:

rate limit · rate_limit · too many requests · 429 · 503 · 502 · 500
timeout · timed out · ECONNRESET · ENOTFOUND · socket hang up
network error · service unavailable · overloaded · capacity

Streaming cannot fall back

Your code awaits generateObject and generateText, so it can catch their errors where it calls them and retry with the fallback provider.

streamText works differently. It returns right away, and its API errors only appear while something downstream reads the stream, long after the wrapper has returned.

A try/catch around streamText therefore never caught the errors it was written for. Those errors slipped past it, and the SDK reported a generic "An error occurred" in the stream.

The streaming path now does what the SDK can actually do:

• maxRetries: 3 retries the same provider with exponential backoff before the stream starts. That genuinely recovers from a brief outage.
• onError logs streaming errors instead of swallowing them, and notes whether the error would have justified a fallback.

When a call finishes, every path records usage in PostHog. It logs the provider, the resolved model, the token counts and whether the call fell back. If telemetry fails, Litestore logs the failure and moves on, so the caller still gets the generated result.

When nothing is configured

withAdminAI returns 501 with "AI features are not configured".

The dashboard's Ask AI action, the URL import option, the inbox draft and assistant, and the panels under Settings → AI all check aiEnabled/isConfigured and stay hidden.

Every route under app/ai/v1/ uses withAdminAI as its shared wrapper, and the 501 is only its first check. A request must pass these checks in order:

1. Admin authentication and the ai:view permission.
2. A configured provider, or the request gets a 501.
3. Zod validation of the request body, or the request gets a 400 with the list of issues.
4. A safety check on the last user message, or the request gets a 400 with blocked.

Litestore translates SDK errors instead of passing them through. A network failure becomes 503 and an unknown model becomes 404. A content-filter block becomes 400, and anything else becomes 500 with the provider's message cut to 200 characters.

Adding an AI feature

If your feature is a route, put it behind withAdminAI. That wrapper already handles the configured check, the permission gate, input validation, the safety check and error translation. If you rewrite any of them yourself, your feature will behave differently from the rest.

Call generateObjectWithFallback or generateTextWithFallback instead of calling generateObject or generateText directly. The wrappers pick the model, handle the failover step and record usage.

Use streamTextWithFallback for anything that streams, and expect it to retry the same provider instead of switching.

Check isAIEnabled() in the UI as well as in the route. A 501 is the right answer to a request, but an operator with no key shouldn't see a button that can only fail.

When you wire up a new AI feature, don't ask whether it works with your key. Ask this instead:

What does this feature look like on a store that has no AI configured at all?

Related