AI

Litestore's AI features call Google Gemini, Anthropic Claude or OpenAI through the Vercel AI SDK, depending on which provider you set in AI_PROVIDER. Read this page when you configure a provider key or build a new AI feature.

Litestore doesn't bundle any AI. When a store has no provider key, it hides the AI features instead of failing. AI is optional in a self-hosted store, so every feature that uses it must be able to disappear cleanly.

Providers

export type Provider = "google" | "anthropic" | "openai"

const DEFAULT_PROVIDER: Provider = "google"

// Use GA / stable, low-latency models — NOT `-exp` previews, which Google shuts down
// on a rolling basis (that's how "AI Connected" ended up pointing at a dead endpoint).
const DEFAULT_MODELS: Record<Provider, string> = {
  google: "gemini-2.0-flash",
  anthropic: "claude-sonnet-4-20250514",
  openai: "gpt-4o",
}

The comment about preview models records a real incident. The provider retired an -exp model, and the store was left pointing at a dead endpoint. That is why the defaults are GA models.

Litestore treats a provider as configured when its key is present and at least 20 characters long. It treats a shorter value as a placeholder or a cut-off paste, and AIConfigError reports that instead of letting the provider fail the request with an unhelpful message.

Configuration

AI_MODEL only applies to the active provider because model names belong to one provider. Applying it to a fallback provider would send a Gemini model id to Anthropic.

Litestore reads AI_FALLBACK_ENABLED from process.env, not from the validated env object.

Failover

Each provider has exactly one successor:

google → anthropic → openai → google

Fallback goes one step, not around every provider. If the successor has no key, or its key is shorter than 20 characters, Litestore has no fallback model and rethrows the original error. Litestore only checks the primary provider's key up front, so a missing successor key means there is no fallback, not a startup failure.

shouldFallback determines whether an error moves on to the successor. It looks at the typed signal first.

A 401 or 403 deliberately does not fall back. A rejected credential is a configuration mistake, and quietly spending money at the next provider would hide it.

Checking the status first also keeps working after SDK upgrades. If Litestore matched message text, a provider rewording "rate limit" would silently break fallback, and nobody would spot the problem until the bill arrived. Litestore only uses the pattern list for errors that have no status at all:

rate limit · rate_limit · too many requests · 429 · 503 · 502 · 500
timeout · timed out · ECONNRESET · ENOTFOUND · socket hang up
network error · service unavailable · overloaded · capacity

Streaming

Your code awaits generateObject and generateText, so it can catch their errors where it calls them and retry with the fallback provider.

streamText works differently. It returns right away, and its API errors only appear while something downstream reads the stream, long after the wrapper has returned. A try/catch around streamText therefore never caught the errors it was written for. Those errors slipped past it, and the SDK reported a generic "An error occurred" in the stream.

The streaming path now does what the SDK can actually do. maxRetries: 3 retries the same provider with exponential backoff before the stream starts, which genuinely recovers from a brief outage. onError logs streaming errors instead of swallowing them, and notes whether the error would have justified a fallback.

When a call finishes, every path records its cost against the monthly budget (recordAiSpend in modules/assistant/spend.ts) and sends a usage event to PostHog with the provider, the resolved model, the token counts and whether the call fell back. If telemetry fails, Litestore logs the failure and moves on, and the caller still gets the generated result.

Limits and budget

Two ceilings from platform/config/ai.ts apply to every call:

Spend is computed from token counts and AI_MODEL_PRICES. A model missing from the price table is priced as the most expensive listed one, so the budget can only trip early.

Grounding

verifyGroundedOutput in modules/assistant/grounding.ts checks AI text that a person will act on or send. A model asked to write from known facts can still invent the specifics a reader trusts most, such as an amount, an order number or a date, and every one of those carries a digit. The check pulls each digit-bearing token out of the output and requires it to appear in the facts the caller fetched.

Fill with AI on the product page holds back a specification whose number isn't in the operator's inputs, and lists copy that carries one for review. The support-reply draft in Messages re-checks its draft the same way. The function is pure, so each caller decides what to do with a failure.

No provider configured

withAdminAI in modules/assistant/admin-ai-route.ts returns 501 with "AI features are not configured". In the UI, every AI affordance (the assistant on the admin home, Fill with AI on the product page, the URL import option and the reply draft in Messages) is wrapped in <AiOnly> or checks useAiEnabled() from modules/assistant/ai-status-provider.tsx, so it stays hidden. Without a key, the admin home shows the To do counts instead of the assistant.

Both routes under app/ai/v1/ (admin-chat and fill-product) use withAdminAI as their shared wrapper, and the 501 is only one of its checks. A request must pass these checks in order:

Litestore translates SDK errors instead of passing them through. A network failure becomes 503 and an unknown model becomes 404. A content-filter block becomes 400, the monthly budget and provider rate limits become 429, a provider quota error becomes 402, and anything else becomes 500 with the provider's message cut to 200 characters.

Adding AI features

When you wire up a new AI feature, don't ask whether it works with your key. Ask this instead: what does this feature look like on a store that has no AI configured at all?

Related