Adding AI to Commerce Modules: Confidence Ranking, Not Black Boxes

Every commerce platform is adding AI. Most are doing it wrong.

They're building black boxes. "AI-powered recommendations" that operators can't explain, debug, or override. Magic ranking that works until it doesn't, and then nobody knows why.

We took a different approach: AI as a bounded multiplier, not a decision maker.

The Core Principle

AI should influence, not control. Here's the formula:

finalScore = baseScore × aiMultiplier

// Where:
// - baseScore comes from deterministic rules (engagement, recency, inventory)
// - aiMultiplier is bounded: 0.5 to 2.0 (can halve or double, but not more)

If AI breaks completely, the worst case is that products rank at half or double their expected position. The store still works. Operators can still explain why products appear where they do.

External Intelligence Architecture

We don't embed AI into core commerce logic. We treat it as an external signal source.

// lib/external-intelligence/types.ts
export type ExternalIntelligence = {
  source: "openai" | "anthropic" | "custom" | "manual";
  confidence: number; // 0.0 to 1.0
  signal: number; // -1.0 to 1.0 (negative = suppress, positive = boost)
  reasoning: string; // Human-readable explanation
  generatedAt: Date;
  expiresAt: Date; // Intelligence goes stale
};

Key design decisions:

Confidence is explicit — AI must say how sure it is
Signal is bounded — -1.0 to 1.0, no extreme values
Reasoning is required — No opaque recommendations
Expiration is mandatory — AI judgments have shelf life

Integrating with Product Ranking

The ranking engine treats AI as just another multiplier:

// lib/ranking.ts
export function computeProductScore(product: Product): ScoredProduct {
  const engagement = computeEngagementScore(product);
  const recency = computeRecencyMultiplier(product);
  const inventory = computeInventoryMultiplier(product);
  const boost = product.manualBoost ?? 1.0;

  // AI is optional — if not present, multiplier is 1.0 (neutral)
  const aiMultiplier = computeAIMultiplier(product.externalIntelligence);

  const finalScore =
    (1 + engagement) * recency * inventory * boost * aiMultiplier;

  return {
    ...product,
    score: finalScore,
    breakdown: {
      engagement,
      recency,
      inventory,
      boost,
      aiMultiplier,
    },
  };
}

The AI Multiplier Function

function computeAIMultiplier(
  intelligence: ExternalIntelligence | null,
): number {
  // No intelligence = neutral
  if (!intelligence) return 1.0;

  // Expired intelligence = neutral
  if (new Date() > intelligence.expiresAt) return 1.0;

  // Low confidence = dampened effect
  const effectiveSignal = intelligence.signal * intelligence.confidence;

  // Convert signal (-1 to 1) to multiplier (0.5 to 2.0)
  // signal = -1.0 → multiplier = 0.5 (halve the score)
  // signal = 0.0  → multiplier = 1.0 (no change)
  // signal = 1.0  → multiplier = 2.0 (double the score)
  const multiplier = 1.0 + effectiveSignal * 0.5;

  // Clamp to bounds (defense in depth)
  return Math.max(0.5, Math.min(2.0, multiplier));
}

Why This Bounds Matter

| Scenario | AI Signal | Confidence | Effective Multiplier | | --------------------- | --------- | ---------- | --------------------------- | | AI loves this product | +1.0 | 0.9 | 1.45 (45% boost) | | AI loves this product | +1.0 | 0.5 | 1.25 (25% boost) | | AI hates this product | -1.0 | 0.9 | 0.55 (45% suppression) | | AI is uncertain | +0.2 | 0.3 | 1.03 (3% boost, negligible) | | AI is missing | - | - | 1.0 (neutral) |

Low confidence AI has minimal effect. High confidence AI has bounded effect. The system degrades gracefully.

Adding AI to Any Module

The pattern is consistent across modules. Here's how to add AI to a new domain:

Step 1: Define the Intelligence Schema

// server/admin/curations/schema.ts
export const curationIntelligenceSchema = z.object({
  source: z.enum(["openai", "anthropic", "custom", "manual"]),
  confidence: z.number().min(0).max(1),
  signal: z.number().min(-1).max(1),
  reasoning: z.string().max(500),
  generatedAt: z.date(),
  expiresAt: z.date(),
  // Domain-specific fields
  suggestedProducts: z.array(z.string()).optional(),
  seasonalRelevance: z.number().min(0).max(1).optional(),
});

Step 2: Add the Field to the Model

model Collection {
  // ... existing fields
  externalIntelligence Json? @db.JsonB
}

Step 3: Create the Intelligence Source

// lib/external-intelligence/curation-analyzer.ts
export async function analyzeCollection(
  curation: Collection,
): Promise<ExternalIntelligence> {
  const prompt = `
    Analyze this e-commerce curation:
    Name: ${curation.name}
    Description: ${curation.description}
    Product count: ${curation.productCount}
    Categories: ${curation.categories.join(", ")}

    Rate its current commercial potential from -1 (suppress) to +1 (boost).
    Consider: seasonality, trend alignment, inventory health, price competitiveness.
    Explain your reasoning in one sentence.
  `;

  const response = await anthropic.messages.create({
    model: "claude-3-5-sonnet-20241022",
    max_tokens: 200,
    messages: [{ role: "user", content: prompt }],
  });

  const parsed = parseAIResponse(response);

  return {
    source: "anthropic",
    confidence: parsed.confidence,
    signal: parsed.signal,
    reasoning: parsed.reasoning,
    generatedAt: new Date(),
    expiresAt: addHours(new Date(), 24), // Valid for 24 hours
  };
}

Step 4: Hook into the Refresh Cycle

// lib/cron/intelligence-refresh.ts
export async function refreshCollectionIntelligence() {
  const curations = await getCollectionsNeedingAnalysis();

  for (const curation of curations) {
    try {
      const intelligence = await analyzeCollection(curation);
      await updateCollectionIntelligence(curation.id, intelligence);
    } catch (error) {
      // AI failure doesn't break the system
      logger.warn("Collection analysis failed", {
        curationId: curation.id,
        error: error.message,
      });
      // Collection continues with existing/no intelligence
    }
  }
}

Step 5: Use in Queries (Optional)

// server/web/curations/queries.ts
export async function getFeaturedCollections() {
  const curations = await db.curation.findMany({
    where: { isFeatured: true },
    orderBy: { position: "asc" },
  });

  // Apply AI multiplier to curation ordering
  return curations
    .map((c) => ({
      ...c,
      effectiveScore: computeCollectionScore(c),
    }))
    .sort((a, b) => b.effectiveScore - a.effectiveScore);
}

Real-World Integration: Product Descriptions

Here's a complete example: AI-assisted product descriptions.

The Problem

Operators write product descriptions. Some are great. Some are SEO disasters. AI can help, but it shouldn't override human decisions.

The Solution

// lib/external-intelligence/description-analyzer.ts
export async function analyzeProductDescription(
  product: Product,
): Promise<DescriptionIntelligence> {
  const response = await anthropic.messages.create({
    model: "claude-3-5-sonnet-20241022",
    max_tokens: 500,
    messages: [
      {
        role: "user",
        content: `
        Analyze this product description for e-commerce effectiveness:

        Title: ${product.name}
        Description: ${product.description}
        Category: ${product.category}

        Evaluate:
        1. SEO quality (keywords, length, structure)
        2. Conversion potential (benefits, urgency, clarity)
        3. Sourcing voice consistency

        Return JSON:
        {
          "seoScore": 0-100,
          "conversionScore": 0-100,
          "suggestions": ["suggestion 1", "suggestion 2"],
          "rewriteSuggestion": "optional improved description"
        }
      `,
      },
    ],
  });

  const analysis = JSON.parse(response.content[0].text);

  return {
    source: "anthropic",
    confidence: 0.8, // Fixed confidence for this use case
    seoScore: analysis.seoScore,
    conversionScore: analysis.conversionScore,
    suggestions: analysis.suggestions,
    rewriteSuggestion: analysis.rewriteSuggestion,
    generatedAt: new Date(),
    expiresAt: addDays(new Date(), 7), // Valid until description changes
  };
}

Admin UI Integration

The admin shows AI analysis alongside the product form:

// app/admin/products/[id]/_components/description-intelligence.tsx
export function DescriptionIntelligence({ product }: Props) {
  const intelligence = product.descriptionIntelligence;

  if (!intelligence) {
    return <Button onClick={analyzeDescription}>Analyze with AI</Button>;
  }

  return (
    <Card>
      <CardHeader>
        <CardTitle>AI Analysis</CardTitle>
        <Badge variant={intelligence.confidence > 0.7 ? "success" : "warning"}>
          {Math.round(intelligence.confidence * 100)}% confident
        </Badge>
      </CardHeader>
      <CardContent>
        <div className="space-y-4">
          <ScoreBar label="SEO" value={intelligence.seoScore} />
          <ScoreBar label="Conversion" value={intelligence.conversionScore} />

          {intelligence.suggestions.length > 0 && (
            <div>
              <h4>Suggestions</h4>
              <ul>
                {intelligence.suggestions.map((s, i) => (
                  <li key={i}>{s}</li>
                ))}
              </ul>
            </div>
          )}

          {intelligence.rewriteSuggestion && (
            <div>
              <h4>Suggested Rewrite</h4>
              <p className="text-muted">{intelligence.rewriteSuggestion}</p>
              <Button
                variant="outline"
                onClick={() => applyRewrite(intelligence.rewriteSuggestion)}
              >
                Apply Suggestion
              </Button>
            </div>
          )}
        </div>
      </CardContent>
    </Card>
  );
}

The operator sees:

AI's confidence level
Specific scores for SEO and conversion
Actionable suggestions
An optional rewrite they can apply with one click

But the operator decides. AI doesn't auto-update descriptions. It provides analysis. Humans approve changes.

AI for Spot Copy

The same pattern works for promotional content:

// lib/external-intelligence/banner-copy-generator.ts
export async function generateSpotCopy(
  banner: Spot,
  curation: Collection,
): Promise<SpotCopyIntelligence> {
  const response = await anthropic.messages.create({
    model: "claude-3-5-sonnet-20241022",
    max_tokens: 300,
    messages: [
      {
        role: "user",
        content: `
        Generate promotional copy for this e-commerce banner:

        Spot type: ${banner.spotType}
        Placement: ${banner.placement}
        Collection: ${curation.name}
        Products: ${curation.productCount} items
        Price range: $${curation.minPrice} - $${curation.maxPrice}

        Generate 3 variants of:
        - Headline (max 60 chars)
        - Subheadline (max 120 chars)
        - CTA text (max 20 chars)

        Match the tone: ${banner.spotType === "promotional" ? "urgent, deal-focused" : "aspirational, lifestyle"}
      `,
      },
    ],
  });

  return {
    source: "anthropic",
    confidence: 0.75,
    variants: parseVariants(response),
    generatedAt: new Date(),
    expiresAt: addHours(new Date(), 48),
  };
}

Operators see three copy variants. They pick one, edit it, or ignore all of them. AI suggests. Humans decide.

Guardrails and Limits

1. Bounded Values

Every AI output is clamped:

const signal = Math.max(-1, Math.min(1, rawSignal));
const confidence = Math.max(0, Math.min(1, rawConfidence));

2. Expiration Enforcement

Stale intelligence is ignored:

if (new Date() > intelligence.expiresAt) {
  return 1.0; // Neutral multiplier
}

3. Graceful Degradation

AI failures don't break commerce:

try {
  const intelligence = await analyzeProduct(product);
  await updateIntelligence(product.id, intelligence);
} catch (error) {
  logger.error("AI analysis failed", { productId: product.id, error });
  // Product continues with neutral AI multiplier
  // No user-facing error
}

4. Audit Trail

Every AI decision is logged:

await db.intelligenceLog.create({
  data: {
    entityType: "product",
    entityId: product.id,
    source: intelligence.source,
    signal: intelligence.signal,
    confidence: intelligence.confidence,
    reasoning: intelligence.reasoning,
    appliedAt: new Date(),
  },
});

Operators can see: "On Jan 15, Claude recommended boosting this product by 40% because 'seasonal trend alignment with Valentine's Day sample searches'."

5. Kill Switch

Global AI influence can be disabled:

// config/features.ts
export const FEATURES = {
  aiRankingEnabled: process.env.AI_SCORING_ENABLED !== "false",
  aiCopyGenerationEnabled: process.env.AI_COPY_ENABLED !== "false",
};

// In ranking.ts
const aiMultiplier = FEATURES.aiRankingEnabled
  ? computeAIMultiplier(product.externalIntelligence)
  : 1.0;

If AI goes haywire, one environment variable disables it system-wide.

What AI Should NOT Do

Make final decisions — AI suggests, humans approve
Modify data without audit — Every change is logged
Have unbounded influence — Multipliers are clamped
Be required — System works without AI
Be opaque — Reasoning is always required

What's Next

The external intelligence system is live. What we're building:

Competitive pricing intelligence — AI monitors competitor prices, suggests adjustments
Trend detection — Identify rising search terms, suggest curation updates
Review sentiment analysis — Surface negative reviews early
Inventory forecasting — Predict stockouts before they happen

All following the same pattern: bounded influence, required reasoning, human approval, graceful degradation.

AI makes commerce smarter. It doesn't make commerce automatic.

Command Menu