AI Health Check

Before you ship an AI feature, it needs to pass 6 checks.

Most AI products fail because PMs skip the basics: no cost model, broken failure UX, terrible data quality. This skill stops you from launching garbage.

Entry Point

When this skill is invoked, start with:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 AI HEALTH CHECK
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Before shipping an AI feature, it needs to pass 6 checks.

What AI feature are you preparing to launch?

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Usage

/ai-health-check [feature-name]

Examples:

• /ai-health-check "AI product recommendations" - Audit specific feature
• /ai-health-check "email composer AI" - Manual description
• /ai-health-check --pre-launch - Full checklist for current sprint

What Happens

1. Invokes the ai-implementation-auditor agent
2. Asks hard questions about your AI feature
3. Grades each of 6 dimensions: Ready / Risk / Blocker
4. Tells you if you can ship

The 6 Dimensions

Verdict Logic

Sample Output

AI Health Check: Email Composer

Overall Readiness: NEEDS WORK (4/6 dimensions ready)

---

Ready: Model Selection, Production Monitoring, System Optimization
Risk: Data Quality, Failure UX
Blocker: Cost Modeling

VERDICT: DON'T SHIP YET

You have 1 blocker:
- No cost model -> Run /ai-cost-check RIGHT NOW

You have 2 risks:
- Data quality strategy undefined
- Failure UX is broken ("Something went wrong" isn't helpful)

---

What To Do Now:

Option A: Fix everything (RECOMMENDED)
1. Run /ai-cost-check (10 min)
2. Define data quality strategy (2 hours)
3. Build better failure UX (3 hours)
4. Rerun /ai-health-check

Option B: Ship with known risks
1. Fix the blocker only
2. Ship knowing data quality and failure UX are weak
3. Plan to fix in week 1

Common Blockers

Cost Modeling missing:

"You're about to launch with zero idea if this bankrupts you at scale." Run /ai-cost-check first.

Failure UX broken:

"Something went wrong" tells users nothing. No confidence indicators = users don't know when to trust the AI.

No monitoring plan:

"Launching without monitoring = flying blind."

Philosophy (Chip Huyen)

• "Most AI failures are UX problems, not technical ones."
• "Data quality beats tool selection."
• "Fine-tuning should be your last resort."
• "The gap between a demo and a product is production engineering."

Related Commands

• /ai-cost-check - Detailed cost modeling (run if cost dimension is blocked)
• /start-evals - Set up quality testing
• /four-risks - Overall feature risk assessment

Best for: Pre-launch validation of AI features Key insight: "Fine-tuning is the last resort. Data quality beats tool selection. Most AI failures are UX problems."