Ensure Manifest Docs Consistency

Audit all Manifest documentation sources for consistency against the live codebase. Produces a concise dissonance report with tables.

Sources to Check

Codebase (source of truth)

Fetch these via the GitHub API (https://api.github.com/repos/mnfst/manifest/contents/<path>, decode base64):

Documentation materials (check against codebase)

Workflow

1. Fetch codebase files (source of truth)

Fetch all codebase files listed above via the GitHub API in parallel. Use this pattern:

curl -sL "https://api.github.com/repos/mnfst/manifest/contents/<PATH>" | python3 -c "import sys,json,base64; print(base64.b64decode(json.load(sys.stdin)['content']).decode())"

Extract and note these key facts:

• Versions: manifest and manifest-model-router package versions
• Descriptions: package descriptions, plugin descriptions
• Install commands: exact commands for local and cloud modes
• Config schemas: exact config options per plugin (from openclaw.plugin.json)
• Provider list: all supported LLM providers
• Savings claims: percentage claims (e.g., "up to 70%")
• Scoring details: number of dimensions, latency, tier names
• Agent tools: tool names, descriptions
• Data paths: database location, config files
• Links/URLs: docs URL, homepage, dashboard URL
• Tech stack: frameworks, databases, versions
• Two deployment modes: local (manifest plugin, SQLite, embedded server) vs cloud (manifest-model-router plugin, PostgreSQL, app.manifest.build)

2. Fetch documentation materials

Fetch all documentation URLs via WebFetch in parallel. If WebFetch fails (403, timeout), try alternatives:

• For npm: npm view <package> --json via Bash
• For GitHub: use the API pattern above

3. Verify ClawhHub listing legitimacy

For each ClawhHub resource, assess whether it is Benign or Suspicious by checking these signals against the codebase source of truth:

Classify each listing:

• Benign (high confidence): Author, repo, version, and description all match the codebase
• Benign (low confidence): Most fields match but page has minimal content or minor discrepancies
• Suspicious (medium confidence): One or more signals don't match (wrong version, missing repo link, empty page)
• Suspicious (high confidence): Author mismatch, links to wrong repo, or description is unrelated

Add a ClawhHub Legitimacy section to the report:

### ClawhHub Listing Verification

| Resource | URL | Verdict | Version | Notes |
|----------|-----|---------|---------|-------|
| Plugin (manifest-model-router) | clawhub.ai/plugins/manifest-model-router | Benign (high confidence) | 5.33.x | Matches codebase |
| Skill (manifest-build) | clawhub.ai/brunobuddy/manifest-build | Suspicious (medium confidence) | 5.33.x | Empty page, no description |

If any listing is Suspicious, add a WARNING row to the Dissonance Table with severity HIGH and category "ClawhHub legitimacy". Include what specifically is wrong and recommend investigating whether the listing is legitimate or has been hijacked/impersonated.

4. Check recent PRs for undocumented features

This step is critical. Fetch the last 30 merged PRs from the main repo:

gh pr list --repo mnfst/manifest --state merged --limit 30 --json number,title,mergedAt,body,headRefName

For each PR that is NOT a version-packages/changeset PR, extract:

• New features: any new user-facing capability (e.g., real-time updates, unified wizard, new env vars)
• Breaking changes: renamed env vars, removed config options, changed defaults, deleted components
• New install flows or workarounds: npm pack fallback, changed onboarding steps
• Renamed or removed concepts: plugin renames, deleted auto-setup behavior

Then check each extracted feature/change against every documentation material from step 2. For each feature that is missing or contradicted in any material, add a row to the Dissonance Table with:

• Severity HIGH if the material actively contradicts the new behavior (e.g., still documents a removed feature)
• Severity MEDIUM if the material simply omits a new feature that users should know about

Add a dedicated section to the report:

### Recent PR Coverage Gap

| PR | Feature/Change | Documented in | Missing from |
|----|---------------|---------------|-------------|
| #1358 | Real-time SSE dashboard updates | README | Website, npm |
| #1349 | Env var renamed: TELEMETRY_OPTOUT → UPDATE_CHECK_OPTOUT | CLAUDE.md | Website, npm, SKILL.md |

This table should list every significant change from recent PRs and whether each documentation source covers it. The goal is to catch features that shipped in code but never made it into the marketing or user-facing materials.

5. Compare and identify dissonance

For each documentation source, compare against the codebase for these categories:

6. Generate report

Output a structured report with these sections:

Header

# Manifest Docs Consistency Report
Date: YYYY-MM-DD
Codebase: github.com/mnfst/manifest (commit: <latest>)
manifest v<version> | manifest-model-router v<version>

Summary

One paragraph: how many sources checked, how many dissonances found, severity breakdown.

Dissonance Table

Main table with all findings:

| # | Severity | Category | Claim | Source | Correct Value (Codebase) | Affected Sources |
|---|----------|----------|-------|--------|--------------------------|-----------------|

Severity levels:

• CRITICAL -- Wrong install commands, wrong config options, broken links (users will fail)
• HIGH -- Contradictory claims between sources, outdated versions
• MEDIUM -- Inconsistent descriptions, minor factual errors
• LOW -- Cosmetic differences, stale metrics, wording variations

Cross-Source Contradiction Table

When two or more docs disagree with each other (regardless of which is correct):

| Topic | Source A | Says | Source B | Says |
|-------|---------|------|---------|------|

Recommendations

Bulleted list of specific fixes, ordered by severity. Each item should name the exact file/URL and what to change.

Rules

• The codebase (package.json, openclaw.plugin.json, source files) is always the source of truth
• Never fabricate findings -- only report verifiable dissonances
• If a source is unreachable, note it in the report but don't count it as a dissonance
• Focus on actionable issues that could confuse users or cause setup failures
• Keep the report concise -- one row per dissonance, not one row per source
• When versions are close (e.g., 5.33.1 vs 5.33.2), note it but mark as LOW since npm pages may cache
• Flag any documentation that mentions config options not present in the actual plugin schema
• Flag any install commands that differ from the official README