Agent skills
test-quality-analysis

Agent skill

test-quality-analysis

Detect test smells, overmocking, flaky tests, and coverage issues. Analyze test effectiveness, maintainability, and reliability. Use when reviewing tests or improving test quality.

View SKILL.md on GitHub Repository

Stars 109

Forks 14

Install this agent skill to your Project

npx add-skill https://github.com/secondsky/claude-skills/tree/main/plugins/test-quality-analysis/skills/test-quality-analysis

SKILL.md

Test Quality Analysis

Expert knowledge for analyzing and improving test quality - detecting test smells, overmocking, insufficient coverage, and testing anti-patterns.

Core Dimensions

Correctness: Tests verify the right behavior
Reliability: Tests are deterministic, not flaky
Maintainability: Tests are easy to understand
Performance: Tests run quickly
Coverage: Tests cover critical code paths
Isolation: Tests don't depend on external state

Test Smells

Overmocking

Problem: Mocking too many dependencies makes tests fragile.

typescript

// ❌ BAD: Overmocked
test('calculate total', () => {
  const mockAdd = vi.fn(() => 10)
  const mockMultiply = vi.fn(() => 20)
  // Testing implementation, not behavior
})

// ✅ GOOD: Mock only external dependencies
test('calculate order total', () => {
  const mockPricingAPI = vi.fn(() => ({ tax: 0.1 }))
  const total = calculateTotal(order, mockPricingAPI)
  expect(total).toBe(38)
})

Detection: More than 3-4 mocks, mocking pure functions, complex mock setup.

Fix: Mock only I/O boundaries (APIs, databases, filesystem).

Fragile Tests

Problem: Tests break with unrelated code changes.

typescript

// ❌ BAD: Tests implementation details
await page.locator('.form-container > div:nth-child(2) > button').click()

// ✅ GOOD: Semantic selector
await page.getByRole('button', { name: 'Submit' }).click()

Flaky Tests

Problem: Tests pass or fail non-deterministically.

typescript

// ❌ BAD: Race condition
test('loads data', async () => {
  fetchData()
  await new Promise(resolve => setTimeout(resolve, 1000))
  expect(data).toBeDefined()
})

// ✅ GOOD: Proper async handling
test('loads data', async () => {
  const data = await fetchData()
  expect(data).toBeDefined()
})

Poor Assertions

typescript

// ❌ BAD: Weak assertion
test('returns users', async () => {
  const users = await getUsers()
  expect(users).toBeDefined() // Too vague!
})

// ✅ GOOD: Strong, specific assertions
test('creates user with correct attributes', async () => {
  const user = await createUser({ name: 'John' })
  expect(user).toMatchObject({
    id: expect.any(Number),
    name: 'John',
  })
})

Analysis Tools

bash

# Vitest coverage (prefer bun)
bun test --coverage
open coverage/index.html

# Check thresholds
bun test --coverage --coverage.thresholds.lines=80

# pytest-cov (Python)
uv run pytest --cov --cov-report=html
open htmlcov/index.html

Best Practices Checklist

Unit Test Quality (FIRST)

Fast: Tests run in milliseconds
Isolated: No dependencies between tests
Repeatable: Same results every time
Self-validating: Clear pass/fail
Timely: Written alongside code

Mock Guidelines

Mock only external dependencies
Don't mock business logic or pure functions
Use real implementations when possible
Limit to 3-4 mocks per test maximum

Coverage Goals

80%+ line coverage for business logic
100% for critical paths (auth, payment)
All error paths tested
Boundary conditions tested

Test Structure (AAA Pattern)

typescript

test('user registration', async () => {
  // Arrange
  const userData = { email: 'user@example.com' }

  // Act
  const user = await registerUser(userData)

  // Assert
  expect(user.email).toBe('user@example.com')
})

Code Review Checklist

Tests verify behavior, not implementation
Assertions are specific and meaningful
No flaky tests (timing, ordering issues)
Proper async/await usage
Test names clearly describe behavior
Minimal code duplication
Critical paths have tests
Both happy path and error cases covered

Common Anti-Patterns

Testing Implementation Details

typescript

// ❌ BAD
const spy = vi.spyOn(Math, 'sqrt')
calculateDistance()
expect(spy).toHaveBeenCalled() // Testing how, not what

// ✅ GOOD
const distance = calculateDistance({ x: 0, y: 0 }, { x: 3, y: 4 })
expect(distance).toBe(5) // Testing output

Mocking Too Much

typescript

// ❌ BAD
const mockAdd = vi.fn((a, b) => a + b)

// ✅ GOOD: Use real implementations
import { add } from './utils'
// Only mock external services
const mockPaymentGateway = vi.fn()

Recommended Agent Skills

Expand your agent's capabilities with these related and highly-rated skills.

secondsky/claude-skills

skill-name

[TODO: Write comprehensive description in third-person. Start with "This skill provides..." or "This skill should be used when..."] [TODO: Add "Use when" scenarios - specific situations where Claude should use this skill] [TODO: Add keywords - technologies, use cases, error messages that should trigger this skill]

109 14

Explore

secondsky/claude-skills

websocket-implementation

Implements real-time WebSocket communication with connection management, room-based messaging, and horizontal scaling. Use when building chat systems, live notifications, collaborative tools, or real-time dashboards.

109 14

Explore

secondsky/claude-skills

ai-sdk-ui

109 14

Explore

secondsky/claude-skills

bun-nextjs

This skill should be used when the user asks about "Next.js with Bun", "Bun and Next", "running Next.js on Bun", "Next.js development with Bun", "create-next-app with Bun", or building Next.js applications using Bun as the runtime.

109 14

Explore

secondsky/claude-skills