Agent skill
run-cpython-tests
Run the CPython regression test suite in this repo via scripts/run_cpython_tests.sh with parallel execution, enforce a per-test timeout, capture output to cpython_full_test_run.log, and summarize failures from that log.
Stars
163
Forks
31
Install this agent skill to your Project
npx add-skill https://github.com/majiayu000/claude-skill-registry/tree/main/skills/development/run-cpython-tests
SKILL.md
Run CPython tests
Run the full suite
- Run with full permissions, including network.
- Use a 3-minute per-test timeout and keep parallelism enabled (run_cpython_tests.sh already uses -j0).
- If the local
cpython/checkout is missing the target branch, re-run withFETCH_CPYTHON=1to allow a network update. - Capture output to a single log file while preserving the exit status:
bash
set -o pipefail
./scripts/run_cpython_tests.sh --timeout 180 2>&1 | tee cpython_full_test_run.log
Summarize failures from the log
- Locate failure anchors:
bash
rg -n "^(FAIL|ERROR|TIMEOUT|CRASHED|INTERRUPTED|LEAKED|ENV_CHANGED):" cpython_full_test_run.log
- Extract each failure block (look for the separator lines of ===) and classify the failure based on the first error line after the traceback.
- Use these categories in the summary:
- FAIL: assertion mismatch or explicit test failure; call out the mismatched expectation.
- ERROR: unexpected exception; report the exception type and message.
- TIMEOUT: test exceeded 180 seconds; call out the hang/timeout.
- CRASHED/FATAL: interpreter crash; mention the fatal error or signal.
- LEAKED/ENV_CHANGED: resource leak or environment mutation; mention the resource.
- Summarize each failing test as:
test_name: <category> - <short reason from output>.
Didn't find tool you were looking for?