ReddRadar
Agent skill
experimental-design
Best practices for designing reproducible ML experiments. Use when planning ablations, baselines, or controlled experiments.
Install this agent skill to your Project
npx add-skill https://github.com/aiming-lab/AutoResearchClaw/tree/main/researchclaw/skills/builtin/experiment/experimental-design
Metadata
Additional technical details for this skill
- author
- researchclaw
- version
- 1.0
- category
- experiment
- priority
- 2
- references
- Bouthillier et al., Accounting for Variance in ML Benchmarks, MLSys 2021
- trigger keywords
- experiment,ablation,baseline,control,hypothesis,reproducib
- applicable stages
- 9,10,12
SKILL.md
Experimental Design Best Practice
- ALWAYS include meaningful baselines (not just random):
- At least one classical method baseline
- At least one recent SOTA method baseline
- A simple-but-strong baseline (e.g., linear probe, k-NN)
- Use MULTIPLE random seeds (minimum 3, ideally 5)
- Report mean +/- std across seeds
- Design ablations that isolate EACH key component:
- Remove one component at a time
- Each ablation must be meaningfully different from baseline
- Control variables: change only ONE thing per comparison
- Use standard splits (train/val/test) — never test on training data
- Report wall-clock time and memory usage alongside accuracy
Recommended Agent Skills
Expand your agent's capabilities with these related and highly-rated skills.
aiming-lab/AutoResearchClaw
scientific-visualization
Publication-ready scientific figure design with matplotlib and seaborn. Use when creating journal submission figures with proper formatting, accessibility, and statistical annotations.
aiming-lab/AutoResearchClaw
hypothesis-formulation
Structured scientific hypothesis generation from observations. Use when formulating testable hypotheses, competing explanations, or experimental predictions.
aiming-lab/AutoResearchClaw
scientific-writing
Academic manuscript writing with IMRAD structure, citation formatting, and reporting guidelines. Use when drafting or revising research papers.
aiming-lab/AutoResearchClaw
a-evolve
Apply A-Evolve's agentic evolution methodology to improve AI agent performance across runs. Use when the user wants to diagnose agent failures, generate targeted skills from error patterns, evolve system prompts, or accumulate episodic knowledge. Works standalone or inside AutoResearchClaw pipelines. Triggers on: "evolve", "self-improve", "diagnose failures", "generate skills from errors", "what went wrong and how to fix it", or any mention of A-Evolve.
aiming-lab/AutoResearchClaw
chemistry-rdkit
Computational chemistry with RDKit for molecular analysis, descriptors, fingerprints, and substructure search. Use when working with SMILES, drug discovery, or cheminformatics tasks.
aiming-lab/AutoResearchClaw
literature-search
Systematic literature review methodology including search strategy, screening, and synthesis. Use when conducting literature reviews or writing background sections.
Didn't find tool you were looking for?