ReddRadar
Agent skill
nlp-pretraining
Best practices for language model pretraining and fine-tuning. Use when generating or reviewing NLP training code.
Install this agent skill to your Project
npx add-skill https://github.com/aiming-lab/AutoResearchClaw/tree/main/researchclaw/skills/builtin/domain/nlp-pretraining
Metadata
Additional technical details for this skill
- author
- researchclaw
- version
- 1.0
- category
- domain
- priority
- 3
- references
- Devlin et al., BERT, NAACL 2019; Hu et al., LoRA, ICLR 2022
- trigger keywords
- language model,pretraining,fine-tuning,bert,gpt,llm,transformer,nlp,text
- applicable stages
- 9,10
SKILL.md
NLP Pretraining/Fine-tuning Best Practice
Fine-tuning recipe:
- Use pre-trained checkpoints (HuggingFace hub)
- AdamW optimizer, lr=2e-5 to 5e-5
- Linear warmup (6% of total steps) + linear decay
- Batch size: 16-32 (use gradient accumulation for larger effective batch)
- 3-5 epochs for classification, 1-2 for generation
- Weight decay: 0.01
Parameter-efficient methods:
- LoRA: r=8-64, alpha=16-128, apply to q/v projections
- Prefix tuning: 10-20 prefix tokens
- Adapters: bottleneck dimension 64-256
Evaluation:
- Classification: accuracy, F1 (macro for imbalanced)
- Generation: perplexity, BLEU/ROUGE, human evaluation
- Use multiple seeds and report mean +/- std
Recommended Agent Skills
Expand your agent's capabilities with these related and highly-rated skills.
aiming-lab/AutoResearchClaw
scientific-visualization
Publication-ready scientific figure design with matplotlib and seaborn. Use when creating journal submission figures with proper formatting, accessibility, and statistical annotations.
aiming-lab/AutoResearchClaw
hypothesis-formulation
Structured scientific hypothesis generation from observations. Use when formulating testable hypotheses, competing explanations, or experimental predictions.
aiming-lab/AutoResearchClaw
scientific-writing
Academic manuscript writing with IMRAD structure, citation formatting, and reporting guidelines. Use when drafting or revising research papers.
aiming-lab/AutoResearchClaw
a-evolve
Apply A-Evolve's agentic evolution methodology to improve AI agent performance across runs. Use when the user wants to diagnose agent failures, generate targeted skills from error patterns, evolve system prompts, or accumulate episodic knowledge. Works standalone or inside AutoResearchClaw pipelines. Triggers on: "evolve", "self-improve", "diagnose failures", "generate skills from errors", "what went wrong and how to fix it", or any mention of A-Evolve.
aiming-lab/AutoResearchClaw
chemistry-rdkit
Computational chemistry with RDKit for molecular analysis, descriptors, fingerprints, and substructure search. Use when working with SMILES, drug discovery, or cheminformatics tasks.
aiming-lab/AutoResearchClaw
literature-search
Systematic literature review methodology including search strategy, screening, and synthesis. Use when conducting literature reviews or writing background sections.
Didn't find tool you were looking for?