ReddRadar
Agent skill
pipeline-design
Design ETL/ELT pipeline architectures with proper patterns for reliability and scalability.
Install this agent skill to your Project
npx add-skill https://github.com/timequity/vibe-coder/tree/main/skills/data/pipeline-design
SKILL.md
Pipeline Design
ETL vs ELT
| Approach | When to Use |
|---|---|
| ETL | Transform before load, limited warehouse compute |
| ELT | Modern warehouses (Snowflake, BigQuery, Redshift) |
Pipeline Patterns
Batch
Source → Extract → Stage → Transform → Load → Target
│ │
└── Checkpoint ──────┘
- Scheduled intervals (hourly, daily)
- Full or incremental loads
- Idempotent operations
Streaming
Source → Kafka/Kinesis → Process → Sink
│
└── State Store
- Real-time requirements
- Event-driven architecture
- Exactly-once semantics
Design Principles
- Idempotent - Safe to re-run
- Incremental - Process only new/changed data
- Observable - Metrics, logs, alerts
- Testable - Unit tests for transformations
- Recoverable - Checkpoints, retry logic
Staging Pattern
-- 1. Land raw data
COPY INTO raw.source_data FROM @stage;
-- 2. Deduplicate
CREATE TABLE staging.deduped AS
SELECT * FROM raw.source_data
QUALIFY ROW_NUMBER() OVER (PARTITION BY id ORDER BY _loaded_at DESC) = 1;
-- 3. Transform to target
MERGE INTO target.dim_customer
USING staging.deduped
ON target.id = staging.id
WHEN MATCHED THEN UPDATE ...
WHEN NOT MATCHED THEN INSERT ...;
Error Handling
- Dead letter queues for failed records
- Retry with exponential backoff
- Alert on threshold breaches
- Quarantine bad data for review
Recommended Agent Skills
Expand your agent's capabilities with these related and highly-rated skills.
timequity/vibe-coder
mvp-help
Help and documentation for Idea to MVP plugin. Use when: user asks about building MVPs, vibe coding, or available commands. Triggers: "help", "what can you do", "mvp help", "how to build".
timequity/vibe-coder
verification-gate
Hidden quality gate that runs before showing "Done!" to user - ensures all tests pass, build succeeds, and requirements met before claiming completion
timequity/vibe-coder
brainstorming
Refine ideas into detailed designs through Socratic dialogue. Use when: user has rough idea, needs to clarify requirements, explore approaches. Triggers: "brainstorm", "discuss idea", "I'm thinking about", "what if", "help me think through", "explore options", "/brainstorm".
timequity/vibe-coder
subagent-creator
Guide for creating effective subagents (custom agents). Use when users want to create a new subagent that can be dispatched via Task tool for autonomous work. Covers frontmatter fields (name, description, tools, model, permissionMode, skills), prompt design, and when to use subagents vs skills.
timequity/vibe-coder
backend-rust
Modern Rust backend with Axum, SQLx, tokio + CI/CD automation. Use when: building Rust APIs, high-performance services, or needing build/test/lint/audit automation. Triggers: "axum", "rust backend", "rust api", "sqlx", "tokio", "cargo build", "cargo test", "clippy", "rustfmt", "cargo-audit", "cross-compile", "rust ci", "release build", "rust security", "shuttle", "actix".
timequity/vibe-coder
test-driven-development
Write failing test first, then minimal code to pass. Red-Green-Refactor cycle. Use when: implementing features, fixing bugs, refactoring code. Triggers: "implement", "add feature", "fix bug", "tdd", "test first", "write tests", "test-driven".
Didn't find tool you were looking for?