What is pypyr?
pypyr is a free and open-source task runner designed for automation pipelines. It allows users to script sequential task workflow steps in YAML, providing features like conditional execution, loops, error handling, and retries. The tool simplifies automation by combining commands, different scripts in various languages, and applications into a single pipeline process, making it easier to manage than complex shell scripts or makefiles.
With pypyr, users can run any combination of commands, shell scripts, external executables, callables, and inline code in the same pipeline. It includes built-in steps for tasks such as reading and writing files, formatting JSON, TOML, and YAML, and manipulating data structures. The pipeline YAML format is human-readable and editable, encouraging manual authoring and easy source control with text-based diffs. pypyr supports control flow with conditional execution, branching, looping, and parallel execution, allowing users to run programs concurrently without coding. It also offers automatic retries, error handling, variable interpolation, configuration file tokenization, and a CLI with sensible defaults, along with a Python API for programmatic invocation.
Features
- Task Runner: Automates sequential workflows by combining commands, scripts, and applications into pipelines
- YAML Format: Uses human-readable and editable YAML for defining pipeline steps, encouraging manual authoring and easy source control
- Control Flow: Supports conditional execution, branching, looping, and parallel execution without requiring coding
- Error Handling: Provides automatic retries, error handling, and compensation mechanisms for robust automation
- Variable Interpolation: Allows string interpolation and substitution of placeholders with variable values, including complex data types
- Configuration Management: Loads, merges, formats, and interpolates values in configuration files like JSON, TOML, and YAML
- CLI and API: Offers a command-line interface with minimal typing and a Python API for programmatic pipeline invocation
- Built-in Steps: Includes over 30 ready-made built-in steps for common tasks, with support for custom step creation
Use Cases
- Automating CI/CD processes for DevOps tasks
- Consolidating ad hoc scripts into repeatable pipelines
- Running the same CI/CD process locally as on cloud providers
- Preparing configuration files for systems like Terraform or CloudFormation
- Managing sequential task workflows in development and operations
- Automating repetitive tasks in software development and deployment
- Handling error-prone scripts with retries and compensation logic
- Creating modular and reusable automation sequences for various applications
