{"data":{"slug":"pditommaso-awesome-pipeline","name":"awesome-pipeline","tagline":"A curated list of awesome pipeline toolkits inspired by Awesome Sysadmin","github_url":"https://github.com/pditommaso/awesome-pipeline","owner":"pditommaso","repo":"awesome-pipeline","owner_avatar_url":"https://avatars.githubusercontent.com/u/816968?v=4","primary_language":null,"stars":6603,"forks":654,"topics":["awesome-list","workflow"],"archived":false,"github_pushed_at":"2026-07-08T08:23:21+00:00","maintenance_label":"Active","url":"https://www.graphcanon.com/tools/pditommaso-awesome-pipeline","markdown_url":"https://www.graphcanon.com/tools/pditommaso-awesome-pipeline.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/pditommaso-awesome-pipeline","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=pditommaso-awesome-pipeline","description":"A curated list of awesome pipeline toolkits inspired by Awesome Sysadmin","homepage_url":null,"license":null,"open_issues":34,"watchers":227,"ai_summary":null,"readme_excerpt":"Awesome Pipeline\n================\n\nA curated list of awesome pipeline toolkits inspired by [Awesome Sysadmin](https://github.com/kahun/awesome-sysadmin)\n\nPipeline frameworks & libraries\n--------------------------------\n\n* [ActionChain](http://docs.stackstorm.com/actionchain.html) - A workflow system for simple linear success/failure workflows.\n* [Adage](https://github.com/diana-hep/adage) - Small package to describe workflows that are not completely known at definition time.\n* [AiiDA](https://github.com/aiidateam/aiida-core) - workflow manager with a strong focus on provenance, performance and extensibility.\n* [Airflow](https://github.com/airbnb/airflow) - Python-based workflow system created by AirBnb.\n* [Anduril](http://www.anduril.org/anduril/site/) - Component-based workflow framework for scientific data analysis.\n* [Antha](https://www.antha-lang.org/) - High-level language for biology.\n* [Argo Workflows](https://argoproj.github.io/argo-workflows/) - Container-native workflow engine for orchestrating parallel data processing, ML, or CI jobs on Kubernetes.\n* [Autosubmit](https://autosubmit.readthedocs.io/) - An open source Python experiment and workflow manager used to manage complex workflows on Cloud and HPC platforms. \n* [AWE](https://github.com/MG-RAST/AWE/) - Workflow and resource management system with CWL support.\n* [Balsam](https://github.com/argonne-lcf/balsam) - Python-based high throughput task and workflow engine.\n* [Bds](http://pcingola.github.io/BigDataScript/) - Scripting language for data pipelines.\n* [Beam](https://beam.apache.org/) - Unified programming model for batch and streaming data-parallel processing pipelines.\n* [BioMake](https://github.com/evoldoers/biomake) - GNU-Make-like utility for managing builds and complex workflows.\n* [BioQueue](https://github.com/liyao001/BioQueue) - Explicit framework with web monitoring and resource estimation.\n* [Bioshake](https://github.com/papenfusslab/bioshake) - Haskell DSL built on shake with strong typing and EDAM support.\n* [Bistro](https://github.com/pveber/bistro) - Library to build and execute typed scientific workflows.\n* [Bpipe](https://github.com/ssadedin/bpipe/) - Tool for running and managing bioinformatics pipelines.\n* [Briefly](https://github.com/bloomreach/briefly) - Python Meta-programming Library for Job Flow Control.\n* [Burr](https://github.com/dagworks-inc/burr) - Python based lightweight graph (i.e. can do loops and conditional branching, and not just DAGs) orchestrator.\n* [Cluster Flow](http://clusterflow.io) - Command-line tool which uses common cluster managers to run bioinformatics pipelines.\n* [Clusterjob](https://github.com/monajemi/clusterjob) - Automated reproducibility, and hassle-free submission of computational jobs to clusters.\n* [Cocoindex](https://github.com/cocoindex-io/cocoindex) - ETL framework to build fresh index.\n* [Compi](https://www.sing-group.org/compi) - Application framework for portable computational pipelines.\n* [Compss](https://www.bsc.es/research-and-development/software-and-apps/software-list/comp-superscalar) - Programming model for distributed infrastructures.\n* [Conan2](https://github.com/tburdett/Conan2) - Light-weight workflow management application.\n* [Consecution](https://github.com/robdmc/consecution) - A Python pipeline abstraction inspired by Apache Storm topologies.\n* [Cosmos](https://mizzou-cbmi.github.io/) - Python library for massively parallel workflows.\n* [Couler](https://github.com/couler-proj/couler) - Unified interface for constructing and managing workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow.\n* [Covalent](https://github.com/AgnostiqHQ/covalent) - Workflow orchestration toolkit for high-performance and quantum computing research and development.\n* [Cromwell](https://github.com/broadinstitute/cromwell) - Workflow Management System geared towards scientific workflows from the Broad Institute.\n* [Cuneiform](https://github.com/joergen7/cuneiform)","github_created_at":"2014-07-19T15:48:56+00:00","created_at":"2026-07-15T10:46:48.83553+00:00","updated_at":"2026-07-15T11:15:21.493917+00:00","categories":[{"slug":"ai-agents","name":"AI Agents","url":"https://www.graphcanon.com/categories/ai-agents","markdown_url":"https://www.graphcanon.com/categories/ai-agents.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/ai-agents"},{"slug":"data-retrieval","name":"Data & Retrieval","url":"https://www.graphcanon.com/categories/data-retrieval","markdown_url":"https://www.graphcanon.com/categories/data-retrieval.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/data-retrieval"},{"slug":"developer-tools","name":"Developer Tools","url":"https://www.graphcanon.com/categories/developer-tools","markdown_url":"https://www.graphcanon.com/categories/developer-tools.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/developer-tools"}],"tags":[{"slug":"awesome-list","name":"awesome-list"},{"slug":"workflow","name":"workflow"}],"trust":{"provenance":{"is_fork":false,"github_id":22013111,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-15T10:46:49.848Z","maintenance":{"label":"Active","score":82,"methodology":"github_public_v1","releases_90d":0,"days_since_push":7,"last_release_at":null},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-15T10:46:50.298Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-15T10:46:49.578Z"}}}}