hamilton
apache/hamilton
Apache Hamilton helps define modular, self-documenting dataflows for portable and expressive data transformation DAGs.
Overview
A lightweight Python library for creating DAGs of data transformations that are highly portable (running where Python is) and offer expressive features like validation and experiment tracking.
Categories
Tags
Similar tools
ECC
affaan-m/ECC
The agent harness performance optimization system
AutoGPT
Significant-Gravitas/AutoGPT
AutoGPT: Build, Deploy, and Run AI Agents
prompts.chat
f/prompts.chat
The world's largest open-source prompt library for AI
transformers
huggingface/transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models
JavaGuide
Snailclimb/JavaGuide
Snailclimb/JavaGuide: 面试 & 后端通用面试指南，覆盖计算机基础、数据库、分布式、高并发、系统设计与 AI 应用开发
langflow
langflow-ai/langflow
Langflow is a powerful platform for building and deploying AI-powered agents and workflows.
Install
git clone https://github.com/apache/hamiltonREADME
Apache Hamilton — portable & expressive
data transformation DAGs
Disclaimer
Apache Hamilton is an effort undergoing incubation at the Apache Software Foundation (ASF), sponsored by the Apache Incubator PMC.
Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects.
While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.
Apache Hamilton (incubating) is a lightweight Python library for directed acyclic graphs (DAGs) of data transformations. Your DAG is portable; it runs anywhere Python runs, whether it's a script, notebook, Airflow pipeline, FastAPI server, etc. Your DAG is expressive; Apache Hamilton has extensive features to define and modify the execution of a DAG (e.g., data validation, experiment tracking, remote execution).
To create a DAG, write regular Python functions that specify their dependencies with their parameters. As shown below, it results in readable code that can always be visualized. Apache Hamilton loads that definition and automatically builds the DAG for you!
B() and C() refer to function A via their parameters
Apache Hamilton brings modularity and structure to any Python application moving data: ETL pipelines, ML workflows, LLM applications, RAG systems, BI dashboards, and the Apache Hamilton UI allows you to automatically visualize, catalog, and monitor execution.
Apache Hamilton is great for DAGs, but if you need loops or conditional logic to create an LLM agent or a simulation, take a look at our sister library Burr 🤖 .
Installation
Apache Hamilton supports Python 3.8+. We include the optional visualization dependency to display our Apache Hamilton DAG. For visualizations, Graphviz needs to be installed on your system separately.
pip install "apache-hamilton[visualization]"
To use the Apache Hamilton UI, install the ui and sdk dependencies.
pip install "apache-hamilton[ui,sdk]"
To try Apache Hamilton i