{"data":{"slug":"hanxiao-dataroom","name":"dataroom","tagline":"Local LLM research harness for querying Pi with Qwen3.6","github_url":"https://github.com/hanxiao/dataroom","owner":"hanxiao","repo":"dataroom","owner_avatar_url":"https://avatars.githubusercontent.com/u/2041322?v=4","primary_language":"Python","stars":193,"forks":17,"topics":["harness","local-llm","pi"],"archived":false,"github_pushed_at":"2026-06-20T21:57:30+00:00","maintenance_label":"Slowing","stars_delta_30d":5,"url":"https://www.graphcanon.com/tools/hanxiao-dataroom","markdown_url":"https://www.graphcanon.com/tools/hanxiao-dataroom.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/hanxiao-dataroom","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=hanxiao-dataroom","description":"Give a query, get a dataroom. Pi + self-hosted Qwen3.6 research harness on a single L4.","homepage_url":"https://dataroom.hanxiao.io","license":"MIT","open_issues":3,"watchers":0,"ai_summary":"Provides infrastructure to conduct local experimentation and queries using an LLM on self-hosted data.","readme_excerpt":"# Dataroom\n\nGive it a query. A local model in a [Pi](https://pi.dev) harness loops search-read-write until it has built a comprehensive, fully-cited **dataroom** on disk - a `.zip` you hand to a frontier model for the long-horizon task.\n\n<p align=\"center\">\n  <img src=\"assets/banner.png\" width=\"860\"\n       alt=\"Give a query to a self-hosted pi + harness + local model loop; it loops search-read-write to build a dataroom and hands you a .zip\" />\n</p>\n\n<p align=\"center\">\n  <b>Live demo → <a href=\"https://dataroom.hanxiao.io\">dataroom.hanxiao.io</a></b>\n</p>\n\n## Why\n\n[For long-horizon task you need a grounded, well-organized knowledge dump before the real work can start.](https://x.com/hxiao/status/2044765001370701981?s=20) That upfront research is mostly a search-read-write loop, and few things are usually wrong with how it gets done today.\n\n- **Research is mechanical, so don't pay frontier tokens for it.** Gathering and organizing sources is tool-calling, not deep reasoning - a small local model in a disciplined harness (search, dedup, cite, verify) does it fine. And because it runs on your own GPU at near-zero marginal token cost, it can keep going for hours until the dataroom is actually comprehensive, instead of stopping when a metered budget runs out.\n- **The output is context for a machine, not a report for a human.** A 2025-style deep-research run ends in a long PDF nobody reads. Dataroom ends in a structured `.zip` - `topics/`, `sources/`, `data/`, a `SUMMARY.md`, every claim cited - built to be consumed by the next agent, not skimmed.\n- **It is stage one of a two-stage pipeline.** Unzip the dataroom into a frontier model's context and let it do the expensive second stage (usually implementation). The research does not have to be perfect - its consumer is intelligent and can spot gaps - it has to be comprehensive and grounded.\n\nEverything runs locally on your own GPU: the model is self-hosted (llama.cpp), and the only thing that leaves the box is the web search/read the agent chooses to do.\n\n## How it works\n\n<p align=\"center\">\n  <img src=\"assets/screenshot-home.png\" width=\"800\"\n       alt=\"Dataroom homepage: a query box and a live list of jobs with status, file counts, and pause/resume/download controls\" />\n</p>\n\nSubmit a query and an async job spins up a headless Pi coding agent backed by a self-hosted Qwen3.6-35B-A3B (llama.cpp). The agent runs its own research loop: `pi --mode json --continue` resumes the same per-cwd session across turns, and on each turn it searches, reads, reranks, and writes sourced files into a `dataroom/` directory on disk.\n\n- Autonomous loop: the agent is not micromanaged. It is handed tools and a one-page methodology, then drives itself - search, read, dedup, write, verify - until the work is done.\n- Outcome-based stopping: `DONE` is honored only once the dataroom holds enough substantive sourced files, all sub-questions are closed, and a `SUMMARY.md` exists. Turns / seconds / Jina-call caps are only hard backstops, and a premature `DONE` is rejected so the agent keeps going. The reason it stopped is surfaced on the dashboard.\n- jina CLI: the `jina` CLI is on PATH (search / read / rerank / embed / dedup), driven from bash and composable via pipes (`jina search Q | jina rerank R`, `cat urls.txt | jina read`, `xargs -P 8` for parallel fan-out) so bulky intermediates stay out of the LLM context. \n- Embedding dedup index: `jina-embeddings-v5-nano` is preloaded for the dataroom index (embed / semantic search / dedup), with server-side reconciliation so it never drifts from disk. The agent must search the index before adding anything, to avoid duplicates and keep structure.\n- Live dashboard: real-time context utilization, throughput, tool-call distribution, live activity feed, warnings/errors, progress-to-floor, a stop-reason banner, and the dataroom file tree, at `GET /jobs/{id}/dashboard`.\n\nThe [live dashboard](https://dataroom.hanxiao.io) for a finished job - progress-to-floor, total tokens, tool-c","github_created_at":"2026-05-31T03:24:22+00:00","created_at":"2026-07-15T11:01:14.473468+00:00","updated_at":"2026-09-20T05:07:12.846416+00:00","categories":[{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"harness","name":"harness"},{"slug":"local-llm","name":"local-llm"},{"slug":"pi","name":"pi"},{"slug":"qwen3-6","name":"qwen3.6"},{"slug":"research-tools","name":"research-tools"}],"trust":{"provenance":{"is_fork":false,"github_id":1254819227,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-20T05:07:09.768Z","maintenance":{"label":"Slowing","score":36,"methodology":"github_public_v1","releases_90d":0,"days_since_push":91,"last_release_at":null,"stars_delta_30d":5,"open_issues_delta_30d":0},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-15T11:01:15.660Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-20T05:07:11.039Z"},"deploy":{"source":"dockerfile:Dockerfile","self_host":true,"observed_at":"2026-09-20T05:07:11.039Z","managed_saas":false},"languages":{"value":["python"],"source":"github.language","observed_at":"2026-09-20T05:07:11.039Z"},"has_docker":{"value":true,"source":"dockerfile:Dockerfile","observed_at":"2026-09-20T05:07:11.039Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-09-20T05:07:11.039Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["When you need to run experiments on locally hosted datasets, as dataroom specifically supports querying Pi with Qwen3.6.","If your project involves fine-tuning or testing LLMs without the dependency on cloud services, dataroom's local setup is ideal."],"when_not_to_use":["Avoid using this tool if you require real-time access to a wide variety of datasets outside of what can be locally hosted.","Do not use it if your project demands integration with other cloud-based AI tools or services, as dataroom focuses on local infrastructure."],"source":"enrich:decision_facts","observed_at":"2026-07-17T07:07:41.510Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"Dataroom is an LLM research platform built for experimenting with self-hosted data using Qwen3.6 in conjunction with Pi."}]}}