{"data":{"slug":"devnen-qwen3-6-windows-server","name":"qwen3.6-windows-server","tagline":"One-click Qwen3.6-27B inference tool for Windows","github_url":"https://github.com/devnen/qwen3.6-windows-server","owner":"devnen","repo":"qwen3.6-windows-server","owner_avatar_url":"https://avatars.githubusercontent.com/u/195903272?v=4","primary_language":"Python","stars":229,"forks":23,"topics":["llm-inference","local-llm","offline-ai","privacy","qwen","qwen3","rtx-3090","textual-tui","vllm","windows"],"archived":false,"github_pushed_at":"2026-05-14T17:14:43+00:00","maintenance_label":"Slowing","stars_delta_30d":2,"url":"https://www.graphcanon.com/tools/devnen-qwen3-6-windows-server","markdown_url":"https://www.graphcanon.com/tools/devnen-qwen3-6-windows-server.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/devnen-qwen3-6-windows-server","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=devnen-qwen3-6-windows-server","description":"One-click Qwen3.6-27B inference on Windows. 158 tok/s on RTX 5090, 72 tok/s on RTX 3090. Native, no WSL, no Docker, no telemetry.","homepage_url":null,"license":null,"open_issues":9,"watchers":3,"ai_summary":"Provides native installation and execution of Qwen3.6-27B model on Windows without WSL or Docker. Offers speed rates at 158 tok/s on RTX 5090, 72 tok/s on RTX 3090.","readme_excerpt":"## Install\n\n**TL;DR for CI / agents / scripted installs**, one line, no TUI:\n\n```powershell\nstart.bat --auto-download --snapshot start_72tps\n```\n\nThat installs the runtime, downloads the model if missing, and starts\nserving on `http://127.0.0.1:5001/v1`. See\n[Headless / scripted install](#headless--scripted-install) below for\nall the flags.\n\n**Hand the install to a coding agent**, copy/paste prompt at\n[`docs/AGENT_INSTALL_PROMPT.md`](docs/AGENT_INSTALL_PROMPT.md). Edit the\none `INSTALL_DIR` line, paste into Claude Code / Cursor / Codex CLI /\nany agent with shell access, and it does the download + extract +\nruntime install + model fetch + smoke test end-to-end while you do\nsomething else.\n\n**Interactive path:**\n\n1. Download the right zip for your GPU from the\n   [latest Release](../../releases/latest):\n   - `qwen3.6-windows-server-portable-x64-ampere.zip` for 30-series / 40-series (Ampere, Ada).\n   - `qwen3.6-windows-server-portable-x64-blackwell.zip` for 50-series (Blackwell).\n\n   Extract anywhere (no admin needed).\n2. Double-click `start.bat`. The first run does two one-time steps,\n   then drops you in the TUI:\n   - **Runtime install** (~5–15 min, several GB). The bundled vLLM\n     wheel + ~150 transitive deps (torch, CUDA wheels, transformers,\n     etc.) install into the embedded Python's `site-packages`. A\n     marker file is written so subsequent launches skip this entirely.\n   - **Model setup.** Looks for `Qwen3.6-27B-int4-AutoRound` weights\n     on your fixed drives (scans `<drive>:\\`, `_models\\`, `models\\`,\n     `AI\\`, `AI\\models\\`, `huggingface\\`, `huggingface\\hub\\`,\n     `models\\Lorbus\\`). If it doesn't find them, offers to\n     **auto-download from Hugging Face** (~16 GB, public, no token)\n     or accepts a path to weights you already have. If your weights\n     live somewhere else, pass `--model-dir <path>` to skip the scan.\n3. Pick a snapshot, press Enter, you're serving on\n   `http://127.0.0.1:5001/v1`.\n\nThe portable zip ships with an embedded Python 3.12 runtime, the\npatched vLLM wheel, the launcher TUI, a portable Windows Terminal,\nand a vendored `get-pip.py`. No conda, no system-Python install, no\nregistry changes, no admin prompts. The runtime install on first run\nis the only network-dependent step besides the model download.\n\nDon't have the model yet? See [`docs/MTP_HEAD.md`](docs/MTP_HEAD.md),\n**use the Lorbus AutoRound quant**, the others won't draft.\n\nDetailed install (including the wheel-only path for users who already\nhave their own venv): [`docs/INSTALL.md`](docs/INSTALL.md).\n\n---\n\n## Optional: install MSVC 2022 for the small decode boost\n\nThe launcher works on a vanilla Windows install, no MSVC required.\nBut if you install **Visual Studio 2022 Build Tools** (free, no full\nIDE) with the **\"Desktop development with C++\"** workload, the\nsnapshots auto-detect it and turn on vLLM's flashinfer sampler path,\nwhich JIT-compiles a faster top-k / top-p kernel on first launch.\n\nWhat it costs:\n- ~7 GB download, one-time install.\n- Extra 30 to 60 s on the first `profile_run` of each new snapshot\n  while the kernel compiles. Subsequent boots reuse the compiled\n  cache.\n\nWhat you get:\n- A small but measurable decode boost on the sampler path.\n\nWithout MSVC, the snapshots transparently fall back to the PyTorch\nsampler, which never JIT-compiles anything. Boot is faster and the\nserver is reliable; you just leave a few percent of decode tok/s on\nthe table. The launcher prints a one-line `[info]` at startup telling\nyou which path it picked.\n\nGet the Build Tools installer here (official Microsoft `aka.ms`\nshortlink, pinned to VS 2022 / 17.x so it stays on the right product\neven after VS 2026 ships):\nhttps://aka.ms/vs/17/release/vs_buildtools.exe\n\nninja (the other half of the JIT toolchain) ships inside the\nlauncher zip, you don't need to install it separately.\n\n---\n\n## Headless / scripted install\n\nEnd-to-end automated install (no TUI, no prompts), useful for CI,\nremote machines, agent installers, or just keeping a repeatabl","github_created_at":"2026-04-29T14:41:37+00:00","created_at":"2026-07-15T11:00:31.259066+00:00","updated_at":"2026-09-20T05:05:19.23834+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"}],"tags":[{"slug":"llm-inference","name":"llm-inference"},{"slug":"local-llm","name":"local-llm"},{"slug":"offline-ai","name":"offline-ai"},{"slug":"privacy","name":"privacy"},{"slug":"qwen","name":"qwen"},{"slug":"vllm","name":"vllm"},{"slug":"windows","name":"windows"}],"trust":{"provenance":{"is_fork":false,"github_id":1224703318,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-20T05:05:16.736Z","maintenance":{"label":"Slowing","score":36,"methodology":"github_public_v1","releases_90d":0,"days_since_push":128,"last_release_at":"2026-05-14T17:10:20Z","stars_delta_30d":2,"open_issues_delta_30d":1},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-15T11:00:32.427Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-20T05:05:17.755Z"},"languages":{"value":["python"],"source":"github.language","observed_at":"2026-09-20T05:05:17.755Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["Need to run Qwen3.6-27B natively on Windows with no reliance on WSL or Docker","Seeking privacy as it offers offline operation without telemetry"],"when_not_to_use":["Require Linux OS, since this tool avoids using WSL for native performance","If looking for multi-GPU support, as specified speeds are single-GPU focused"],"source":"enrich:decision_facts","observed_at":"2026-07-17T08:12:01.946Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"Native Windows tool for Qwen3.6-27B inference without WSL or Docker. Offers speed of 158 tok/s on RTX 5090, 72 tok/s on RTX 3090."}]}}