{"data":{"slug":"heshengtao-comfyui-llm-party","name":"comfyui_LLM_party","tagline":"LLM Agent Framework in ComfyUI with various nodes and adapters for different LLMs and VLMs","github_url":"https://github.com/heshengtao/comfyui_LLM_party","owner":"heshengtao","repo":"comfyui_LLM_party","owner_avatar_url":"https://avatars.githubusercontent.com/u/20573295?v=4","primary_language":"Python","stars":2361,"forks":206,"topics":["agent","comfyui","dify","flux","gemini","gguf","gpt-sovits","graphrag","janus-pro","linux","llama","mcp","o1","ocr","ollama","omost","openai","vlm","workflow"],"archived":false,"github_pushed_at":"2026-07-29T18:05:53+00:00","maintenance_label":"Steady","stars_delta_30d":31,"url":"https://www.graphcanon.com/tools/heshengtao-comfyui-llm-party","markdown_url":"https://www.graphcanon.com/tools/heshengtao-comfyui-llm-party.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/heshengtao-comfyui-llm-party","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=heshengtao-comfyui-llm-party","description":"LLM Agent Framework in ComfyUI includes MCP sever, Omost,GPT-sovits, ChatTTS,GOT-OCR2.0, and FLUX prompt nodes,access to Feishu,discord,and adapts to all llms with similar openai / aisuite interfaces, such as o1,ollama, gemini, grok, qwen, GLM, deepseek, kimi,doubao. Adapted to local llms, vlm, gguf such as llama-3.3 Janus-Pro, Linkage graphRAG","homepage_url":null,"license":"AGPL-3.0","open_issues":87,"watchers":17,"ai_summary":"Comprehensive framework supporting MCP server, Omost, GPT-sovits, ChatTTS, GOT-OCR2.0, FLUX prompt nodes, Feishu and Discord access, adaptable to many LLM types including Qwen, GLM, Gemini, Grok. Supports local models like llama-3.3, Janus-Pro, Linkage graphRAG.","readme_excerpt":"## Quick Start\n0. If you have never used ComfyUI and encounter some dependency issues while installing the LLM party in ComfyUI, please click [here](https://drive.google.com/file/d/1T9C7gEbd-w_zf9GqZO1VeI3z8ek8clpX/view?usp=sharing) to download the **Windows** portable package that includes the LLM party. Please note that this portable package contains only the party and manager plugins, and is exclusively compatible with the Windows operating system.(If you need to install LLM party into an existing comfyui, this step can be skipped.) \n1. Drag the following workflows into your comfyui, then use [comfyui-Manager](https://github.com/ltdrdata/ComfyUI-Manager) to install the missing nodes.\n  - Use API to call LLM: [start_with_LLM_api](workflow/start_with_LLM_api.json)\n  - Using aisuite to call LLM: [start_with_aisuite](workflow/start_with_aisuite.json)\n  - Manage local LLM with ollama: [start_with_Ollama](workflow/ollama.json)\n  - Use local LLM in distributed format: [start_with_LLM_local](workflow/start_with_LLM_local.json)\n  - Use local LLM in GGUF format: [start_with_LLM_GGUF](workflow/start_with_GGUF.json)\n  - Use local VLM in distributed format: [start_with_VLM_local](workflow/start_with_VLM_local.json) (Currently, support is extended for [Llama-3.2-Vision](https://huggingface.co/meta-llama/Llama-3.2-11B-Vision-Instruct)/[Qwen/Qwen2.5-VL](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct)/[deepseek-ai/Janus-Pro](https://huggingface.co/deepseek-ai/Janus-Pro-1B).)\n  - Use local VLM in GGUF format: [start_with_VLM_GGUF](workflow/start_with_llava.json)\n  - Utilize API calls to LLM for generating SD prompts and images: [start_with_VLM_API_for_SD](workflow/start_with_VLM_API_for_SD.json)\n  - Employ ollama to call minicpm for generating SD prompts and images: [start_with_ollama_minicpm_for_SD](workflow/start_with_ollama_minicpm_for_SD.json)\n  - Utilize the local qwen-vl to generate SD prompts and images: [start_with_qwen_vl_local_for_SD](workflow/start_with_qwen_vl_local_for_SD.json)\n2. If you are using API, fill in your `base_url` (it can be a relay API, make sure it ends with `/v1/`), for example: `https://api.openai.com/v1/` and `api_key` in the API LLM loader node.\n3. If you are using ollama, turn on the `is_ollama` option in the API LLM loader node, no need to fill in `base_url` and `api_key`.\n4. If you are using a local model, fill in your model path in the local model loader node, for example: `E:\\model\\Llama-3.2-1B-Instruct`. You can also fill in the Huggingface model repo id in the local model loader node, for example: `lllyasviel/omost-llama-3-8b-4bits`.\n5. Due to the high usage threshold of this project, even if you choose the quick start, I hope you can patiently read through the project homepage.\n\n---\n\n## Environment Deployment\n1. Navigate to the `comfyui_LLM_party` project folder.\n2. Enter `pip install -r requirements.txt` in the terminal to deploy the third-party libraries required by the project into the comfyui environment. Please ensure you are installing within the comfyui environment and pay attention to any `pip` errors in the terminal.\n3. If you are using the comfyui launcher, you need to enter `path_in_launcher_configuration\\python_embeded\\python.exe -m pip install -r requirements.txt` in the terminal to install. The `python_embeded` folder is usually at the same level as your `ComfyUI` folder.\n4. If you have some environment configuration problems, you can try to use the dependencies in `requirements_fixed.txt`.\n\n---\n\n## Next Steps Plan:\n1. More model adaptations;\n2. More ways to build agents;\n3. More automation features;\n4. More knowledge base management features;\n5. More tools, more personas.","github_created_at":"2024-04-13T13:42:49+00:00","created_at":"2026-07-15T10:48:50.11849+00:00","updated_at":"2026-09-20T04:27:37.494865+00:00","categories":[{"slug":"data-retrieval","name":"Data & Retrieval","url":"https://www.graphcanon.com/categories/data-retrieval","markdown_url":"https://www.graphcanon.com/categories/data-retrieval.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/data-retrieval"},{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"},{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"agent","name":"agent"},{"slug":"comfyui","name":"comfyui"},{"slug":"dify","name":"dify"},{"slug":"flux","name":"flux"},{"slug":"gemini","name":"gemini"},{"slug":"gguf","name":"gguf"},{"slug":"gpt-sovits","name":"gpt-sovits"},{"slug":"graphrag","name":"graphrag"}],"trust":{"provenance":{"is_fork":false,"github_id":786118500,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-17T06:00:06.021Z","maintenance":{"label":"Steady","score":60,"methodology":"github_public_v1","releases_90d":0,"days_since_push":49,"last_release_at":"2025-01-15T09:34:04Z","stars_delta_30d":31,"open_issues_delta_30d":9},"security_summary":{"status":"ok","scanner":"osv@v1","low_count":0,"high_count":0,"last_scan_at":"2026-07-15T10:48:51.518Z","medium_count":0,"scan_profile":"deps","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-17T06:00:06.866Z"},"languages":{"value":["python"],"source":"github.language+pyproject.toml","observed_at":"2026-09-17T06:00:06.866Z"},"license_spdx":{"value":"AGPL-3.0","source":"github.license","observed_at":"2026-09-17T06:00:06.866Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":{"notes":["The project requires patience and thorough reading due to its high usage threshold"],"min_ram_gb":null},"constraints":{"min_ram_gb":null},"when_to_use":["- When you need to work with multiple Large Language Models using the ComfyUI interface","- If you are focusing on frameworks that seamlessly integrate a variety of LLMs through a single framework like MCP server or Omost","- When handling both local and cloud-based LLM/VLM models, including those in distributed format and GGUF"],"when_not_to_use":["- Avoid if your primary environment is not Windows, as some portable packages are exclusively for this OS","- Not recommended if you require specific features or support that is exclusive to a particular competitor's framework"],"source":"enrich:decision_facts","observed_at":"2026-07-17T07:20:28.830Z"},"constraint_facets":{"min_ram_gb":null},"decision_summary":[{"label":"Requirements","value":"The project requires patience and thorough reading due to its high usage threshold"},{"label":"Adopt for","value":"ComfyUI_LLM_party: A Python-based agent framework adapted for LLMs like Qwen, GLM, Gemini, and local models including llama-3.3, Janus-Pro."},{"label":"License detail","value":"AGPL-3.0"}]}}