comfyui_LLM_party

Quick Start

0. If you have never used ComfyUI and encounter some dependency issues while installing the LLM party in ComfyUI, please click here to download the Windows portable package that includes the LLM party. Please note that this portable package contains only the party and manager plugins, and is exclusively compatible with the Windows operating system.(If you need to install LLM party into an existing comfyui, this step can be skipped.)
1. Drag the following workflows into your comfyui, then use comfyui-Manager to install the missing nodes.

• Use API to call LLM: start_with_LLM_api
• Using aisuite to call LLM: start_with_aisuite
• Manage local LLM with ollama: start_with_Ollama
• Use local LLM in distributed format: start_with_LLM_local
• Use local LLM in GGUF format: start_with_LLM_GGUF
• Use local VLM in distributed format: start_with_VLM_local (Currently, support is extended for Llama-3.2-Vision/Qwen/Qwen2.5-VL/deepseek-ai/Janus-Pro.)
• Use local VLM in GGUF format: start_with_VLM_GGUF
• Utilize API calls to LLM for generating SD prompts and images: start_with_VLM_API_for_SD
• Employ ollama to call minicpm for generating SD prompts and images: start_with_ollama_minicpm_for_SD
• Utilize the local qwen-vl to generate SD prompts and images: start_with_qwen_vl_local_for_SD

2. If you are using API, fill in your base_url (it can be a relay API, make sure it ends with /v1/), for example: https://api.openai.com/v1/ and api_key in the API LLM loader node.
3. If you are using ollama, turn on the is_ollama option in the API LLM loader node, no need to fill in base_url and api_key.
4. If you are using a local model, fill in your model path in the local model loader node, for example: E:\model\Llama-3.2-1B-Instruct. You can also fill in the Huggingface model repo id in the local model loader node, for example: lllyasviel/omost-llama-3-8b-4bits.
5. Due to the high usage threshold of this project, even if you choose the quick start, I hope you can patiently read through the project homepage.

Environment Deployment

1. Navigate to the comfyui_LLM_party project folder.
2. Enter pip install -r requirements.txt in the terminal to deploy the third-party libraries required by the project into the comfyui environment. Please ensure you are installing within the comfyui environment and pay attention to any pip errors in the terminal.
3. If you are using the comfyui launcher, you need to enter path_in_launcher_configuration\python_embeded\python.exe -m pip install -r requirements.txt in the terminal to install. The python_embeded folder is usually at the same level as your ComfyUI folder.
4. If you have some environment configuration problems, you can try to use the dependencies in requirements_fixed.txt.

Next Steps Plan:

1. More model adaptations;
2. More ways to build agents;
3. More automation features;
4. More knowledge base management features;
5. More tools, more personas.