Home/Developer Tools/LLM-As-Chatbot

LLM-As-Chatbot

deep-diver/LLM-As-Chatbot

LLM as a Chatbot Service

GitHub

GraphCanon updated 2w · GitHub synced 2w

3.3k stars375 forksLast push 2y Python Apache-2.0

Decision brief

LLM-As-Chatbot enables integration of instruction-following fine-tuned LLM models into chatbots via Gradio or Discord interfaces, including an internet search feature with the Serper API.

Good fit when

When you need to test and deploy a variety of open-source LLMS as chatbots supported by both Gradio applications and Discord bots.
If you have access to Jarvislabs.ai's cloud GPU VMs where model weights are pre-downloaded, allowing for quick setup without extensive resource consumption.

Avoid when

If your environment only supports Python versions prior to 3.9 or uses a Gradio version below 3.32.0.
When you do not have access to the Serper API key required for enabling internet search capabilities.

Observed Jul 12, 2026 · Source: enrich:decision_facts

LLM-As-Chatbot alternatives Trust report

Verify the decision

Maintenance and security

Full trust report

Maintenance: Dormant (985d since push); As of 2w
Provenance: Not a fork · Personal account; As of 2w
Security (OSV): 9 low (9 low); As of 1mo

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Install

pip install LLM-As-Chatbot

PyPI

Similar tools

Same-category neighbours. No typed graph edges are catalogued for this tool yet.

LibreChat

Enhanced ChatGPT Clone with extensive features and integrations for self-hosting

41k★

ChuanhuChatGPT

GUI for ChatGPT and LLMs with features like agents and file-based QA.

15k★

litgpt

High-performance LLMs with recipes for pretraining, finetuning and deployment

14k★

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

This repository allows the integration of instruction-following fine-tuned LLM models into chatbot services, supporting both Gradio application and Discord bot interfaces with options to enable internet search capabilities.

Capability facts

Languages: python
Source: github.language · Aug 2, 2026

Compatibility

Sourced claims from the README excerpt - not unsourced marketing copy.

Python runtimePython

Source: README excerpt (regex_v1, Aug 2, 2026)

Note that the code only works `Python >= 3.9` and `gradio >= 3.32.0`

Source link

README

UPDATE

Internet search support: you can enable internet search capability in Gradio application and Discord bot. For gradio, there is a internet mode option in the control panel. For discord, you need to specify --internet option in your prompt. For both cases, you need a Serper API Key which you can get one from serper.dev. By signing up, you will get free 2,500 free google searches which is pretty much sufficient for a long-term test.
Discord Bot support: you can serve any model from the model zoo as Discord Bot. Find how to do this in the instruction section below.

💬🚀 LLM as a Chatbot Service

The purpose of this repository is to let people to use lots of open sourced instruction-following fine-tuned LLM models as a Chatbot service. Because different models behave differently, and different models require differently formmated prompts, I made a very simple library Ping Pong for model agnostic conversation and context managements.

Also, I made GradioChat UI that has a similar shape to HuggingChat but entirely built in Gradio. Those two projects are fully integrated to power this project.

Easiest way to try out ( ✅ Gradio, 🚧 Discord Bot )

Jarvislabs.ai

This project has become the one of the default framework at jarvislabs.ai. Jarvislabs.ai is one of the cloud GPU VM provider with the cheapest GPU prices. Furthermore, all the weights of the supported popular open source LLMs are pre-downloaded. You don't need to waste of your money and time to wait until download hundreds of GBs to try out a collection of LLMs. In less than 10 minutes, you can try out any model.

for further instruction how to run Gradio application, please follow the official documentation on the llmchat framework.

dstack

dstack is an open-source tool that allows to run LLM-based apps in a a cloud of your choice via single command. dstack supports AWS, GCP, Azure, Lambda Cloud, etc.

Use the gradio.dstack.yml and discord.dstack.yml configurations to run the Gradio app and Discord bot via dstack.

for more details on how to run this repo with dstack, read the official documentation by dstack.

Instructions

Standalone Gradio app

Prerequisites

Note that the code only works Python >= 3.9 and gradio >= 3.32.0
```
$ conda create -n llm-serve python=3.9
$ conda activate llm-serve
```

Install dependencies.

$ cd LLM-As-Chatbot
$ pip install -r requirements.txt

Run Gradio application

There is no required parameter to run the Gradio application. However, there are some small details worth being noted. When --local-files-only is set, application won't try to look up the Hugging Face Hub(remote). Instead, it will only use the files already downloaded and cached.

Hugging Face libraries stores downloaded contents under ~/.cache by default, and this application assumes so. However, if you downloaded weights in different location for some reasons, you can set HF_HOME environment variable. Find more about the environment variables here

In order to leverage internet search capability, you need Serper API Key. You can set it manually in the control panel or in CLI. When specifying the Serper API Key in CLI, it will be injected into the corresponding UI control. If you don't have it yet, please get one from serper.dev. By signing up, you will get free 2,500 free google searches which is pretty much sufficient for a long-term test.
```
$ python app.py --root-path "" \
                --local-files-only \
```

For agents

This page has a .md twin and JSON over the API.

/tools/deep-diver-llm-as-chatbot.md GET /api/graphcanon/tools/deep-diver-llm-as-chatbot GET /api/graphcanon/graph?tool=deep-diver-llm-as-chatbot