{"data":{"slug":"liltom-eth-llama2-webui","name":"llama2-webui","tagline":"Run Llama 2 locally with gradio UI on GPU or CPU","github_url":"https://github.com/liltom-eth/llama2-webui","owner":"liltom-eth","repo":"llama2-webui","owner_avatar_url":"https://avatars.githubusercontent.com/u/11456256?v=4","primary_language":"Jupyter Notebook","stars":1937,"forks":199,"topics":["llama-2","llama2","llm","llm-inference"],"archived":false,"github_pushed_at":"2024-03-22T09:50:24+00:00","maintenance_label":"Dormant","stars_delta_30d":0,"url":"https://www.graphcanon.com/tools/liltom-eth-llama2-webui","markdown_url":"https://www.graphcanon.com/tools/liltom-eth-llama2-webui.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/liltom-eth-llama2-webui","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=liltom-eth-llama2-webui","description":"Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Use `llama2-wrapper` as your local llama2 backend for Generative Agents/Apps.  ","homepage_url":null,"license":"MIT","open_issues":26,"watchers":21,"ai_summary":"This repository provides an interface to run Llama 2 models using the Gradio framework for local model inference and deployment","readme_excerpt":"### Install Issues:\n`bitsandbytes >= 0.39` may not work on older NVIDIA GPUs. In that case, to use `LOAD_IN_8BIT`, you may have to downgrade like this:\n\n-  `pip install bitsandbytes==0.38.1`\n\n`bitsandbytes` also need a special install for Windows:\n\n```\npip uninstall bitsandbytes\npip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.0-py3-none-win_amd64.whl\n```\n\n---\n\n# pip install llama2-wrapper\nfrom llama2_wrapper import LLAMA2_WRAPPER, get_prompt \nllama2_wrapper = LLAMA2_WRAPPER()\n\n---\n\n## License\n\nMIT - see [MIT License](LICENSE)\n\nThis project enables users to adapt it freely for proprietary purposes without any restrictions.","github_created_at":"2023-07-20T02:03:38+00:00","created_at":"2026-07-11T11:43:20.056423+00:00","updated_at":"2026-08-25T00:00:48.910454+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"},{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"}],"tags":[{"slug":"gradio","name":"gradio"},{"slug":"llama-2","name":"llama-2"},{"slug":"local-inference","name":"local-inference"}],"trust":{"provenance":{"is_fork":false,"github_id":668513898,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-08-25T00:00:48.152Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":885,"last_release_at":"2023-10-04T04:58:19Z","stars_delta_30d":0,"open_issues_delta_30d":0},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T11:43:21.310Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-08-25T00:00:48.625Z"},"languages":{"value":["jupyter notebook","python"],"source":"github.language+pyproject.toml","observed_at":"2026-08-25T00:00:48.625Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-08-25T00:00:48.625Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":{"notes":["- Ensure you have access to Llama 2 models before proceeding.","- Requires a Python environment capable of running Jupyter Notebooks along with the specified dependencies such as Gradio."]},"constraints":null,"when_to_use":["- When you want to run Llama 2 models locally with minimal setup across various operating systems like Linux, Windows, and Mac.","- If your preference is for a tool that supports both CPU and GPU inference, offering flexibility in resource use."],"when_not_to_use":["- Avoid if you are looking for broader support beyond Llama 2 models; this tool is specifically tailored to work with the Llama 2 series.","- Not recommended if your project strictly requires a web-based deployment without a local server component, as it emphasizes local model inference."],"source":"enrich:decision_facts","observed_at":"2026-07-14T20:02:48.319Z"},"constraint_facets":null,"decision_summary":[{"label":"Requirements","value":"- Ensure you have access to Llama 2 models before proceeding.; - Requires a Python environment capable of running Jupyter Notebooks along with the specified dependencies such as Gradio."},{"label":"Adopt for","value":"llama2-webui - Run Llama 2 models locally via Gradio UI on GPU or CPU."},{"label":"License detail","value":"The MIT License grants permissive software rights making llama2-webui suitable for both proprietary and open-source projects."}]}}