GraphCanon
Home/Compare/ray-llm vs exllama

Comparison

ray-llm vs exllama

Verdict

Pick ray-llm if ray-LLM is an archived project for deploying large language models on Ray, with its functionalities now integrated into the Ray framework through the ray.serve.llm and ray.data.llm APIs; pick exllama if exLlama provides a memory-efficient implementation of the LLaMa model with support for quantized weights, primarily aimed at users with NVIDIA GPUs from the 30-series onwards.

Markdown twin · ray-llm alternatives · exllama alternatives

GraphCanon updated Sep 20, 2026

7views this month

ray-llm logo

ray-llm

ray-project/ray-llm

★ 1.3kpushed Mar 13, 2025
vs
exllama logo

exllama

turboderp/exllama

★ 2.9kpushed Sep 30, 2023

Trust & integrity

Signalray-llmexllama
Maintenance
Archived (554d since push)
As of Sep 18, 2026 · github_public_v1
Dormant (1072d since push)
As of Sep 7, 2026 · github_public_v1
Provenance
Not a fork · Organization account
As of Sep 18, 2026 · github_public_v1
Not a fork · Personal account
As of Sep 7, 2026 · github_public_v1
OSV dependency advisories
No lockfile (source not queried)
As of Sep 18, 2026 · osv@v1
Published findings
As of Jul 11, 2026 · osv@v1
deps.dev advisories
Not queried
deps.dev@v1
Not queried
deps.dev@v1
OpenSSF Scorecard
Not queried
openssf-scorecard@v1
Not queried
openssf-scorecard@v1
ray-llm trust reportexllama trust reportTrust methodology

Tagline

ray-llm
Archived repository for LLMs on Ray, now integrated into Ray.
exllama
Memory-efficient rewrite of HF transformers for Llama with quantized weights

Stars

ray-llm
1.3k
exllama
2.9k

Forks

ray-llm
90
exllama
220

Open issues

ray-llm
0
exllama
65

Language

ray-llm
-
exllama
Python

Adopt for

ray-llm
Ray-LLM is an archived project for deploying large language models on Ray, with its functionalities now integrated into the Ray framework through the ray.serve.llm and ray.data.llm APIs.
exllama
ExLlama provides a memory-efficient implementation of the LLaMa model with support for quantized weights, primarily aimed at users with NVIDIA GPUs from the 30-series onwards.

Persona

ray-llm
-
exllama
-

Runtime

ray-llm
-
exllama
-

License

ray-llm
License information is not provided.
exllama
MIT

Last pushed

ray-llm
Mar 13, 2025
exllama
Sep 30, 2023

Categories

ray-llm
Inference & Serving, Model Training
exllama
Inference & Serving, LLM Frameworks

Trust and health

Maintenance

ray-llm
Archived (8%)
exllama
Dormant (18%)

Days since push

ray-llm
554d
exllama
1072d

Archived on GitHub

ray-llm
Yes
exllama
No

Open issues (now)

ray-llm
0
exllama
65

Stars delta

ray-llm
0 (30d)
exllama
+6 (30d)

Owner type

ray-llm
Organization
exllama
User

OSV dependency advisories

ray-llm
No lockfile (source not queried)
exllama
Published findings

Full report

ray-llm
Trust report
exllama
Trust report

Choose ray-llm if…

  • Pricing: Pricing details are not available..
  • Requirements: Ensure you have the latest version of Ray installed to use the ray.serve.llm and ray.data.llm APIs..
  • Tags unique to ray-llm: llm, llm-serving, ray.
  • Also covers Model Training.
  • If you are working with Ray and need to deploy large language models, use the ray.serve.llm and ray.data.llm APIs, which are now integrated into the Ray framework.

When NOT to use ray-llm

  • Do not use the archived Ray-LLM repository for active development or deployment, as it is no longer maintained and its functionalities have been integrated into the Ray framework.
  • If you require the latest features and support for deploying large language models, avoid the archived repository and use the ray.serve.llm and ray.data.llm APIs directly.
GitHubDetails

Choose exllama if…

  • Tags unique to exllama: docker, llama model, memory-efficient, nvidia gpu.
  • Also covers LLM Frameworks.
  • exllama ships Docker support for self-hosted deployment.
  • - When deploying LLaMa models on NVIDIA GPUs from the 30-series or later that have strong FP16 support.

When NOT to use exllama

  • - If you are operating older GPUs such as Pascal series, which lack robust FP16 support; alternatives like AutoGPTQ might perform better.
  • - In scenarios that involve AMD GPU hardware (due to limited testing and optimization efforts).
GitHubDetails

Explore

ray-llm trust report →exllama trust report →Inference & Serving category →Model Training category →LLM Frameworks category →All comparisonsStack workflowsTrending tools

Sources

Every stat on this page traces to a dated GitHub sync, license file, enrichment field, or trust scan.

GitHub stars on cards: ray-llm 1.3k · exllama 2.9k (synced Sep 20, 2026).

Common questions

What is the difference between ray-llm and exllama?
ray-llm: Archived repository for LLMs on Ray, now integrated into Ray.. exllama: Memory-efficient rewrite of HF transformers for Llama with quantized weights. See the comparison table for live GitHub stats and shared categories.
When should I choose ray-llm over exllama?
Choose ray-llm over exllama when Pricing: Pricing details are not available.; Requirements: Ensure you have the latest version of Ray installed to use the ray.serve.llm and ray.data.llm APIs.; Tags unique to ray-llm: llm, llm-serving, ray; Also covers Model Training; If you are working with Ray and need to deploy large language models, use the ray.serve.llm and ray.data.llm APIs, which are now integrated into the Ray framework.
When should I choose exllama over ray-llm?
Choose exllama over ray-llm when Tags unique to exllama: docker, llama model, memory-efficient, nvidia gpu; Also covers LLM Frameworks; exllama ships Docker support for self-hosted deployment; - When deploying LLaMa models on NVIDIA GPUs from the 30-series or later that have strong FP16 support.
When should I avoid ray-llm?
Do not use the archived Ray-LLM repository for active development or deployment, as it is no longer maintained and its functionalities have been integrated into the Ray framework. If you require the latest features and support for deploying large language models, avoid the archived repository and use the ray.serve.llm and ray.data.llm APIs directly.
When should I avoid exllama?
- If you are operating older GPUs such as Pascal series, which lack robust FP16 support; alternatives like AutoGPTQ might perform better. - In scenarios that involve AMD GPU hardware (due to limited testing and optimization efforts).
Is ray-llm or exllama more popular on GitHub?
exllama has more GitHub stars (2,943 vs 1,261). Stars measure visibility, not whether either tool fits your constraints.
Are ray-llm and exllama open source?
Yes - both are open-source projects on GitHub.
Where can I find alternatives to ray-llm or exllama?
GraphCanon lists graph-backed alternatives at ray-llm alternatives and exllama alternatives (ray-llm markdown twin, exllama markdown twin), ranked by typed relationship edges rather than popularity votes.
Is there a machine-readable version of this comparison?
Yes. The markdown twin at this comparison mirrors this page for agents and LLM crawlers, with the same stats table and FAQ answers.
Which is better maintained, ray-llm or exllama?
ray-llm: Archived. exllama: Dormant. Compare maintenance labels, days since push, and release cadence in the trust section below - stars alone do not measure maintenance.
Where are the full trust reports for ray-llm and exllama?
GraphCanon publishes per-repo trust reports with dated maintenance, provenance, and scan summaries: ray-llm trust report; exllama trust report.

Was this helpful?

Anonymous feedback helps us improve pages and translations.