“memory-efficient”

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

exllama

More memory-efficient rewrite of HF transformers for Llama with quantized weights

magnitude

A fast, efficient universal vector embedding utility package.

mxnet

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning Framework

OpenRLHF

Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray