Note: we’ve temporarily stopped development on the open-source version of OpenPipe to integrate some proprietary third-party code. We hope to make the non-proprietary parts of the repository open again under an open core model once we have the bandwidth to do so!

OpenPipe

Open-source fine-tuning and model-hosting platform.

Demo - Running Locally - Docs

Use powerful but expensive LLMs to fine-tune smaller and cheaper models suited to your exact needs. Query your past requests and evaluate models against one another. Switch between OpenAI and fine-tuned models with one line of code.

Features

• Easy integration with OpenAI's SDK in both Python and TypeScript.
  • Python SDK
  • TypeScript SDK
• OpenAI-compatible chat completions endpoint.
• Fine-tune GPT 3.5, Mistral, and Llama 2 models. Host on-platform or download the weights.
  • Model output is OpenAI-compatible.
  • Switching from GPT 4 to a fine-tuned Mistral model only requires changing the model name.
• Query logs using powerful built-in filters.
• Import datasets in OpenAI-compatible JSONL files.
• Prune large chunks of duplicate text like system prompts.
• Compare output accuracy against base models like gpt-3.5-turbo.

Supported Base Models

• mistralai/Mixtral-8x7B-Instruct-v0.1
• OpenPipe/mistral-ft-optimized-1227
• meta-llama/Llama-3-8B
• meta-llama/Llama-3-70B
• gpt-3.5-turbo-0613
• gpt-3.5-turbo-1106
• gpt-3.5-turbo-0125

Documentation

• See docs

Running Locally

1. Install Postgresql.
2. Install NodeJS 20 (earlier versions will very likely work but aren't tested).
3. Install pnpm: npm i -g pnpm
4. Clone this repository: git clone https://github.com/openpipe/openpipe
5. Install the dependencies: cd openpipe && pnpm install
6. Create a .env file (cd app && cp .env.example .env) and enter your OPENAI_API_KEY.
7. If you just installed postgres and wish to use the default DATABASE_URL run the following commands:

psql postgres
CREATE ROLE postgres WITH LOGIN PASSWORD 'postgres';
ALTER ROLE postgres SUPERUSER;

8. Update DATABASE_URL if necessary to point to your Postgres instance an