Adaline Gateway

The only fully local, production-grade Super SDK that provides a simple, unified, and powerful interface for calling more than 300+ LLMs.

• Production-ready and trusted by enterprises
• Fully local and NOT a proxy - deploy it anywhere
• Built-in batching, retries, caching, callbacks, and OpenTelemetry support
• Extensible with custom plugins for caching, logging, HTTP clients, and more - use it like building blocks to integrate with your infrastructure
• Supports plug-and-play providers - run fully custom providers while leveraging all the benefits of Adaline Gateway

Features

• 🔧 Strongly typed in TypeScript
• 📦 Isomorphic - works everywhere
• 🔒 100% local, private, and NOT a proxy
• 🛠️ Tool calling support across all compatible LLMs
• 📊 Batching for all requests with custom queue support
• 🔄 Automatic retries with exponential backoff
• ⏳ Caching with custom cache plug-in support
• 📞 Callbacks for comprehensive instrumentation and hooks
• 🔍 OpenTelemetry integration for existing infrastructure
• 🔌 Plug-and-play custom providers for local and custom models

Providers

Installation

Core packages

npm install @adaline/gateway @adaline/types

Provider packages

Dependencies for providers are optional. You can install them as needed. For example:

npm install @adaline/openai @adaline/anthropic @adaline/google @adaline/open-router @adaline/bedrock

Quickstart

Chat

This example demonstrates how to invoke an LLM with a simple text prompt in both non-streaming and streaming modes.

import { Gateway } from "@adaline/gateway";
import { OpenAI } from "@adaline/openai";
import { Config, ConfigType, MessageType } from "@adaline/types";

const OPENAI_API_KEY = "your-api-key"; // Replace with your OpenAI API key

const gateway = new Gateway();
const openai = new OpenAI();

const gpt4o = openai.chatModel({
  modelName: "gpt-4o",
  apiKey: OPENAI_API_KEY,
});

const config: ConfigType = Config().parse({
  temperature: 0.7,
  maxTokens: 300,
});

const messages: MessageType[] = [
  {
    role: "system",
    content: [{
      modality: "text",
      value: "You are a helpful assistant. You are extremely concise.",
    }],
  },
  {
    role: "user",
    content: [{
      modality: "text",
      value: `What is ${Math.floor(Math.random() * 100) + 1} + ${Math.floor(Math.random() * 100) + 1}?`,
    }],
  },
];

// * Complete chat
async function runCompleteChat() {
  const completeChat = await gateway.completeChat({
    model: gpt4o,
    config,
    messages,
    tools: [],
  });
  console.log(completeChat.provider.request); // HTTP Request sent to Provider
  console.log(completeChat.provider.response); // HTTP Response from Provider
  console.log(completeChat.cached); // Whether the response was cached
  console.log(completeChat.latencyInMs); // Latency in milliseconds
  console.log(completeChat.request); // Request in Gateway types
  console.log(completeChat.response); // Response in Gateway types, E.g.:
  // {
  //   "messages": [
  //     {
  //       "role": "assistant",
  //       "c