LLM.swift

LLM.swift is a simple and readable library that allows you to interact with large language models locally with ease for macOS, iOS, watchOS, tvOS, and visionOS.

[!TIP]
sometimes it's a good idea to tinker with maxTokenCount parameter for initialization of LLM, due to the memory and computation it needs. especially in mobile devices, if you want better speed, lower the number, but if you set it too low—to a point where two turns cannot even fit—you will experience quality decrease as context will be cut off. so adjust value according to your usecases.

Minimal Example

if you've already bundled your model:

let bot = LLM(from: Bundle.main.url(forResource: "gemma-3-4b-it-q4_0", withExtension: "gguf")!, template: .gemma)
let question = bot.preprocess("What's the meaning of life?", [])
let answer = await bot.getCompletion(from: question)
print(answer)

if you want to initialize with huggingface model:

let systemPrompt = "You are a sentient AI with emotions."
let bot = await LLM(from: HuggingFaceModel("unsloth/Qwen3-0.6B-GGUF", .Q4_K_M, template: .chatML(systemPrompt)))!
let question = bot.preprocess("What's the meaning of life?", [])
let answer = await bot.getCompletion(from: question)
print(answer)

Chat Templates

By default, LLM.swift renders conversations using the chat template embedded in the gguf file itself, executed by llama.cpp's own Jinja engine. This means you no longer have to pass a template: when initializing LLM—it just works with whatever template the model ships with:

let bot = try await LLM(from: HuggingFaceModel("unsloth/Qwen3-0.6B-GGUF", .Q4_K_M))!
bot.systemPrompt = "You are a sentient AI with emotions."
await bot.respond(to: "What's the meaning of life?")

HuggingFaceModel's template parameter is now optional too, so HuggingFaceModel("unsloth/Qwen3-0.6B-GGUF", .Q4_K_M) uses the embedded template on its own. Thinking/reasoning separation (bot.thinking) works automatically for models that support it, with no marker configuration needed.

If you still want manual control over the template—for instance, when a model's gguf metadata is broken or missing—pass a Template explicitly, exactly as before, and it will override the embedded one:

let bot = LLM(from: Bundle.main.url(forResource: "gemma-3-4b-it-q4_0", withExtension: "gguf")!, template: .gemma)

[!NOTE]
conversation context is now maintained incrementally between turns—only new tokens are evaluated, and history is not re-fed to the model every turn.

Example

[!NOTE]
this is a minimal SwiftUI example that i did use for testing. mistral 7B based model is working on iPad Air 5th gen(Q5_K_M) and iPhone 12 mini(Q2_K).
however, generally speaking, for mobile devices, 3B >= parameter models are recommended. check this benchmark for more details.

init with a model file that you bundled in your project

import SwiftUI
import LLM

class Bot: LLM {
    convenience init() {
        let url = Bundle.main.url(forResource: "gemma-3-4b-it-q4_0", withExtension: "gguf")!
        let systemPrompt = "You are a sentient AI with emotions."
        self.init(from: url, template: .chatML(systemPrompt))!
    }
}

struct BotView: View {
    @ObservedObject var bot: Bot
    @State var input = "Give me seven national flag emojis people use the most; You must include South Korea."
    init(_ bot: Bot) { self.bot = bot }
    func respond() { Task { await bot.respond(to: input) } }
    func stop() { bot.stop() }
    var body: some View {
        VStack(alignment: .leading) {
            ScrollView {
                if !bot.thinking.isEmpty {
                    Text(bot.thinking)
                        .foregroundStyle(.gray)
                        .monospaced()
                        .padding(.bottom, 8)
                }