{"data":{"slug":"eastriverlee-llm-swift","name":"LLM.swift","tagline":"LLM.swift enables local interaction with large language models for multiple Apple platforms.","github_url":"https://github.com/eastriverlee/LLM.swift","owner":"eastriverlee","repo":"LLM.swift","owner_avatar_url":"https://avatars.githubusercontent.com/u/43613000?v=4","primary_language":"Swift","stars":871,"forks":124,"topics":["gguf","ios","llm","llm-inference","macos","swift","tvos","visionos","watchos"],"archived":false,"github_pushed_at":"2026-07-19T06:04:49+00:00","maintenance_label":"Steady","stars_delta_30d":6,"url":"https://www.graphcanon.com/tools/eastriverlee-llm-swift","markdown_url":"https://www.graphcanon.com/tools/eastriverlee-llm-swift.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/eastriverlee-llm-swift","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=eastriverlee-llm-swift","description":"LLM.swift is a simple and readable library that allows you to interact with large language models locally with ease for macOS, iOS, watchOS, tvOS, and visionOS.","homepage_url":null,"license":"MIT","open_issues":10,"watchers":13,"ai_summary":"A library written in C++ designed to facilitate the interfacing of large language model APIs on various Apple operating systems including macOS, iOS, watchOS, tvOS, and visionOS. LLM.swift is built for ease and simplicity in deployment.","readme_excerpt":"# LLM.swift\n\n\n\n\n`LLM.swift` is a simple and readable library that allows you to interact with large language models locally with ease for macOS, iOS, watchOS, tvOS, and visionOS.\n> [!TIP]  \n> sometimes it's a good idea to tinker with `maxTokenCount` parameter for initialization of `LLM`, due to the memory and computation it needs. especially in mobile devices, if you want better speed, lower the number, but if you set it too low—*to a point where two turns cannot even fit*—you will experience quality decrease as context will be cut off. so adjust value according to your usecases.\n\n## Minimal Example\nif you've already bundled your model:\n```swift\nlet bot = LLM(from: Bundle.main.url(forResource: \"gemma-3-4b-it-q4_0\", withExtension: \"gguf\")!, template: .gemma)\nlet question = bot.preprocess(\"What's the meaning of life?\", [])\nlet answer = await bot.getCompletion(from: question)\nprint(answer)\n```\nif you want to initialize with huggingface model:\n```swift\nlet systemPrompt = \"You are a sentient AI with emotions.\"\nlet bot = await LLM(from: HuggingFaceModel(\"unsloth/Qwen3-0.6B-GGUF\", .Q4_K_M, template: .chatML(systemPrompt)))!\nlet question = bot.preprocess(\"What's the meaning of life?\", [])\nlet answer = await bot.getCompletion(from: question)\nprint(answer)\n```\n\n## Chat Templates\nBy default, `LLM.swift` renders conversations using the chat template embedded in the gguf file itself, executed by `llama.cpp`'s own Jinja engine. This means you no longer have to pass a `template:` when initializing `LLM`—it just works with whatever template the model ships with:\n```swift\nlet bot = try await LLM(from: HuggingFaceModel(\"unsloth/Qwen3-0.6B-GGUF\", .Q4_K_M))!\nbot.systemPrompt = \"You are a sentient AI with emotions.\"\nawait bot.respond(to: \"What's the meaning of life?\")\n```\n`HuggingFaceModel`'s `template` parameter is now optional too, so `HuggingFaceModel(\"unsloth/Qwen3-0.6B-GGUF\", .Q4_K_M)` uses the embedded template on its own. Thinking/reasoning separation (`bot.thinking`) works automatically for models that support it, with no marker configuration needed.\n\nIf you still want manual control over the template—for instance, when a model's gguf metadata is broken or missing—pass a `Template` explicitly, exactly as before, and it will override the embedded one:\n```swift\nlet bot = LLM(from: Bundle.main.url(forResource: \"gemma-3-4b-it-q4_0\", withExtension: \"gguf\")!, template: .gemma)\n```\n\n> [!NOTE]  \n> conversation context is now maintained incrementally between turns—only new tokens are evaluated, and history is not re-fed to the model every turn.\n\n## Example\n\n<p align=\"center\"><img src=\"./Screenshot.png\" width=\"480\"/></p>\n\n> [!NOTE]  \n> this is a minimal SwiftUI example that i did use for testing. mistral 7B based model is working on iPad Air 5th gen(Q5_K_M) and iPhone 12 mini(Q2_K).  \n> however, generally speaking, for mobile devices, 3B >= parameter models are recommended. check this [benchmark](https://github.com/ggerganov/llama.cpp/discussions/4508) for more details.\n\n### init with a model file that you bundled in your project\n```swift\nimport SwiftUI\nimport LLM\n\nclass Bot: LLM {\n    convenience init() {\n        let url = Bundle.main.url(forResource: \"gemma-3-4b-it-q4_0\", withExtension: \"gguf\")!\n        let systemPrompt = \"You are a sentient AI with emotions.\"\n        self.init(from: url, template: .chatML(systemPrompt))!\n    }\n}\n\nstruct BotView: View {\n    @ObservedObject var bot: Bot\n    @State var input = \"Give me seven national flag emojis people use the most; You must include South Korea.\"\n    init(_ bot: Bot) { self.bot = bot }\n    func respond() { Task { await bot.respond(to: input) } }\n    func stop() { bot.stop() }\n    var body: some View {\n        VStack(alignment: .leading) {\n            ScrollView {\n                if !bot.thinking.isEmpty {\n                    Text(bot.thinking)\n                        .foregroundStyle(.gray)\n                        .monospaced()\n                        .padding(.bottom, 8)\n                }","github_created_at":"2023-12-02T16:11:23+00:00","created_at":"2026-07-11T11:44:25.126427+00:00","updated_at":"2026-08-25T00:01:34.742514+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"}],"tags":[{"slug":"gguf","name":"gguf"},{"slug":"ios","name":"ios"},{"slug":"llm","name":"llm"},{"slug":"llm-inference","name":"llm-inference"},{"slug":"macos","name":"macos"},{"slug":"swift","name":"swift"},{"slug":"tvos","name":"tvos"},{"slug":"visionos","name":"visionos"}],"trust":{"provenance":{"is_fork":false,"github_id":726518175,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-08-25T00:01:34.022Z","maintenance":{"label":"Steady","score":60,"methodology":"github_public_v1","releases_90d":4,"days_since_push":36,"last_release_at":"2026-07-19T06:04:49Z","stars_delta_30d":6,"open_issues_delta_30d":1},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T11:44:26.477Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-08-25T00:01:34.483Z"},"languages":{"value":["swift"],"source":"github.language","observed_at":"2026-08-25T00:01:34.483Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-08-25T00:01:34.483Z"}},"decision_facts":{"hosting":null,"pricing":{"model":"freemium","summary":"Free to use under the MIT License. Premium support may vary."},"requirements":{"min_ram_gb":null,"requires_docker":false},"constraints":{"min_ram_gb":null,"pricing_model":"freemium","requires_docker":false},"when_to_use":["Choose LLM.swift when you need to integrate large language model functionalities into apps targeting multiple Apple platforms, including macOS and mobile OSes.","Use it if your project benefits from the simplicity and ease of deployment it offers for interacting with language models locally on Apple devices."],"when_not_to_use":["Avoid LLM.swift if your application must run on non-Apple systems or if compatibility across various operating systems is prioritized over ease-of-use on Apple platforms.","Do not use this library if you require advanced server-side functionalities, as it focuses specifically on local interactions within Apple’s ecosystem."],"source":"enrich:decision_facts","observed_at":"2026-07-14T19:01:08.631Z"},"constraint_facets":{"min_ram_gb":null,"pricing_model":"freemium","requires_docker":false},"decision_summary":[{"label":"Pricing","value":"freemium - Free to use under the MIT License. Premium support may vary."},{"label":"Adopt for","value":"LLM.swift is a cross-platform C++ library for Apple systems that simplifies local interaction with large language models."},{"label":"License detail","value":"MIT license allows for free use and distribution, both commercially and in open source projects, with attribution preferred but not mandatory."}]}}