{"data":{"slug":"staghado-vit-cpp","name":"vit.cpp","tagline":"Inference Vision Transformer (ViT) in plain C/C++ with ggml","github_url":"https://github.com/staghado/vit.cpp","owner":"staghado","repo":"vit.cpp","owner_avatar_url":"https://avatars.githubusercontent.com/u/84044788?v=4","primary_language":"C++","stars":318,"forks":28,"topics":["ai","c","computer-vision","cpp","cpu","edge-computing","ggml","image-classification","llamacpp","vision-transformer","whisper-cpp"],"archived":false,"github_pushed_at":"2024-04-11T07:35:06+00:00","maintenance_label":"Dormant","url":"https://www.graphcanon.com/tools/staghado-vit-cpp","markdown_url":"https://www.graphcanon.com/tools/staghado-vit-cpp.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/staghado-vit-cpp","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=staghado-vit-cpp","description":"Inference Vision Transformer (ViT) in plain C/C++ with ggml","homepage_url":null,"license":"MIT","open_issues":9,"watchers":10,"ai_summary":null,"readme_excerpt":"# vit.cpp\n\nInference Vision Transformer (ViT) in plain C/C++ using ggml without any extra dependencies\n\n## Description\n\n\nThis project presents a standalone implementation of the well known Vision Transformer (ViT) model family, used in a broad spectrum of applications and SOTA models like Large Multimodal Models(LMM). The primary goal is to develop a C/C++ inference engine tailored for ViT models, utilizing [ggml](https://github.com/ggerganov/ggml) to enhance performance, particularly on edge devices. Designed to be both lightweight and self-contained, this implementation can be run across diverse platforms.\n\n<details>\n<summary>Table of Contents</summary>\n\n1. [Description](#Description)\n2. [Features](#features)\n3. [Vision Transformer Architecture](#vision-transformer-architecture)\n4. [Quick Example](#quick-example)\n5. [Convert PyTorch to GGUF](#convert-pytorch-to-gguf)\n6. [Build](#build)\n   - [Simple Build](#simple-build)\n   - [Per Device Optimizations](#per-device-optimizations)\n   - [OpenMP](#using-openmp)\n7. [Run](#run)\n8. [Benchmark against PyTorch](#benchmark-against-pytorch)\n   - [ViT Inference](#vit-inference)\n   - [Benchmark on Your Machine](#benchmark-on-your-machine)\n9. [Quantization](#quantization)\n10. [To-Do List](#to-do-list)\n</details>\n\n\n## Features\n\n\n- Dependency-free and lightweight inference thanks to [ggml](https://github.com/ggerganov/ggml).\n- 4-bit, 5-bit and 8-bit quantization support.\n- Support for timm ViTs with different variants out of the box.\n\nAn important aspect of using `vit.cpp` is that it has short startup times compared to common DL frameworks, which makes it suitable for serverless deployments where the cold start is an issue.\n\n## Vision Transformer architecture\n\n\nThe implemented architecture is based on the original Vision Transformer from:\n  - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://arxiv.org/abs/2010.11929)\n\n<p align=\"center\">\n  <img src=\"assets/image.png\" alt=\"Vision Transformer overview\" width=\"60%\" height=\"auto\">\n</p>\n<p align=\"center\">\n  ViT architecture. Taken from the <a href=\"https://arxiv.org/abs/2010.11929\">original paper</a>.\n</p>\n\n## Quick example\n\n\n<details>\n  <p align=\"center\">\n    <img src=\"assets/magpie.jpeg\" alt=\"example input\" width=\"50%\" height=\"auto\">\n  </p>\n\n  <summary>See output</summary>\n  <pre>\n  $ ./bin/vit -t 4 -m ../ggml-model-f16.gguf -i ../assets/magpie.jpeg -k 5\n  main: seed = 1701176263\n  main: n_threads = 4 / 8\n  vit_model_load: loading model from &apos;../ggml-model-f16.gguf&apos; - please wait\n  vit_model_load: hidden_size            = 192\n  vit_model_load: num_hidden_layers      = 12\n  vit_model_load: num_attention_heads    = 3\n  vit_model_load: patch_size             = 16\n  vit_model_load: img_size               = 224\n  vit_model_load: num_classes            = 1000\n  vit_model_load: ftype                  = 1\n  vit_model_load: qntvr                  = 0\n  operator(): ggml ctx size =  11.13 MB\n  vit_model_load: ................... done\n  vit_model_load: model size =    11.04 MB / num tensors = 152\n  main: loaded image &apos;../assets/magpie.jpeg&apos; (500 x 470)\n  vit_image_preprocess: scale = 2.232143\n  processed, out dims : (224 x 224)\n\n\n  &gt; magpie : 0.87\n  &gt; goose : 0.02\n  &gt; toucan : 0.01\n  &gt; drake : 0.01\n  &gt; king penguin, Aptenodytes patagonica : 0.01\n\n\n  main:    model load time =    17.92 ms\n  main:    processing time =   146.96 ms\n  main:    total time      =   164.88 ms\n  </pre>\n</details>\n\n\n## Convert PyTorch to GGUF\n\n\n    # clone the repo recursively\n    git clone --recurse-submodules https://github.com/staghado/vit.cpp.git\n\n    cd vit.cpp\n\n    # install torch and timm\n    pip install torch timm\n\n    # list available models if needed; note that not all models are supported\n    python convert-pth-to-ggml.py --list\n\n    # convert the weights to gguf : vit tiny with patch size of 16 and an image \n    # size of 384 pre-trained on ImageNet21k and fine-tuned on ImageNet1k\n    python convert-pth","github_created_at":"2023-11-02T14:38:02+00:00","created_at":"2026-07-11T12:31:31.225946+00:00","updated_at":"2026-07-11T12:31:40.352772+00:00","categories":[{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"},{"slug":"speech-audio","name":"Speech & Audio","url":"https://www.graphcanon.com/categories/speech-audio","markdown_url":"https://www.graphcanon.com/categories/speech-audio.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/speech-audio"},{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"}],"tags":[{"slug":"ggml","name":"ggml"},{"slug":"ai","name":"ai"},{"slug":"cpp","name":"cpp"},{"slug":"image-classification","name":"image-classification"},{"slug":"c","name":"c"},{"slug":"cpu","name":"cpu"},{"slug":"edge-computing","name":"edge-computing"},{"slug":"computer-vision","name":"computer-vision"}],"trust":{"provenance":{"is_fork":false,"github_id":713446585,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-11T12:31:33.584Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":821,"last_release_at":"2023-12-11T13:53:16Z"},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T12:31:34.561Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-11T12:31:34.077Z"},"languages":{"value":["c++"],"source":"github.language","observed_at":"2026-07-11T12:31:34.077Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-07-11T12:31:34.077Z"}}}}