vearch
vearch/vearch
Distributed vector search for AI-native applications
Overview
Vearch is a cloud-native distributed vector database designed for efficient similarity search of embedding vectors in AI applications, offering hybrid search capabilities, high performance, and scalability.
Categories
Tags
Similar tools
transformers
huggingface/transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models
langflow
langflow-ai/langflow
Langflow is a powerful platform for building and deploying AI-powered agents and workflows.
firecrawl
firecrawl/firecrawl
The API to search, scrape, and interact with the web at scale.
PaddleOCR
PaddlePaddle/PaddleOCR
PaddleOCR: A powerful OCR toolkit for transforming PDFs/images into structured data
graphify
Graphify-Labs/graphify
AI coding assistant skill that transforms various file types into a queryable knowledge graph
anything-llm
Mintplex-Labs/anything-llm
Stop renting your intelligence. Own it with AnythingLLM.
Install
go get github.com/vearch/vearchREADME
Overview
Vearch is a cloud-native distributed vector database for efficient similarity search of embedding vectors in your AI applications.
Key features
-
Hybrid search: Both vector search and scalar filtering.
-
Performance: Fast vector retrieval - search from millions of objects in milliseconds.
-
Scalability & Reliability: Replication and elastic scaling out.
Document
Restful APIs
OpenAPIs
SDK
| SDK | Description |
|---|---|
| Python SDK | Python client for Vearch |
| Go SDK | Go client for Vearch |
| Java SDK | Java client for Vearch |
| Rust SDK | Rust client for Vearch |
Usage Cases
Use Vearch as a Memory Backend
Vearch integrates with popular AI frameworks:
| Framework | Integration |
|---|---|
| Langchain | Use Vearch as vector store in Langchain |
| LlamaIndex | Integrate with LlamaIndex for knowledge bases |
| Langchaingo | Go implementation of Langchain with Vearch support |
| LangChain4j | Java implementation with Vearch integration |
Real world Demos
- VisualSearch: Vearch can be leveraged to build a complete visual search system to index billions of images. The image retrieval plugin for object detection and feature extraction is also required.
Quick start
# Via Helm Repository
$ helm repo add vearch https://vearch.github.io/vearch-helm
$ helm repo update && helm install my-release vearch/vearch
# Or from Local Charts
$ git clone https://github.com/vearch/vearch-helm.git && cd vearch-helm
$ helm install my-release ./charts -f ./charts/values.yaml
Docker Compose Deployment
# Standalone Mode
$ cd cloud && cp ../config/config.toml .
$ docker-compose --profile standalone up -d
# Cluster Mode
$ cd cloud && cp ../config/config_cluster.toml .
$ docker-compose --profile cluster up -d
Other Deployment Methods
- DeployByDocker: Deploy Vearch by Docker
- SourceCompileDeployment: Compile Vearch from source code
Components
Vearch Architecture
Master: Responsible for schema management, cluster-level metadata, and resource coordination.
Router: Provides RESTful API: upsert, delete, search and query; request routing, and result merging.
PartitionServer (PS): Hosts document partitions with raft-based replication. Gamma is the core vector search engine implemented based on faiss. It provides the ability of storing, indexing and retrieving the vectors and scalars.
Technical Reference
Academic Citation
When using Vearch in academic or research projects, please cite our paper:
@misc{li2019design,
title={The Design and Implementation of a Real Time Visual Search System on JD E-commerce Platform},
author={Jie Li and Haifeng Liu and Chuanghua Gui and Jianyu Chen and Zhenyun Ni and Ning Wang},
year={2019},
eprint={1908.07389},
archivePrefix=