GraphCanon
Home/Model Training/deeplake

deeplake

activeloopai/deeplake

Deeplake: AI Data Runtime for Agents

GitHub Website

GraphCanon updated today · GitHub synced today

9.2k
Stars
721
Forks
69
Open issues
95
Watchers
1mo
Last push
C++ Apache-2.0Created Aug 9, 2019

Compare deeplake

deeplake vs milvus

Relationship graph

Overview

Deeplake is a scalable data management system optimized for deep learning applications, offering serverless PostgreSQL integration, multimodal support, and vector search capabilities, making it suitable for large language model-based projects.

Capability facts

Languages
c++
Source: github.language · Jul 8, 2026

Categories

Model TrainingData & RetrievalVector Databases

Tags

filesystemlargelanguage-modelsdeep-learningdatalakellmaiagentic-ragagent

README


Deep Lake: Database for AI

PyPI version PyPI version

Docs • Get Started • API Reference • LangChain & VectorDBs Course • Blog • Whitepaper • Slack • Twitter

What is Deep Lake?

Deep Lake is a Database for AI powered by a storage format optimized for deep-learning applications. Deep Lake can be used for:

  1. Storing and searching data plus vectors while building LLM applications
  2. Managing datasets while training deep learning models

Deep Lake simplifies the deployment of enterprise-grade LLM-based products by offering storage for all data types (embeddings, audio, text, videos, images, dicom, pdfs, annotations, and more), querying and vector search, data streaming while training models at scale, data versioning and lineage, and integrations with popular tools such as LangChain, LlamaIndex, Weights & Biases, and many more. Deep Lake works with data of any size, it is serverless, and it enables you to store all of your data in your own cloud and in one place. Deep Lake is used by Intel, Bayer Radiology, Matterport, ZERO Systems, Red Cross, Yale, & Oxford.

Deep Lake includes the following features:

Multi-Cloud Support (S3, GCP, Azure) Use one API to upload, download, and stream datasets to/from S3, Azure, GCP, Activeloop cloud, local storage, or in-memory storage. Compatible with any S3-compatible storage such as MinIO.
Native Compression with Lazy NumPy-like Indexing Store images, audio, and videos in their native compression. Slice, index, iterate, and interact with your data like a collection of NumPy arrays in your system's memory. Deep Lake lazily loads data only when needed, e.g., when training a model or running queries.
Dataloaders for Popular Deep Learning Frameworks Deep Lake comes with built-in dataloaders for Pytorch and TensorFlow. Train your model with a few lines of code - we even take care of dataset shuffling. :)
Integrations with Powerful Tools Deep Lake has integrations with Langchain and LLamaIndex as a vector store for LLM apps, Weights & Biases for data lineage during

Command menu

Search tools or jump to a page