Home/Data & Retrieval/DS-1000

DS-1000

xlang-ai/DS-1000

Benchmark and code for evaluating large language models on data science tasks

GitHub Website

GraphCanon updated Oct 8, 2026 · GitHub synced Oct 8, 2026

40views this month

278 stars31 forksLast push Oct 30, 2024 Python CC-BY-SA-4.0

Decision brief

The DS-1000 benchmark evaluates the code generation capabilities of large language models for data science tasks across Python libraries like Matplotlib, Numpy, Pandas, etc.

Good fit when

When you want to assess how well a large language model can generate reliable and accurate code for data science projects involving popular Python libraries.

Avoid when

Avoid using DS-1000 if your project does not involve data science or if the models do not generate code in Python.
It is unsuitable for evaluating text generation abilities unrelated to coding, such as natural language processing tasks.

Observed Jul 17, 2026 · Source: enrich:decision_facts

DS-1000 alternatives Trust report

Verify the decision

Maintenance and security

Full trust report

Maintenance: Dormant (707d since push); As of Oct 8, 2026
Provenance: Not a fork · Organization account; As of Oct 8, 2026
Security (OSV): No lockfile; As of Jul 11, 2026

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Install

pip install DS-1000

PyPI

Similar tools

Same-category neighbours. No typed graph edges are catalogued for this tool yet.

datasets

Largest hub of ready-to-use datasets for AI models

22k★

pua

A skill package for enhancing the functionality of AI agents within development environments

19k★

SWE-bench

Benchmark for assessing language models' capability to resolve real-world Github issues

6.0k★

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

Provides benchmark and code generation assessment tools for large-language-models focused on data science applications. Involves testing the reliability and accuracy of generated code across various libraries including Matplotlib, Numpy, Pandas, Pytorch, Scipy, Sklearn, Tensorflow.

Capability facts

Languages: python
Source: github.language · Oct 8, 2026

Compatibility

Sourced claims from the README excerpt - not unsourced marketing copy.

Python runtimePython

Source: README excerpt (regex_v1, Oct 8, 2026)

python test_ds1000.py

Source link

README

the test code also needs: pip install datasets tqdm python test ds1000.py Codex002 count mean lib Matplotlib 155 0.548 Numpy 220 0.432 Pandas 291 0.265 Pytorch 68 0.397 Scipy 106 0.349 Sklearn 115 0.435 Tensorflow 45 0.378 DS 1000 overall mean 0.388 ``` See also results on new mo...

For agents

This page has a .md twin and JSON over the API.

/tools/xlang-ai-ds-1000.md GET /api/graphcanon/tools/xlang-ai-ds-1000 GET /api/graphcanon/graph?tool=xlang-ai-ds-1000