{"data":{"slug":"xlang-ai-ds-1000","name":"DS-1000","tagline":"[ICML 2023] Data and code release for the paper \"DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation\".","github_url":"https://github.com/xlang-ai/DS-1000","owner":"xlang-ai","repo":"DS-1000","owner_avatar_url":"https://avatars.githubusercontent.com/u/128829376?v=4","primary_language":"Python","stars":273,"forks":31,"topics":["benchmark","code-generation","data-science","large-language-models","semantic-parsing"],"archived":false,"github_pushed_at":"2024-10-30T17:43:46+00:00","maintenance_label":"Dormant","url":"https://www.graphcanon.com/tools/xlang-ai-ds-1000","markdown_url":"https://www.graphcanon.com/tools/xlang-ai-ds-1000.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/xlang-ai-ds-1000","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=xlang-ai-ds-1000","description":"[ICML 2023] Data and code release for the paper \"DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation\".","homepage_url":"https://ds1000-code-gen.github.io","license":"CC-BY-SA-4.0","open_issues":2,"watchers":7,"ai_summary":null,"readme_excerpt":"# the test code also needs: pip install datasets tqdm\npython test_ds1000.py\n```\n\nExpected output:\n\n``` Codex002\n           count   mean\nlib                    \nMatplotlib   155  0.548\nNumpy        220  0.432\nPandas       291  0.265\nPytorch       68  0.397\nScipy        106  0.349\nSklearn      115  0.435\nTensorflow    45  0.378\n\nDS-1000 overall\nmean      0.388\n```\n\nSee also results on new models in the [results](./results) folder.\n\nThe test script executes generated code, so your own sandbox is strongly encouraged, but the reference code and provided solutions seems safe to run.\nYour sandbox has to allow some file operations (e.g. saving plot in matplotlib) and os operations (e.g. tensorflow, sklearn)","github_created_at":"2022-11-15T11:49:27+00:00","created_at":"2026-07-11T23:46:29.06028+00:00","updated_at":"2026-07-11T23:46:41.952761+00:00","categories":[{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"},{"slug":"evaluation-observability","name":"Evaluation & Observability","url":"https://www.graphcanon.com/categories/evaluation-observability","markdown_url":"https://www.graphcanon.com/categories/evaluation-observability.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/evaluation-observability"}],"tags":[{"slug":"data-science","name":"data-science"},{"slug":"benchmark","name":"benchmark"},{"slug":"python","name":"python"},{"slug":"large-language-models","name":"large-language-models"},{"slug":"semantic-parsing","name":"semantic-parsing"},{"slug":"code-generation","name":"code-generation"}],"trust":{"provenance":{"is_fork":false,"github_id":566303764,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-11T23:46:30.177Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":619,"last_release_at":null},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T23:46:30.654Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-11T23:46:29.933Z"},"languages":{"value":["python"],"source":"github.language","observed_at":"2026-07-11T23:46:29.933Z"},"license_spdx":{"value":"CC-BY-SA-4.0","source":"github.license","observed_at":"2026-07-11T23:46:29.933Z"}}}}