{"data":{"slug":"niansong1996-lever","name":"lever","tagline":"Supports learning to verify language-to-code generation with execution","github_url":"https://github.com/niansong1996/lever","owner":"niansong1996","repo":"lever","owner_avatar_url":"https://avatars.githubusercontent.com/u/10934810?v=4","primary_language":"Python","stars":90,"forks":8,"topics":[],"archived":false,"github_pushed_at":"2023-07-05T09:06:23+00:00","maintenance_label":"Dormant","url":"https://www.graphcanon.com/tools/niansong1996-lever","markdown_url":"https://www.graphcanon.com/tools/niansong1996-lever.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/niansong1996-lever","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=niansong1996-lever","description":"Code for paper \"LEVER: Learning to Verifiy Language-to-Code Generation with Execution\" (ICML'23)","homepage_url":"https://arxiv.org/abs/2302.08468","license":"MIT","open_issues":2,"watchers":4,"ai_summary":"Provides code for the paper LEVER from ICML'23 that focuses on verifying language-to-code generation through execution.","readme_excerpt":"### Installation\n(Recommended) Create a new conda environment\n```bash\nconda create -n lever python=3.8\nconda activate lever\n```\nInstall the dependencies\n```bash\npip install -r requirements.txt\n```\n> **NOTE**: all of the pipelines are only tested on Linux machines, you may need to build your own `tree-sitter` parsers if a different platform is used.","github_created_at":"2023-02-16T06:01:49+00:00","created_at":"2026-07-11T23:45:09.573884+00:00","updated_at":"2026-08-05T18:00:43.423406+00:00","categories":[{"slug":"evaluation-observability","name":"Evaluation & Observability","url":"https://www.graphcanon.com/categories/evaluation-observability","markdown_url":"https://www.graphcanon.com/categories/evaluation-observability.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/evaluation-observability"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"code-verification","name":"code verification"},{"slug":"execution-based-verification","name":"execution based verification"},{"slug":"language-to-code","name":"language-to-code"}],"trust":{"provenance":{"is_fork":false,"github_id":602401834,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-08-05T18:00:41.561Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":1127,"last_release_at":null},"security_summary":{"status":"ok","scanner":"osv@v1","low_count":0,"high_count":0,"last_scan_at":"2026-07-11T23:45:12.480Z","medium_count":0,"scan_profile":"deps","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-08-05T18:00:43.076Z"},"languages":{"value":["python"],"source":"github.language","observed_at":"2026-08-05T18:00:43.076Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-08-05T18:00:43.076Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":{"notes":["All of the pipelines have been tested on Linux machines only, requiring possibly custom `tree-sitter` parsers for other platforms."],"min_ram_gb":null,"requires_docker":false},"constraints":{"min_ram_gb":null,"requires_docker":false},"when_to_use":["When the development team needs to verify accuracy of generated code from language inputs based on execution, and has access to Linux machines to ensure seamless functionality.","If you are working on projects that require rigorous testing and validation of code derived from natural language descriptions."],"when_not_to_use":["Avoid Lever if developing in an environment other than Linux as it requires the use of tree-sitter parsers, which may not be compatible with your system.","Do not choose Lever if you seek a tool that does not require setting up a conda environment and installing specific dependencies to operate."],"source":"enrich:decision_facts","observed_at":"2026-07-17T13:28:39.797Z"},"constraint_facets":{"min_ram_gb":null,"requires_docker":false},"decision_summary":[{"label":"Requirements","value":"All of the pipelines have been tested on Linux machines only, requiring possibly custom `tree-sitter` parsers for other platforms."},{"label":"Adopt for","value":"Lever offers support for verifying language-to-code generation through actual code execution."},{"label":"License detail","value":"Lever's source code is freely available under an MIT License for modification and distribution in both personal and commercial projects."}]}}