{"data":{"slug":"re-align-just-eval","name":"just-eval","tagline":"A simple GPT-based evaluation tool for multi-aspect, interpretable assessment of LLMs.","github_url":"https://github.com/Re-Align/just-eval","owner":"Re-Align","repo":"just-eval","owner_avatar_url":"https://avatars.githubusercontent.com/u/150300417?v=4","primary_language":"Python","stars":90,"forks":7,"topics":["evaluation","gpt4","llm","llm-eval","llm-evaluation","llm-evaluation-toolkit"],"archived":false,"github_pushed_at":"2024-01-29T23:45:40+00:00","maintenance_label":"Dormant","stars_delta_30d":0,"url":"https://www.graphcanon.com/tools/re-align-just-eval","markdown_url":"https://www.graphcanon.com/tools/re-align-just-eval.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/re-align-just-eval","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=re-align-just-eval","description":"A simple GPT-based evaluation tool for multi-aspect, interpretable assessment of LLMs.","homepage_url":"https://allenai.github.io/re-align/","license":"MIT","open_issues":2,"watchers":4,"ai_summary":"just-eval is an evaluation toolkit based on GPT technology designed to assess language models across multiple dimensions in an understandable way.","readme_excerpt":"## Installation \n\n```bash \ngit clone https://github.com/Re-Align/just-eval.git\ncd just_eval\npip install .\n```\n\nor \n```bash \npip install git+https://github.com/Re-Align/just-eval.git\n```\n\n***Setup OpenAI API Key***\n\n```bash \nexport OPENAI_API_KEY=<your secret key>\n```","github_created_at":"2023-11-19T02:51:26+00:00","created_at":"2026-07-15T10:39:19.477847+00:00","updated_at":"2026-10-18T06:00:10.641491+00:00","categories":[{"slug":"evaluation-observability","name":"Evaluation & Observability","url":"https://www.graphcanon.com/categories/evaluation-observability","markdown_url":"https://www.graphcanon.com/categories/evaluation-observability.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/evaluation-observability"}],"tags":[{"slug":"evaluation","name":"evaluation"},{"slug":"gpt4","name":"gpt4"},{"slug":"llm","name":"llm"},{"slug":"llm-eval","name":"llm-eval"},{"slug":"llm-evaluation","name":"llm-evaluation"},{"slug":"llm-evaluation-toolkit","name":"llm-evaluation-toolkit"}],"trust":{"provenance":{"is_fork":false,"github_id":720615291,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-10-18T06:00:09.486Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":992,"last_release_at":null,"stars_delta_30d":0,"open_issues_delta_30d":0},"security_summary":{"last_scan_at":"2026-07-15T10:39:21.85274+00:00","scan_profile":"deps","status":"no_lockfile","critical_count":0,"high_count":0,"medium_count":0,"low_count":0,"scanner":"osv@v1","scan_id":"a8f98f95-f1de-479a-8f93-c6242aee5a27","pending_review_count":0}},"capability_facts":{"mcp":{"method":"github-tree-and-contents@2","reason":"No explicit MCP server declaration was found in the inspected manifests. Other layouts and implementations were not tested.","source":"repo_scan","status":"unknown","revision":"3e1a1265e210be1d6ad71624c91da3efc36493ca","source_url":"https://github.com/Re-Align/just-eval/tree/3e1a1265e210be1d6ad71624c91da3efc36493ca","observed_at":"2026-10-18T06:00:09.899Z","server_manifest":null},"scan":{"method":"github-tree-and-contents@2","source":"repo_scan","status":"complete","version":2,"coverage":{"scope":"bounded-repository-manifests","tree_complete":true,"selected_files":0,"candidate_files":0},"revision":"3e1a1265e210be1d6ad71624c91da3efc36493ca","source_url":"https://github.com/Re-Align/just-eval/tree/3e1a1265e210be1d6ad71624c91da3efc36493ca","observed_at":"2026-10-18T06:00:09.899Z","observations":[{"path":"@revision","bytes":1391,"status":"present","revision":"3e1a1265e210be1d6ad71624c91da3efc36493ca","source_url":"https://github.com/Re-Align/just-eval/commit/3e1a1265e210be1d6ad71624c91da3efc36493ca","http_status":200,"observed_at":"2026-10-18T06:00:10.082Z"},{"path":"@tree","bytes":36341,"status":"present","revision":"3e1a1265e210be1d6ad71624c91da3efc36493ca","source_url":"https://github.com/Re-Align/just-eval/tree/3e1a1265e210be1d6ad71624c91da3efc36493ca","http_status":200,"observed_at":"2026-10-18T06:00:10.257Z"},{"path":"docker-compose.yml","reason":"not-listed-in-complete-tree","status":"not-found","revision":"3e1a1265e210be1d6ad71624c91da3efc36493ca","source_url":"https://github.com/Re-Align/just-eval/tree/3e1a1265e210be1d6ad71624c91da3efc36493ca","observed_at":"2026-10-18T06:00:10.258Z"},{"path":"Dockerfile","reason":"not-listed-in-complete-tree","status":"not-found","revision":"3e1a1265e210be1d6ad71624c91da3efc36493ca","source_url":"https://github.com/Re-Align/just-eval/tree/3e1a1265e210be1d6ad71624c91da3efc36493ca","observed_at":"2026-10-18T06:00:10.258Z"},{"path":"mcp.json","reason":"not-listed-in-complete-tree","status":"not-found","revision":"3e1a1265e210be1d6ad71624c91da3efc36493ca","source_url":"https://github.com/Re-Align/just-eval/tree/3e1a1265e210be1d6ad71624c91da3efc36493ca","observed_at":"2026-10-18T06:00:10.258Z"},{"path":"package.json","reason":"not-listed-in-complete-tree","status":"not-found","revision":"3e1a1265e210be1d6ad71624c91da3efc36493ca","source_url":"https://github.com/Re-Align/just-eval/tree/3e1a1265e210be1d6ad71624c91da3efc36493ca","observed_at":"2026-10-18T06:00:10.258Z"},{"path":"pyproject.toml","reason":"not-listed-in-complete-tree","status":"not-found","revision":"3e1a1265e210be1d6ad71624c91da3efc36493ca","source_url":"https://github.com/Re-Align/just-eval/tree/3e1a1265e210be1d6ad71624c91da3efc36493ca","observed_at":"2026-10-18T06:00:10.258Z"},{"path":"server.json","reason":"not-listed-in-complete-tree","status":"not-found","revision":"3e1a1265e210be1d6ad71624c91da3efc36493ca","source_url":"https://github.com/Re-Align/just-eval/tree/3e1a1265e210be1d6ad71624c91da3efc36493ca","observed_at":"2026-10-18T06:00:10.258Z"}]},"languages":{"value":["python"],"method":"github-tree-and-contents@2","source":"github.language","revision":"3e1a1265e210be1d6ad71624c91da3efc36493ca","source_url":"https://github.com/Re-Align/just-eval/tree/3e1a1265e210be1d6ad71624c91da3efc36493ca","observed_at":"2026-10-18T06:00:09.899Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-10-18T06:00:09.899Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["When your team requires an in-depth yet interpretable evaluation of LLM performance, focusing on various aspects.","If you aim to leverage GPT technology for assessing the quality and reliability of language models without complex setup procedures."],"when_not_to_use":["Avoid if direct integration with competing API services is needed as just-eval focuses solely on OpenAI API key configuration.","Not suitable when your project demands a custom evaluation framework that extends beyond simple GPT-based assessments."],"source":"enrich:decision_facts","observed_at":"2026-07-16T20:57:41.061Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"Unverified catalog guidance: just-eval provides a straightforward GPT-based framework for evaluating LLMs across multiple dimensions with clear outputs."}]}}