{"data":{"slug":"hithink-research-bizfinbench","name":"BizFinBench","tagline":"A Business-Driven Real-World Financial Benchmark for Evaluating LLMs","github_url":"https://github.com/HiThink-Research/BizFinBench","owner":"HiThink-Research","repo":"BizFinBench","owner_avatar_url":"https://avatars.githubusercontent.com/u/186997494?v=4","primary_language":"Python","stars":167,"forks":12,"topics":["benchmark","finance","llm","llm-benchmarking","llm-evaluation"],"archived":false,"github_pushed_at":"2026-05-01T01:36:34+00:00","maintenance_label":"Steady","url":"https://www.graphcanon.com/tools/hithink-research-bizfinbench","markdown_url":"https://www.graphcanon.com/tools/hithink-research-bizfinbench.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/hithink-research-bizfinbench","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=hithink-research-bizfinbench","description":"A Business-Driven Real-World Financial Benchmark for Evaluating LLMs","homepage_url":"https://hithink-research.github.io/BizFinBench/","license":null,"open_issues":0,"watchers":2,"ai_summary":null,"readme_excerpt":"### Install requirements\n```sh\npip install -r requirements.txt\n```\n\n---\n\n### Quick Start – Evaluate a Local Model\n\n```sh\nexport MODEL_PATH=model/Qwen2.5-0.5B   # Path to the model to be evaluated\nexport REMOTE_MODEL_PORT=16668\nexport REMOTE_MODEL_URL=http://127.0.0.1:${REMOTE_MODEL_PORT}/model\nexport MODEL_NAME=Qwen2.5-0.5B\nexport PROMPT_TYPE=chat_template   # Hithink llama3 llama2 none qwen chat_template; chat_template is recommended\n\n---\n\n### Quick Start – Evaluate a Local Model and Score with a Judge Model\n\n```sh\nexport MODEL_PATH=model/Qwen2.5-0.5B   # Path to the model to be evaluated\nexport REMOTE_MODEL_PORT=16668\nexport REMOTE_MODEL_URL=http://127.0.0.1:${REMOTE_MODEL_PORT}/model\nexport MODEL_NAME=Qwen2.5-0.5B\nexport PROMPT_TYPE=chat_template   # llama3 llama2 none qwen chat_template; chat_template is recommended\n\n---\n\n### Quick Start – Evaluate external apis (e.g., chatgpt)\n\n```sh\nexport API_NAME=chatgpt # The api name, currently support chatgpt\nexport API_KEY=xxx # Your api key\nexport MODEL_NAME=gpt-4.1\n\n---\n\n## 📄 License\n  **Usage and License Notices**: The data and code are intended and licensed for research use only.\nLicense: Attribution-NonCommercial 4.0 International It should abide by the policy of OpenAI: https://openai.com/policies/terms-of-use","github_created_at":"2025-05-07T07:28:26+00:00","created_at":"2026-07-11T12:01:35.952814+00:00","updated_at":"2026-07-11T12:01:55.619756+00:00","categories":[{"slug":"evaluation-observability","name":"Evaluation & Observability","url":"https://www.graphcanon.com/categories/evaluation-observability","markdown_url":"https://www.graphcanon.com/categories/evaluation-observability.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/evaluation-observability"},{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"}],"tags":[{"slug":"benchmark","name":"benchmark"},{"slug":"finance","name":"finance"},{"slug":"llm","name":"llm"},{"slug":"llm-benchmarking","name":"llm-benchmarking"},{"slug":"llm-evaluation","name":"llm-evaluation"},{"slug":"python","name":"python"}],"trust":{"provenance":{"is_fork":false,"github_id":979225416,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-11T12:01:36.723Z","maintenance":{"label":"Steady","score":60,"methodology":"github_public_v1","releases_90d":0,"days_since_push":71,"last_release_at":null},"security_summary":{"status":"findings","scanner":"osv@v1","low_count":90,"high_count":0,"last_scan_at":"2026-07-11T12:01:43.843Z","medium_count":0,"scan_profile":"deps","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-11T12:01:43.259Z"},"languages":{"value":["python"],"source":"github.language","observed_at":"2026-07-11T12:01:43.259Z"}}}}