{"data":{"slug":"01-ai-yi","name":"Yi","tagline":"A series of large language models trained from scratch","github_url":"https://github.com/01-ai/Yi","owner":"01-ai","repo":"Yi","owner_avatar_url":"https://avatars.githubusercontent.com/u/139308978?v=4","primary_language":"Jupyter Notebook","stars":7833,"forks":499,"topics":["large-language-models"],"archived":false,"github_pushed_at":"2024-11-27T09:25:57+00:00","maintenance_label":"Dormant","stars_delta_30d":11,"url":"https://www.graphcanon.com/tools/01-ai-yi","markdown_url":"https://www.graphcanon.com/tools/01-ai-yi.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/01-ai-yi","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=01-ai-yi","description":"A series of large language models trained from scratch by developers @01-ai","homepage_url":"https://01.ai","license":"Apache-2.0","open_issues":31,"watchers":107,"ai_summary":"Provides large language models and instructions for local deployment and inference.","readme_excerpt":"## Quick start\n\n> **💡 Tip**: If you want to get started with the Yi model and explore different methods for inference, check out the [Yi Cookbook](https://github.com/01-ai/Yi/tree/main/Cookbook).\n\n---\n\n### Quick start - pip \n\nThis tutorial guides you through every step of running **Yi-34B-Chat locally on an A800 (80G)** and then performing inference.\n\n#### Step 0: Prerequisites\n\n- Make sure Python 3.10 or a later version is installed.\n\n- If you want to run other Yi models, see [software and hardware requirements](#deployment).\n\n#### Step 1: Prepare your environment \n\nTo set up the environment and install the required packages, execute the following command.\n\n```bash\ngit clone https://github.com/01-ai/Yi.git\ncd yi\npip install -r requirements.txt\n```\n\n#### Step 2: Download the Yi model\n\nYou can download the weights and tokenizer of Yi models from the following sources:\n\n- [Hugging Face](https://huggingface.co/01-ai)\n- [ModelScope](https://www.modelscope.cn/organization/01ai/)\n- [WiseModel](https://wisemodel.cn/organization/01.AI)\n\n#### Step 3: Perform inference\n\nYou can perform inference with Yi chat or base models as below.\n\n##### Perform inference with Yi chat model\n\n1. Create a file named  `quick_start.py` and copy the following content to it.\n\n    ```python\n    from transformers import AutoModelForCausalLM, AutoTokenizer\n\n    model_path = '<your-model-path>'\n\n    tokenizer = AutoTokenizer.from_pretrained(model_path, use_fast=False)\n\n    # Since transformers 4.35.0, the GPT-Q/AWQ model can be loaded using AutoModelForCausalLM.\n    model = AutoModelForCausalLM.from_pretrained(\n        model_path,\n        device_map=\"auto\",\n        torch_dtype='auto'\n    ).eval()\n\n    # Prompt content: \"hi\"\n    messages = [\n        {\"role\": \"user\", \"content\": \"hi\"}\n    ]\n\n    input_ids = tokenizer.apply_chat_template(conversation=messages, tokenize=True, add_generation_prompt=True, return_tensors='pt')\n    output_ids = model.generate(input_ids.to('cuda'))\n    response = tokenizer.decode(output_ids[0][input_ids.shape[1]:], skip_special_tokens=True)\n\n    # Model response: \"Hello! How can I assist you today?\"\n    print(response)\n    ```\n\n2. Run `quick_start.py`.\n\n    ```bash\n    python quick_start.py\n    ```\n\n    Then you can see an output similar to the one below. 🥳\n\n    ```bash\n    Hello! How can I assist you today?\n    ```\n\n##### Perform inference with Yi base model\n\n- Yi-34B\n\n  The steps are similar to [pip - Perform inference with Yi chat model](#perform-inference-with-yi-chat-model).\n\n  You can use the existing file [`text_generation.py`](https://github.com/01-ai/Yi/tree/main/demo).\n\n  ```bash\n  python demo/text_generation.py  --model <your-model-path>\n  ```\n\n  Then you can see an output similar to the one below. 🥳\n\n  <details>\n\n  <summary>Output. ⬇️ </summary>\n\n  <br>\n\n  **Prompt**: Let me tell you an interesting story about cat Tom and mouse Jerry,\n\n  **Generation**: Let me tell you an interesting story about cat Tom and mouse Jerry, which happened in my childhood. My father had a big house with two cats living inside it to kill mice. One day when I was playing at home alone, I found one of the tomcats lying on his back near our kitchen door, looking very much like he wanted something from us but couldn’t get up because there were too many people around him! He kept trying for several minutes before finally giving up...\n\n  </details>\n\n- Yi-9B\n  \n  Input\n\n  ```bash\n  from transformers import AutoModelForCausalLM, AutoTokenizer\n  \n  MODEL_DIR = \"01-ai/Yi-9B\"\n  model = AutoModelForCausalLM.from_pretrained(MODEL_DIR, torch_dtype=\"auto\")\n  tokenizer = AutoTokenizer.from_pretrained(MODEL_DIR, use_fast=False)\n  \n  input_text = \"# write the quick sort algorithm\"\n  inputs = tokenizer(input_text, return_tensors=\"pt\").to(model.device)\n  outputs = model.generate(**inputs, max_length=256)\n  print(tokenizer.decode(outputs[0], skip_special_tokens=True))\n  ```\n\n  Output\n\n  ```bash\n  # write the quick sort algorithm\n  def quick_sort(arr):\n      if len(a","github_created_at":"2023-11-03T16:08:37+00:00","created_at":"2026-07-07T17:34:27.186664+00:00","updated_at":"2026-09-30T21:18:17.567733+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"},{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"}],"tags":[{"slug":"inference","name":"inference"},{"slug":"jupyter-notebook","name":"jupyter-notebook"},{"slug":"large-language-models","name":"large-language-models"},{"slug":"local-deployment","name":"local-deployment"},{"slug":"pip","name":"pip"},{"slug":"transformers","name":"transformers"},{"slug":"yi","name":"yi"}],"trust":{"provenance":{"is_fork":false,"github_id":713967573,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-18T08:51:22.757Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":659,"last_release_at":null,"stars_delta_30d":11,"open_issues_delta_30d":0},"security_summary":{"status":"findings","scanner":"osv@v1","low_count":149,"high_count":0,"last_scan_at":"2026-09-18T08:51:24.943Z","medium_count":0,"scan_profile":"deps","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-18T08:51:21.666Z"},"deploy":{"source":"dockerfile:Dockerfile","self_host":true,"observed_at":"2026-09-18T08:51:21.666Z","managed_saas":false},"languages":{"value":["jupyter notebook","python"],"source":"github.language+pyproject.toml","observed_at":"2026-09-18T08:51:21.666Z"},"has_docker":{"value":true,"source":"dockerfile:Dockerfile","observed_at":"2026-09-18T08:51:21.666Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-09-18T08:51:21.666Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":{"notes":["Ensure Python 3.10 or later is installed.","Hardware requirements vary depending on the specific Yi model being used."],"min_ram_gb":80,"requires_docker":false},"constraints":{"min_ram_gb":80,"requires_docker":false},"when_to_use":["Use Yi if you require a large language model that can be deployed and run locally, especially on hardware like the A800 (80G).","Choose Yi if you are working with Python 3.10 or later and have the specific software and hardware requirements met for running Yi models."],"when_not_to_use":["Avoid Yi if your hardware does not meet the minimum requirements for running the models, such as the A800 (80G) for Yi-34B-Chat.","Do not use Yi if you prefer cloud-based solutions or do not have the capability to install and manage the local environment required for Yi."],"source":"enrich:decision_facts","observed_at":"2026-09-18T10:59:00.959Z"},"constraint_facets":{"min_ram_gb":80,"requires_docker":false},"decision_summary":[{"label":"Requirements","value":"Min 80 GB RAM; Ensure Python 3.10 or later is installed.; Hardware requirements vary depending on the specific Yi model being used."},{"label":"Adopt for","value":"Yi is a series of large language models trained from scratch, offering local deployment and inference capabilities. It is suitable for users who need to run large language models locally and have the necessary hardware,如"}]}}