The Practical Guides for Large Language Models

A curated (still actively updated) list of practical guide resources of LLMs. It's based on our survey paper: Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond and efforts from @xinyadu. The survey is partially based on the second half of this Blog. We also build an evolutionary tree of modern Large Language Models (LLMs) to trace the development of language models in recent years and highlights some of the most well-known models.

These sources aim to help practitioners navigate the vast landscape of large language models (LLMs) and their applications in natural language processing (NLP) applications. We also include their usage restrictions based on the model and data licensing information. If you find any resources in our repository helpful, please feel free to use them (don't forget to cite our paper! 😃). We welcome pull requests to refine this figure!

    @article{yang2023harnessing,
        title={Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond}, 
        author={Jingfeng Yang and Hongye Jin and Ruixiang Tang and Xiaotian Han and Qizhang Feng and Haoming Jiang and Bing Yin and Xia Hu},
        year={2023},
        eprint={2304.13712},
        archivePrefix={arXiv},
        primaryClass={cs.CL}
    }

Latest News💥

• We added usage and restrictions section.
• We used PowerPoint to plot the figure and released the source file pptx for our GIF figure. [4/27/2023]
• We released the source file for the still version pptx, and replaced the figure in this repo with the still version. [4/29/2023]
• Add AlexaTM, UniLM, UniLMv2 to the figure, and correct the logo for Tk. [4/29/2023]
• Add usage and Restrictions (for commercial and research purposes) section. Credits to Dr. Du. [5/8/2023]

Other Practical Guides for LLMs

• Why did all of the public reproduction of GPT-3 fail? In which tasks should we use GPT-3.5/ChatGPT? 2023, Blog
• Building LLM applications for production, 2023, Blog
• Data-centric Artificial Intelligence, 2023, Repo/Blog/Paper

Catalog

• The Practical Guides for Large Language Models
  • Practical Guide for Models
    • BERT-style Language Models: Encoder-Decoder or Encoder-only
    • GPT-style Language Models: Decoder-only
  • Practical Guide for Data
    • Pretraining data
    • Finetuning data
    • Test data/user data
  • Practical Guide for NLP Tasks
    • Traditional NLU tasks
    • Generation tasks
    • Knowledge-intensive tasks
    • Abilities with Scaling
    • Specific tasks
    • Real-World ''Tasks''
    • Efficiency
    • [