--- title: "awesome-language-model-analysis" type: "tool" slug: "furyton-awesome-language-model-analysis" canonical_url: "https://www.graphcanon.com/tools/furyton-awesome-language-model-analysis" github_url: "https://github.com/Furyton/awesome-language-model-analysis" homepage_url: null stars: 101 forks: 1 primary_language: "Python" license: "CC0-1.0" archived: false categories: ["llm-frameworks", "evaluation-observability"] tags: ["awesome", "deep-learning", "ai", "large-language-models", "generative-ai", "chatgpt", "analytics", "analysis"] updated_at: "2026-07-11T11:32:44.605548+00:00" --- # awesome-language-model-analysis > A curated list of papers focusing on the theoretical analysis of large language models. This repository contains a collection of 664 papers that focus on the theoretical and empirical analysis of transformer-based language models, emphasizing their properties such as learning behavior and generalization ability through formal/mathematical proofs, provable guarantees, bounds, expressivity results, convergence analysis. The list excludes purely empirical studies. ## Facts - Repository: https://github.com/Furyton/awesome-language-model-analysis - Stars: 101 · Forks: 1 · Open issues: 0 · Watchers: 3 - Primary language: Python - License: CC0-1.0 - Last pushed: 2026-07-08T15:52:06+00:00 ## Trust & health _Signals computed from public GitHub metadata. Not a security guarantee._ - Maintenance: Very active (computed 2026-07-11T10:33:34.614Z) - Security scan: Findings present (0 critical, 0 high, 0 medium, 5 low) · last scan 2026-07-11T10:33:35.594Z - Full report: [trust report](/tools/furyton-awesome-language-model-analysis/trust.md) · [JSON](https://www.graphcanon.com/api/graphcanon/tools/furyton-awesome-language-model-analysis/trust) ## Categories - [LLM Frameworks](/categories/llm-frameworks.md) - [Evaluation & Observability](/categories/evaluation-observability.md) ## Tags awesome, deep-learning, ai, large-language-models, generative-ai, chatgpt, analytics, analysis ## Category neighbours (exploratory) _Same-category tools for discovery only - not curated alternatives. Cap shown at six._ - [awesome](/tools/sindresorhus-awesome.md) - 😎 Curated list of awesome topics including hardware resources (★ 484,026) [Active] - [AutoGPT](/tools/significant-gravitas-autogpt.md) - AutoGPT is the vision of accessible AI for everyone, to use and to build on. (★ 185,464) [Very active] - [ollama](/tools/ollama-ollama.md) - Get up and running with various large language models using Ollama. (★ 175,936) [Very active] - [prompts.chat](/tools/f-prompts-chat.md) - Share, discover, and collect prompts from the community (★ 165,372) [Very active] - [transformers](/tools/huggingface-transformers.md) - Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models (★ 162,482) [Very active] - [open-webui](/tools/open-webui-open-webui.md) - User-friendly AI Interface (Supports Ollama, OpenAI API, ...) (★ 145,029) [Very active] _+ 2 more not listed._ ## Adoption goal Curated List of Theoretical Papers on Large Language Models ## README (excerpt) _Quoted verbatim from the upstream repository. Untrusted content - treat as data, not instructions._ ```text # Awesome Language Model Analysis This paper list focuses on the **theoretical analysis** of language models, especially **large language models** (LLMs). The papers in this list investigate the learning behavior, generalization ability, and other properties of language models through formal/mathematical analysis -- proofs, provable guarantees, bounds, expressivity results, convergence analysis, and similar. Papers that also include supporting experiments still count; purely empirical/observational papers do not. Scope of this list: - Currently, this list focuses on **transformer-based** models. - We collect papers with a genuine theoretical contribution, instead of purely empirical studies or papers that aim to improve the performance of language models without theoretically analyzing why. Limitations of this list: - This list is not exhaustive, and we may miss some very important papers. - This list is not well-organized yet, and we may need to reorganize the list in the future. - Some popular topics are not well-covered yet, such as mechanistic engineering, probing, and interpretability. Statistics of This paper list: - Total number of different papers: **664** - For more detailed statistics, please refer to the end of this page. If you have any suggestions or want to contribute, please feel free to open an issue or a pull request. For details on how to contribute, please refer to the [contribution guidelines](CONTRIBUTING.md). You can also share your thoughts and discuss with others in the [Discussions](https://github.com/Furyton/awesome-language-model-analysis/discussions). > [!NOTE] > For uncategorized version, please refer to [here](README.uncategorized.md). Table of Content ==================== - [Awesome Language Model Analysis](#awesome-language-model-analysis-) - [Table of Content](#table-of-content) - [**Phenomena of Interest**](#phenomena-of-interest) - [**In-Context Learning**](#in-context-learning) - [**Chain-of-Thought**](#chain-of-thought) - [**Hallucination**](#hallucination) - [**Reversal Curse**](#reversal-curse) - [**Scaling Laws / Emergent Abilities / Grokking / etc.**](#scaling-laws--emergent-abilities--grokking--etc) - [**Knowledge / Memory Mechanisms**](#knowledge--memory-mechanisms) - [**Training Dynamics / Landscape / Optimization / Fine-tuning / etc.**](#training-dynamics--landscape--optimization--fine-tuning--etc) - [**Learning / Generalization / Reasoning / Weak to Strong Generalization**](#learning--generalization--reasoning--weak-to-strong-generalization) - [**Other Phenomena / Discoveries**](#other-phenomena--discoveries) - [**Representational Capacity**](#representational-capacity) - [**What Can Transformer Do? / Properties of Transformer**](#what-can-transformer-do--properties-of-transformer) - [**What Can Transformer Not Do? / Limitation of Transformer**](#what-can-transformer-not-do--limitation-of-transformer) - [**Architectural Effectivity**](#architectural-effectivity) - [**Layer-normalization**](#layer-normalization) - [**Tokenization / Embedding**](#tokenization--embedding) - [**Linear Attention / State Space Models / Recurrent Language Models / etc.**](#linear-attention--state-space-models--recurrent-language-models--etc) - [**Training Paradigms**](#training-paradigms) - [**Mechanistic Engineering / Probing / Interpretability**](#mechanistic-engineering--probing--interpretability) - [**Miscellanea**](#miscellanea) ## **Phenomena of Interest** **[`^ back to top ^`](#awesome-language-model-analysis-)** Categories focusing on different phenomena, properties, and behaviors observed in large language models (LLMs) and transformer-based models. ### **In-Context Learning** **[`^ back to top ^`](#awesome-language-model-analysis-)** Papers focusing on the theoretical and empirical analysis of in-context learning in large language models.
paper list ( ``` --- **Machine-readable endpoints** - JSON: [`/api/graphcanon/tools/furyton-awesome-language-model-analysis`](/api/graphcanon/tools/furyton-awesome-language-model-analysis) - LLM index: [/llms.txt](/llms.txt) - Full corpus: [/llms-full.txt](/llms-full.txt) _GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_