awesome-language-model-analysis
A curated list of papers focusing on the theoretical analysis of large language models.
GraphCanon updated 2w · GitHub synced 2w
Decision brief
Curated List of Theoretical Papers on Large Language Models
Good fit when
- When you seek an in-depth theoretical understanding and formal/mathematical proofs related to the learning behavior and generalization ability of transformer-based large language models.
- If you are interested in research involving formal proofs, provable guarantees, bounds, expressivity results, convergence analysis specifically for transformer-based models.
Avoid when
- Avoid relying on this list if purely empirical or observational studies are more relevant to your needs as they are excluded from the repository.
- You should not use this resource if a comprehensive coverage of mechanistic engineering, probing, and interpretability is required, as these topics are currently less covered.
- Requirements:
- Some knowledge in theoretical computer science or mathematics is advised to fully comprehend the papers listed.; Python proficiency might be beneficial for implementing models based on theoretical findings.
Observed Jul 11, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Active (8d since push)
- As of 2w
- Provenance
- Not a fork · Personal account
- As of 2w
- Security (OSV)
- 5 low (5 low)
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install awesome-language-model-analysisSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
This repository contains a collection of 664 papers that focus on the theoretical and empirical analysis of transformer-based language models, emphasizing their properties such as learning behavior and generalization ability through formal/mathematical proofs, provable guarantees, bounds, expressivity results, convergence analysis. The list excludes purely empirical studies.
Capability facts
- Languages
- python
Source: github.language · Aug 6, 2026
Categories
Tags
README
Awesome Language Model Analysis
This paper list focuses on the theoretical analysis of language models, especially large language models (LLMs). The papers in this list investigate the learning behavior, generalization ability, and other properties of language models through formal/mathematical analysis -- proofs, provable guarantees, bounds, expressivity results, convergence analysis, and similar. Papers that also include supporting experiments still count; purely empirical/observational papers do not.
Scope of this list:
- Currently, this list focuses on transformer-based models.
- We collect papers with a genuine theoretical contribution, instead of purely empirical studies or papers that aim to improve the performance of language models without theoretically analyzing why.
Limitations of this list:
- This list is not exhaustive, and we may miss some very important papers.
- This list is not well-organized yet, and we may need to reorganize the list in the future.
- Some popular topics are not well-covered yet, such as mechanistic engineering, probing, and interpretability.
Statistics of This paper list:
- Total number of different papers: 721
- For more detailed statistics, please refer to the end of this page.
If you have any suggestions or want to contribute, please feel free to open an issue or a pull request.
For details on how to contribute, please refer to the contribution guidelines.
You can also share your thoughts and discuss with others in the Discussions.
[!NOTE]
For uncategorized version, please refer to here.
Table of Content
- Awesome Language Model Analysis
- Table of Content
- Phenomena of Interest
- In-Context Learning
- Chain-of-Thought
- Hallucination
- Reversal Curse
- Scaling Laws / Emergent Abilities / Grokking / etc.
- Knowledge / Memory Mechanisms
- Training Dynamics / Landscape / Optimization / Fine-tuning / etc.
- Learning / Generalization / Reasoning / Weak to Strong Generalization
- Other Phenomena / Discoveries
- Representational Capacity
- What Can Transformer Do? / Properties of Transformer
- What Can Transformer Not Do? / Limitation of Transformer
- Architectural Effectivity
- Layer-normalization
- Tokenization / Embedding
- Linear Attention / State Space Models / Recurrent Language Models / etc.
- Training Paradigms
- Mechanistic Engineering / Probing / Interpretability
- Miscellanea
- Phenomena of Interest
Phenomena of Interest
^ back to top ^
Categories focusing on different phenomena, properties, and behaviors observed in large language models (LLMs) and transformer-based models.
In-Context Learning
^ back to top ^
Papers focusing on the theoretical and empirical analysis of in-context learning in large language models.
paper list (
For agents
This page has a .md twin and JSON over the API.