Awesome-LLM-in-Social-Science

🔗 Recommended Resource:
Check out Awesome-LLM-Psychometrics for a comprehensive collection of papers and resources on LLM psychometrics, including evaluation, validation, and enhancement.

Below we compile awesome papers that

• evaluate Large Language Models (LLMs) from a perspective of Social Science.
• align LLMs from a perspective of Social Science.
• employ LLMs to facilitate research, address issues, and enhance tools in Social Science.
• contribute surveys, perspectives, and datasets on the above topics.

The above taxonomies are by no means orthogonal. For example, evaluations require simulations. We categorize these papers based on our understanding of their focus. This collection has a special focus on Psychology and intrinsic values.

Welcome to contribute and discuss!

🤩 Papers marked with a ⭐️ are contributed by the maintainers of this repository. If you find them useful, we would greatly appreciate it if you could give the repository a star and cite our paper.

@article{ye2025large,
  title={Large Language Model Psychometrics: A Systematic Review of Evaluation, Validation, and Enhancement},
  author={Ye, Haoran and Jin, Jing and Xie, Yuhang and Zhang, Xin and Song, Guojie},
  journal={arXiv preprint arXiv:2505.08245},
  year={2025},
  note={Project website: \url{https://llm-psychometrics.com}, GitHub: \url{https://github.com/ValueByte-AI/Awesome-LLM-Psychometrics}}
}

Table of Contents

• 1. 📚 Survey
• 2. 🗂️ Dataset
• 3. 🔎 Evaluating LLM
  • 3.1. ❤️ Value
  • 3.2. 🩷 Personality
  • 3.3. 🔞 Morality
  • 3.4. 🎤 Opinion
  • 3.5. 💚 General Preference
  • 3.6. 🧠 Ability
  • 3.7. ⚠️ Risk
• 4. ⚒️ Tool enhancement
• 5. ⛑️ Alignment
  • 5.1. 🌈 Pluralistic Alignment
• 6. 🚀 Simulation
• 7. 👁️‍🗨️ Perspective and Position

1. 📚 Survey

• ⭐️ Large Language Model Psychometrics: A Systematic Review of Evaluation, Validation, and Enhancement, 2025.05, [paper].
• Missing the Margins: A Systematic Literature Review on the Demographic Representativeness of LLMs, ACL 2025, [paper], [collection].
• Large language models (LLM) in computational social science: prospects, current state, and challenges, 2025.03, Social Network Analysis and Mining, [paper].
• Towards Scientific Intelligence: A Survey of LLM-based Scientific Agents, 2025.03, [paper].
• On the Trustworthiness of Generative Foundation Models: Guideline, Assessment, and Perspective, 2025.02, [paper].
• The Road to Artificial SuperIntelligence: A Comprehensive Survey of Superalignment, 2024.12, [paper].
• Large Language Model Safety: A Holistic Survey, 2024.12, [paper].
• Political-LLM: Large Language Models in Political Science, 2024.12, [paper], [website].
• LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods, 2024.12, [paper].
• From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agents, 2024.12, [paper], [repo].
• A Survey on Human-Centric LLMs, 2024.11, [paper].
• **Survey of Cultural Aware