{"data":{"slug":"bigscience-workshop-model-card","name":"model_card","tagline":"Model Card for BLOOM LM","github_url":"https://github.com/bigscience-workshop/model_card","owner":"bigscience-workshop","repo":"model_card","owner_avatar_url":"https://avatars.githubusercontent.com/u/82455566?v=4","primary_language":null,"stars":25,"forks":5,"topics":[],"archived":false,"github_pushed_at":"2022-07-11T18:57:23+00:00","maintenance_label":"Dormant","stars_delta_30d":-1,"url":"https://www.graphcanon.com/tools/bigscience-workshop-model-card","markdown_url":"https://www.graphcanon.com/tools/bigscience-workshop-model-card.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/bigscience-workshop-model-card","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=bigscience-workshop-model-card","description":null,"homepage_url":null,"license":"Apache-2.0","open_issues":0,"watchers":2,"ai_summary":"This repository contains the model card for BLOOM, a large, multilingual language model developed by BigScience. The model card provides details on the model's development, training data, risks, and evaluation.","readme_excerpt":"---\nlicense: bigscience-bloom-rail-1.0\nlanguage:\n- ak\n- ar\n- as\n- bm\n- bn\n- ca\n- code\n- en\n- es\n- eu\n- fon\n- fr\n- gu\n- hi\n- id\n- ig\n- ki\n- kn\n- lg\n- ln\n- ml\n- mr\n- ne\n- nso\n- ny\n- or\n- pa\n- pt\n- rn\n- rw\n- sn\n- st\n- sw\n- ta\n- te\n- tn\n- ts\n- tum\n- tw\n- ur\n- vi\n- wo\n- xh\n- yo\n- zh\n- zhs\n- zht\n- zu\n---\n\n# DEPRECATED. This is now live at https://huggingface.co/bigscience/bloom . Please make additional changes there!\n\n# <p>BLOOM LM<br/> _BigScience Large Open-science Open-access Multilingual Language Model_ <br/>Model Card</p>\n<img src=\"https://assets.website-files.com/6139f3cdcbbff3a68486761d/613cd8997b270da063e230c5_Tekengebied%201-p-500.png\" alt=\"BigScience Logo\" width=\"200\"/>\n\n\nVersion 1.0 / 25.May.2022\n\n## Table of Contents\n1. [Model Details](#model-details)\n2. [Uses](#uses)\n3. [Training Data](#training-data)\n4. [Risks and Limitations](#risks-and-limitations)\n5. [Evaluation](#evaluation)\n6. [Recommendations](#recommendations)\n7. [Glossary and Calculations](#glossary-and-calculations)\n8. [More Information](#more-information)\n9. [Model Card Authors](#model-card-authors)\n\n## Model Details  \n\n### Basics\n*This section provides information for anyone who wants to know about the model.*\n\n<details>\n<summary>Click to expand</summary> <br/>\n    \n**Developed by:** BigScience ([website](https://bigscience.huggingface.co))\n\n* All collaborators are either volunteers or have an agreement with their employer. *(Further breakdown of participants forthcoming.)*\n    \n**Model Type:** Transformer-based Language Model\n\n**Version:** 1.0.0\n\n**Languages:** Multiple; see [training data](#training-data)\n\n**License:** RAIL License v1.0 ([link](https://huggingface.co/spaces/bigscience/license))\n\n**Release Date Estimate:** Monday, 11.July.2022\n\n**Send Questions to:** bigscience-contact@googlegroups.com\n\n**Cite as:** BigScience, _BigScience Language Open-source Open-access Multilingual (BLOOM) Language Model_. International, May 2021-May 2022\n\n**Funded by:** \n    \n* The French government.\n\n* Hugging Face ([website](https://huggingface.co)).\n\n* Organizations of contributors.  *(Further breakdown of organizations forthcoming.)*\n\n</details>\n\n### Technical Specifications\n*This section provides information for people who work on model development.*\n\n<details>\n<summary>Click to expand</summary><br/>\n\nPlease see [the BLOOM training README](https://github.com/bigscience-workshop/bigscience/tree/master/train/tr11-176B-ml#readme) for full details on replicating training.\n\n**Model Architecture:** Modified from Megatron-LM GPT2 (see [paper](https://arxiv.org/abs/1909.08053), [BLOOM Megatron code](https://github.com/bigscience-workshop/Megatron-DeepSpeed)):\n\n* Decoder-only architecture\n\n* Layer normalization applied to word embeddings layer (`StableEmbedding`; see [code](https://github.com/facebookresearch/bitsandbytes), [paper](https://arxiv.org/pdf/2110.02861.pdf))\n\n* ALiBI positional encodings (see [paper](https://arxiv.org/pdf/2108.12409.pdf)), with GeLU activation functions\n\n* 176 billion parameters:\n\n    * 70 layers, 112 attention heads\n\n    * Hidden layers are 14336-dimensional\n\n    * Sequence length of 2048 tokens used (see [BLOOM tokenizer](https://huggingface.co/bigscience/tokenizer), [tokenizer description](#tokenization))\n\n**Objective Function:** Cross Entropy with mean reduction (see [API documentation](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html#torch.nn.CrossEntropyLoss)).\n    \n**Compute infrastructure:** Jean Zay Public Supercomputer, provided by the French government (see [announcement](https://www.enseignementsup-recherche.gouv.fr/fr/signature-du-marche-d-acquisition-de-l-un-des-supercalculateurs-les-plus-puissants-d-europe-46733)).\n\n* Hardware: 384 A100 80GB GPUs (48 nodes):\n    \n    * Additional 32 A100 80GB GPUs (4 nodes) in reserve\n\n    *  8 GPUs per node Using NVLink 4 inter-gpu connects, 4 OmniPath links\n\n    *   CPU: AMD\n\n    *   CPU memory: 512GB per node\n\n    *   GPU memory: 640GB per node\n\n    *   Inter-node connec","github_created_at":"2022-05-05T18:19:05+00:00","created_at":"2026-07-11T23:08:31.266307+00:00","updated_at":"2026-09-20T04:29:16.111468+00:00","categories":[{"slug":"evaluation-observability","name":"Evaluation & Observability","url":"https://www.graphcanon.com/categories/evaluation-observability","markdown_url":"https://www.graphcanon.com/categories/evaluation-observability.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/evaluation-observability"},{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"}],"tags":[{"slug":"bigscience","name":"bigscience"},{"slug":"bloom","name":"bloom"},{"slug":"language-model","name":"language-model"},{"slug":"model-card","name":"model card"},{"slug":"multilingual","name":"multilingual"}],"trust":{"provenance":{"is_fork":false,"github_id":489082066,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-18T08:14:13.631Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":1529,"last_release_at":null,"stars_delta_30d":-1,"open_issues_delta_30d":0},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-09-18T08:14:15.264Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-18T08:14:12.552Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-09-18T08:14:12.552Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":{"notes":["The model card is intended for use with the BLOOM language model and provides details on its architecture, training data, and evaluation.","It is recommended to refer to the BLOOM model card for comprehensive information before using the model in any application."]},"constraints":null,"when_to_use":["When you need detailed information on the development and training of BLOOM, a large multilingual language model.","If you are working with a wide range of languages, including less-resourced languages, as BLOOM supports over 40 languages.","For evaluating the risks and limitations associated with using BLOOM in various applications.","When you require a transparent overview of the model's architecture, training data, and evaluation metrics."],"when_not_to_use":["If you are looking for a model card for a different language model, as this card is specific to BLOOM.","When you need real-time updates or interactive features, as the model card is static and does not provide dynamic information.","If you require a model card for a proprietary or closed-source model, as BLOOM is an open-access model.","When you need a model card that includes specific use cases or applications beyond the general information provided."],"source":"enrich:decision_facts","observed_at":"2026-09-18T09:46:45.141Z"},"constraint_facets":null,"decision_summary":[{"label":"Requirements","value":"The model card is intended for use with the BLOOM language model and provides details on its architecture, training data, and evaluation.; It is recommended to refer to the BLOOM model card for comprehensive information before using the model in any application."},{"label":"Adopt for","value":"The model_card for BLOOM LM provides comprehensive details on the development, training data, risks, and evaluation of a large, multilingual language model. It is useful for researchers and developers working with multil"},{"label":"License detail","value":"The model card is licensed under the Apache-2.0 license, allowing for broad use and modification with attribution."}]}}