{"data":{"slug":"hannibal046-rwkv-howto","name":"RWKV-howto","tagline":"possibly useful materials for learning RWKV language model","github_url":"https://github.com/Hannibal046/RWKV-howto","owner":"Hannibal046","repo":"RWKV-howto","owner_avatar_url":"https://avatars.githubusercontent.com/u/38466901?v=4","primary_language":null,"stars":26,"forks":2,"topics":[],"archived":false,"github_pushed_at":"2023-06-08T15:54:11+00:00","maintenance_label":"Dormant","url":"https://www.graphcanon.com/tools/hannibal046-rwkv-howto","markdown_url":"https://www.graphcanon.com/tools/hannibal046-rwkv-howto.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/hannibal046-rwkv-howto","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=hannibal046-rwkv-howto","description":"possibly useful materials for learning RWKV language model.","homepage_url":null,"license":null,"open_issues":0,"watchers":2,"ai_summary":"Materials and tutorials focused on understanding the RWVK language model which aims to combine the benefits of RNNs with transformer-like performance.","readme_excerpt":"# RWKV-howto\n\npossibly useful materials and tutorial for learning [RWKV](https://www.rwkv.com).\n\n> RWKV: Parallelizable RNN with Transformer-level LLM Performance.\n\n### Relevant Papers\n\n- :star2:(2023-05) RWKV: Reinventing RNNs for the Transformer Era [arxiv](https://arxiv.org/abs/2305.13048)\n- (2023-03) Resurrecting Recurrent Neural Networks for Long Sequences [arxiv](https://arxiv.org/abs/2303.06349)\n\n- (2023-02) SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks [arxiv](https://arxiv.org/abs/2302.13939)\n- (2022-08) Simplified State Space Layers for Sequence Modeling [ICLR2023](https://openreview.net/forum?id=Ai8Hw3AXqks)\n\n- :star2:(2021-05) An Attention Free Transformer [arxiv](https://arxiv.org/abs/2105.14103)\n\n- (2021-10) Efficiently Modeling Long Sequences with Structured State Spaces [ICLR2022](https://arxiv.org/abs/2111.00396) \n\n- (2020-08) Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention [ICML2020](https://arxiv.org/abs/2006.16236)\n- (2018) Parallelizing Linear Recurrent Neural Nets Over Sequence Length [ICLR2018](https://openreview.net/forum?id=HyUNwulC-)\n- (2017-09) Simple Recurrent Units for Highly Parallelizable Recurrence [EMNLP2017](https://arxiv.org/abs/1709.02755)\n- (2017-10) MinimalRNN: Toward More Interpretable and Trainable Recurrent Neural Networks [Neurips2017](https://arxiv.org/abs/1711.06788)\n- (2017-06) Attention Is All You Need [Neurips2017](https://arxiv.org/abs/1706.03762)\n- (2016-11) Quasi-Recurrent Neural Networks [ICLR2017](https://arxiv.org/abs/1611.01576)\n\n### Resources\n\n- Introducing RWKV - An RNN with the advantages of a transformer [Hugging Face](https://huggingface.co/blog/rwkv)\n- 有了Transformer框架后是不是RNN完全可以废弃了？[知乎](https://www.zhihu.com/question/302392659/answer/2954997969)\n- RNN最简单有效的形式是什么？[知乎](https://zhuanlan.zhihu.com/p/616357772)\n- :star2:RWKV的RNN CNN二象性 [知乎](https://zhuanlan.zhihu.com/p/614311961)\n- RNN的隐藏层需要非线性吗？[知乎](https://zhuanlan.zhihu.com/p/615672175)\n- Google新作试图“复活”RNN：RNN能否再次辉煌？ [苏剑林](https://kexue.fm/archives/9554)\n- :star2:How the RWKV language model works [Johan Sokrates Wind](https://www.mn.uio.no/math/english/people/aca/johanswi/index.html)\n\n- :star2:The RWKV language model: An RNN with the advantages of a transformer [Johan Sokrates Wind](https://johanwind.github.io/2023/03/23/rwkv_overview.html)\n- The Unreasonable Effectiveness of Recurrent Neural Networks [Andrej Karpathy blog](http://karpathy.github.io/2015/05/21/rnn-effectiveness/)\n\n### Code\n\n- [RKWV-LM](https://github.com/BlinkDL/RWKV-LM)\n- [ChatRWKV](https://github.com/BlinkDL/ChatRWKV)\n- [RWKV_in_150_lines](https://github.com/BlinkDL/ChatRWKV/blob/main/RWKV_in_150_lines.py)","github_created_at":"2023-05-21T08:54:14+00:00","created_at":"2026-07-11T10:32:13.775738+00:00","updated_at":"2026-07-11T11:33:22.280087+00:00","categories":[{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"}],"tags":[{"slug":"transformer","name":"transformer"},{"slug":"language-model","name":"language-model"},{"slug":"rnn","name":"rnn"}],"trust":{"provenance":{"is_fork":false,"github_id":643466665,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-11T10:32:14.369Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":1128,"last_release_at":null},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T10:32:19.600Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-11T11:32:57.378Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":{"notes":["The specific language and license details are not available for this repository. Review documentation directly from the RWKV repo provided."]},"constraints":null,"when_to_use":["- When you want to understand how an RNN can perform like a transformer while maintaining parallelizability.","- If your project requires efficient handling of long sequences without losing interpretability or trainability."],"when_not_to_use":["- When your focus is on standard transformers that don't require the combination of RNN benefits with modern transformer designs.","- If you need models that perform exceptionally well in tasks strictly dependent on attention mechanisms like those used in Vision Transformers."],"source":"enrich:decision_facts","observed_at":"2026-07-11T11:33:21.986Z"},"constraint_facets":null,"decision_summary":[{"label":"Requirements","value":"The specific language and license details are not available for this repository. Review documentation directly from the RWKV repo provided."},{"label":"Adopt for","value":"Materials and tutorials specific to the RWKV language model, which merges RNN benefits with transformer-like performance."}]}}