# RMSNorm

**Repository Path**: hazdzz/RMSNorm

## Basic Information

- **Project Name**: RMSNorm
- **Description**: The PyTorch implementation of RMSNorm.
- **Primary Language**: Python
- **License**: MIT
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2024-06-13
- **Last Updated**: 2024-06-13

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# RMSNorm

## About
The PyTorch implementation of RMSNorm from the paper *Root Mean Square Layer Normalization*.

## Paper
https://arxiv.org/abs/1910.07467

## Citation
```
@inproceedings{NEURIPS2019_1e8a1942,
  title     = {Root Mean Square Layer Normalization},
  author    = {Zhang, Biao and Sennrich, Rico},
  booktitle = {Advances in Neural Information Processing Systems},
  editor    = {H. Wallach and H. Larochelle and A. Beygelzimer and F. d\textquotesingle Alch{\'{e}}-Buc and E. Fox and R. Garnett},
  pages     = {12360--12371},
  publisher = {Curran Associates, Inc.},
  volume    = {32},
  year      = {2019}
}
```