# gpt2-quickly-样例

**Repository Path**: qzl121/gpt2-quick-sample

## Basic Information

- **Project Name**: gpt2-quickly-样例
- **Description**: https://github.com/mymusise/gpt2-quickly/
- **Primary Language**: Python
- **License**: Not specified
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2021-09-06
- **Last Updated**: 2021-09-06

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

<h1 align="center">
GPT2 Quickly
</h1>

<h3 align="center">
<p>Build your own GPT2 quickly, without doing many useless work.
</h3>

<p align="center">
    <a href="https://colab.research.google.com/github/mymusise/gpt2-quickly/blob/main/examples/gpt2_quickly.ipynb">
        <img alt="Build" src="https://colab.research.google.com/assets/colab-badge.svg">
    </a>
</p>

This project is base on 🤗 transformer. This tutorial show you how to train your own language(such as chinese or Japanese) GPT2 model in a few code with Tensorflow 2.

You can try this project in [colab](https://colab.research.google.com/github/mymusise/gpt2-quickly/blob/main/examples/gpt2_quickly.ipynb) right now.

## Main file

``` 

├── configs
│   ├── test.py
│   └── train.py
├── build_tokenizer.py
├── predata.py
├── predict.py
└── train.py
```

## Preparation

``` bash
git clone git@github.com:mymusise/gpt2-quickly.git
cd gpt2-quickly
python3 -m venv venv
source venv/bin/activate

pip install -r requirements.txt
```

## 0x00. prepare your raw dataset

this is a example of raw dataset: [raw.txt](dataset/test/raw.txt)


## 0x01. Build vocab

```bash
python build_tokenizer.py
```


## 0x02. Tokenize

```bash
python predata.py --n_processes=2
```


## 0x03 Train

```bash
python train.py
```


## 0x04 Predict

```bash
python predict.py
```

## 0x05 Fine-Tune

```bash
ENV=FINETUNE python finetune.py
```