# raspberry_kws

**Repository Path**: geek_dog/raspberry_kws

## Basic Information

- **Project Name**: raspberry_kws
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-05-14
- **Last Updated**: 2026-05-15

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Xiaoyun KWS ONNX

ModelScope `iic/speech_charctc_kws_phone-xiaoyun` converted to ONNX, with offline test and Raspberry Pi 4B realtime microphone demo.

## Files

- `models/xiaoyun_fsmn_kws.onnx`: FP32 ONNX model
- `models/xiaoyun_fsmn_kws.int8.onnx`: dynamic int8 ONNX model
- `examples/rpi_realtime_kws.py`: Raspberry Pi realtime KWS
- `examples/offline_wav_kws.py`: offline WAV test
- `examples/benchmark_kws_runtime.py`: runtime benchmark
- `scripts/export_xiaoyun_onnx.py`: export ONNX
- `scripts/quantize_xiaoyun_onnx.py`: quantize ONNX

## Raspberry Pi Setup

```bash
python3 -m venv .venv
. .venv/bin/activate
pip install -r requirements-rpi.txt
sudo apt-get update
sudo apt-get install -y libportaudio2 portaudio19-dev alsa-utils
```

List devices:

```bash
python examples/rpi_realtime_kws.py --list-devices
```

List ALSA controls. If device is `hw:3,0`, card is `3`:

```bash
python examples/rpi_realtime_kws.py --alsa-card 3 --list-alsa-controls
```

## Realtime Run

```bash
python examples/rpi_realtime_kws.py \
  --onnx models/xiaoyun_fsmn_kws.int8.onnx \
  --tokens thirdparty/speech_charctc_kws_phone-xiaoyun/funasr/tokens_2599.txt \
  --cmvn thirdparty/speech_charctc_kws_phone-xiaoyun/funasr/am.mvn.dim80_l2r2 \
  --device seeed2micvoicec \
  --input-channel mix \
  --alsa-card 3 \
  --alsa-set PGA=30dB \
  --ort-threads 2 \
  --eval-interval 0.5
```

Monitor output includes `score`, `rms`, `peak`, `avg_ms`, `rtf`, `queue`, and `dropped`.

- `rtf < 1.0`: processing is realtime
- `queue=0 dropped=0`: no audio backlog
- speech `peak` target: roughly `0.3-0.8`
- if `peak` is near `1.0`, reduce `PGA` or `--input-gain`

DC removal:

```bash
--dc-remove mean   # default, stateful running mean
--dc-remove hpf    # one-pole high-pass DC blocker
--dc-remove off    # raw audio
```

Do not remove DC independently per 100 ms audio block; it creates boundary clicks. The realtime script uses a stateful remover before both KWS and audio logging.

## Audio Logs

All audio logs use the same entry point: `--audio-log-dir` + `--audio-log-mode`.

```bash
# Save wake-hit windows only
--audio-log-dir logs/audio --audio-log-mode wake

# Save short periodic windows for calibration
--audio-log-dir logs/audio --audio-log-mode periodic --audio-log-interval 5

# Save the whole long-running process as segmented WAV files
--audio-log-dir logs/audio --audio-log-mode continuous --audio-log-segment-sec 600

# Save continuous audio plus wake/periodic windows
--audio-log-dir logs/audio --audio-log-mode all --audio-log-interval 10 --audio-log-segment-sec 600
```

Metadata is written to `events.jsonl` and `continuous_segments.jsonl` in the same log directory.

## Benchmark

```bash
python examples/benchmark_kws_runtime.py --onnx models/xiaoyun_fsmn_kws.onnx --ort-threads 2
python examples/benchmark_kws_runtime.py --onnx models/xiaoyun_fsmn_kws.int8.onnx --ort-threads 2
```

Use the model and thread count with the lower `avg_ms` on your Pi.

## Offline Test

```bash
python examples/offline_wav_kws.py \
  --onnx models/xiaoyun_fsmn_kws.int8.onnx \
  --wav thirdparty/speech_charctc_kws_phone-xiaoyun/example/kws_xiaoyunxiaoyun.wav
```

## Export / Quantize

```bash
python scripts/export_xiaoyun_onnx.py \
  --checkpoint thirdparty/speech_charctc_kws_phone-xiaoyun/funasr/finetune_fsmn_4e_l10r2_250_128_fdim80_t2599_xiaoyun_xiaoyun.pt \
  --output models/xiaoyun_fsmn_kws.onnx

python scripts/quantize_xiaoyun_onnx.py \
  --input models/xiaoyun_fsmn_kws.onnx \
  --output models/xiaoyun_fsmn_kws.int8.onnx
```