# volcengine_speech_plugin

**Repository Path**: pome/volcengine_speech_plugin

## Basic Information

- **Project Name**: volcengine_speech_plugin
- **Description**: 豆包asr
- **Primary Language**: Unknown
- **License**: MIT
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-07-17
- **Last Updated**: 2026-07-19

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Volcengine Speech Plugin

A Flutter plugin for [Volcengine (Volcano Engine / ByteDance)](https://www.volcengine.com) speech services, supporting **ASR (Automatic Speech Recognition)** and **TTS (Text-to-Speech)** on Android and iOS.

## Features

- **ASR** — Real-time streaming speech-to-text with VAD, partial / final results, volume levels.
- **TTS** — Text-to-speech with voice, speed, volume, pitch, and emotion.
- **Platforms** — Android and iOS via method channels.
- **Callbacks** — Engine lifecycle, recognition results, volume, TTS playback progress.

## Getting Started

```yaml
dependencies:
  volcengine_speech_plugin: ^0.0.2
  permission_handler: ^11.0.0 # microphone permission
```

### Android

`android/app/src/main/AndroidManifest.xml`:

```xml
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.INTERNET" />
```

### iOS

`ios/Runner/Info.plist`:

```xml
<key>NSMicrophoneUsageDescription</key>
<string>需要使用麦克风进行语音识别</string>
```

## Credentials

Do **not** hardcode production secrets in source control. Prefer remote config / secure storage:

```dart
const appId = 'your_app_id';
const token = 'Bearer;your_token';
const asrCluster = 'volcengine_input_common';
const ttsCluster = 'volcano_tts';
const voiceType = 'BV700_V2_streaming';
```

## Usage — ASR hold to talk (recommended)

Product flow: **press → start → release to send / cancel → wait for final text**.

Core plugin APIs:

| Method | Role |
| --- | --- |
| `registerListener` / `unRegisterListener` | Bind ASR / TTS callbacks |
| `initASREngine` / `unInitASREngine` | Create / destroy ASR engine |
| `startASREngine` / `stopASREngine` | Start / stop recognition |
| `onEnginePartialResult` | Streaming partial text |
| `onEngineFinalResult` | Final text after `stop` (`seFinalResult`) |
| `onEngineStop` | Native engine stopped |
| `onEngineError` | Native JSON error string |
| `onVolumeLevel` | Recording volume (string) |

### 1. Init + listen

```dart
import 'dart:async';
import 'dart:convert';

import 'package:flutter/widgets.dart';
import 'package:volcengine_speech_plugin/volcengine_speech_plugin.dart';

final plugin = VolcengineSpeechPlugin();

String sentences = '';
bool needsEngineReinit = false;
Completer<void>? engineStopCompleter;
Completer<void>? finalResultCompleter;

Future<bool?> initAsr({bool showNluPun = false}) async {
  try {
    await plugin.unInitASREngine();
  } catch (_) {}

  await plugin.registerListener(
    SpeechPluginListener(
      onEnginePartialResult: (text) {
        _absorbText(text, isFinal: false);
      },
      // After stopASREngine(), native may emit seFinalResult.
      // Always handle final, otherwise trailing text can be lost.
      onEngineFinalResult: (text) {
        _absorbText(text, isFinal: true);
        _completeFinalWait();
      },
      onVolumeLevel: (volumeLevel) {
        final volume = double.tryParse(volumeLevel) ?? 0;
        // Drive waveform UI when volume > 0.15
        debugPrint('volume: $volume');
      },
      onEngineError: (msg) {
        needsEngineReinit = true;
        // Common codes in JSON:
        // 5000 — app entered background during recognition
        // 4001 / 4000 / 4003 — recv timeout / connection issues
        try {
          final map = jsonDecode(msg as String) as Map<String, dynamic>;
          debugPrint('ASR err_code=${map['err_code']} msg=${map['err_msg']}');
        } catch (_) {
          debugPrint('ASR error: $msg');
        }
        _completeEngineStopWait();
      },
      onEngineStop: () {
        _completeEngineStopWait();
      },
    ),
  );

  final ok = await plugin.initASREngine(
    appId: appId,
    token: token,
    cluster: asrCluster,
    showNluPun: showNluPun,
  );
  needsEngineReinit = ok != true;
  return ok;
}

void _absorbText(String text, {required bool isFinal}) {
  final normalized = text.replaceAll(RegExp(r'\s+'), '');
  if (normalized.isEmpty) return;

  if (isFinal) {
    if (sentences.isEmpty ||
        normalized.length >= sentences.length ||
        normalized.startsWith(sentences)) {
      sentences = normalized;
    } else if (!sentences.contains(normalized)) {
      sentences = '$sentences$normalized';
    }
  } else if (normalized.length >= sentences.length) {
    sentences = normalized;
  }
}
```

### 2. Press to start / release to finish or cancel

```dart
/// Finger down — start recognition.
Future<void> startHoldRecognize() async {
  // 1) Request microphone permission first (see below).
  // 2) Reuse engine unless needsEngineReinit.
  if (needsEngineReinit) {
    await initAsr();
  }

  sentences = '';
  final started = await plugin.startASREngine(
    resultType: 'full', // 'full' or 'single'
    vadSilenceTime: 10000, // leading silence (ms)
    vadEndSilenceTime: 2000, // trailing silence / auto-stop (ms)
    maxSpeechDuration: 600000, // max input duration (ms)
    enableGetVolume: true,
  );
  if (started != true) {
    await initAsr();
    final retry = await plugin.startASREngine(
      resultType: 'full',
      vadSilenceTime: 10000,
      vadEndSilenceTime: 2000,
      maxSpeechDuration: 600000,
    );
    if (retry != true) {
      throw StateError('ASR engine start failed');
    }
  }
}

/// Finger up — send. Wait for final text before reading [sentences].
Future<void> finishHoldRecognize() async {
  engineStopCompleter = Completer<void>();
  finalResultCompleter = Completer<void>();
  await plugin.stopASREngine();
  await Future.wait<void>([
    _waitEngineStopped(),
    _waitFinalResult(),
  ]);
  // Use [sentences] as the utterance result.
}

/// Finger up — cancel. Discard current text.
Future<void> cancelHoldRecognize() async {
  engineStopCompleter = Completer<void>();
  await plugin.stopASREngine();
  await _waitEngineStopped();
  sentences = '';
}

Future<void> disposeAsr() async {
  await plugin.unRegisterListener();
  await plugin.unInitASREngine();
}

void _completeEngineStopWait() {
  final c = engineStopCompleter;
  if (c == null || c.isCompleted) return;
  c.complete();
}

void _completeFinalWait() {
  final c = finalResultCompleter;
  if (c == null || c.isCompleted) return;
  c.complete();
}

Future<void> _waitEngineStopped({
  Duration timeout = const Duration(milliseconds: 1200),
}) async {
  final c = engineStopCompleter;
  if (c == null) return;
  try {
    await c.future.timeout(timeout);
  } catch (_) {}
  engineStopCompleter = null;
}

Future<void> _waitFinalResult({
  Duration timeout = const Duration(milliseconds: 1000),
}) async {
  final c = finalResultCompleter;
  if (c == null) return;
  try {
    await c.future.timeout(timeout);
  } catch (_) {
    // Timeout: fall back to last partial in [sentences].
  }
  finalResultCompleter = null;
}
```

### 3. Microphone permission

Request permission **before** `startASREngine`. Example with `permission_handler`:

```dart
import 'dart:io';
import 'package:permission_handler/permission_handler.dart';

Future<bool> ensureMicrophonePermission({
  Future<void> Function()? onNeedSettings,
}) async {
  final permission = Permission.microphone;
  if (await permission.request().isGranted) return true;

  final status = await permission.status;
  if (status.isGranted) return true;

  if (status.isDenied ||
      status.isPermanentlyDenied ||
      status.isRestricted) {
    await onNeedSettings?.call(); // show rationale / openAppSettings()
  }
  return false;
}
```

### 4. Background interrupt

Observe app lifecycle. If recognition is interrupted in background, mark the engine dirty and re-init on next start:

| `err_code` | Meaning |
| --- | --- |
| `5000` | Interrupted because app entered background |
| `4001` / `4000` / `4003` | Recv timeout / connection issues during resume |

```dart
class AsrLifecycleObserver with WidgetsBindingObserver {
  bool isResumed = true;

  @override
  void didChangeAppLifecycleState(AppLifecycleState state) {
    if (state == AppLifecycleState.paused) {
      isResumed = false;
      return;
    }
    if (state == AppLifecycleState.resumed) {
      isResumed = true;
      // Optional: auto-retry startHoldRecognize() for init/timeout interrupts.
    }
  }
}
```

### ASR tips

| Topic | Suggestion |
| --- | --- |
| Session queue | Serialize start / finish / cancel so stop does not race start. |
| Engine reuse | Reuse after stop; re-`initASREngine` only after failure or fatal error. |
| Final text | On finish, wait for `onEngineFinalResult` (with timeout), then fall back to last partial. |
| `resultType` | Use `'full'` for hold-to-talk continuous text; `'single'` for one-shot. |
| Microphone | Deny path must not call `startASREngine`. |

## Usage — ASR minimal one-shot

```dart
final plugin = VolcengineSpeechPlugin();

await plugin.registerListener(
  SpeechPluginListener(
    onEnginePartialResult: (r) => print('partial: $r'),
    onEngineFinalResult: (r) => print('final: $r'),
    onEngineError: (e) => print('error: $e'),
    onEngineStop: () => print('stop'),
  ),
);

await plugin.initASREngine(
  appId: appId,
  token: token,
  cluster: asrCluster,
);

await plugin.startASREngine(resultType: 'full');
// ... user speaks ...
await plugin.stopASREngine();
await plugin.unInitASREngine();
```

## Usage — TTS

```dart
await plugin.registerListener(
  SpeechPluginListener(
    onTTsSynthesisBegin: () => print('Synthesis started'),
    onTTsSynthesisEnd: () => print('Synthesis ended'),
    onTTsStartPlaying: () => print('Playback started'),
    onTTsFinishPlaying: () => print('Playback finished'),
    onTTsProgress: (progress) => print('Progress: $progress'),
  ),
);

await plugin.initTTSEngine(
  appId: appId,
  token: token,
  ttsCluster: ttsCluster,
);

// Single shot
await plugin.startTTSEngine(
  isNormal: true,
  ttsText: 'Hello, world!',
  voiceType: voiceType,
);

// Continuous synthesis
await plugin.startTTSEngine(
  isNormal: false,
  ttsText: '这是连续合成模式',
  voiceType: voiceType,
  ttsSpeed: 10,
  ttsVolume: 10,
);

await plugin.pausePlayback();
await plugin.resumePlayback();

await plugin.stopTTSEngine();
await plugin.unInitTTSEngine();
```

## Documentation

Official Volcengine speech API documentation: https://www.volcengine.com/docs/6561/79837

## License

MIT