@pilotoo/voice

Introduction

@pilotoo/voice adds speech to the chat: reading replies aloud, dictating a message, the voice preferences and the models they need. Engines are plugged in behind ports, so the same code runs with browser engines or native ones. @pilotoo/ui already depends on it; you install it directly to supply your own engines.

Installation

@pilotoo/voice is at version 0.1.0 and is not published to a registry. It is also not part of the tarballs that pnpm release:check packs, so today it is used from the kit workspace:

pnpm install
pnpm --filter @pilotoo/voice build

Inside the workspace, depend on it with "@pilotoo/voice": "workspace:^". It is bundled into @pilotoo/ui as a dependency.

Basic usage

Split a streamed reply into sentences that can be spoken as they arrive:

import { SentenceChunker } from '@pilotoo/voice';

const chunker = new SentenceChunker();
for (const delta of deltas) {
  for (const sentence of chunker.push(delta)) speak(sentence);
}

Load an engine only when voice is first used:

import { lazyVoiceBackend } from '@pilotoo/voice';

const backend = lazyVoiceBackend(() => createBackend(), { label: 'Loading…' });

Entry points

Import	Provides
@pilotoo/voice	SpeechReader, Dictation, SentenceChunker, lazyVoiceBackend, the VoiceBackend port and the preferences. No dependencies.
@pilotoo/voice/web	Browser engines: createWebPageVoice, WebAudioOutput, WorkerSpeechToText, WorkerTextToSpeech.

Peer dependencies

Only @pilotoo/voice/web uses them, and all three are optional:

• @ricky0123/vad-web 0.0.31
• onnxruntime-web ^1.30.0
• @huggingface/transformers ^4.3.1

Learn more

• Voice: setting it up, including the Vite configuration.
• @pilotoo/voice-native for on-device engines.