Documentation

API

Classes

Stream Transcriber

undefined

Class: StreamTranscriber

Defined in: streamTranscriber.ts:12

Implements real-time transcription of an audio stream sourced from a WebAudio-compliant MediaStream object.

Read more about working with MediaStreams: https://developer.mozilla.org/en-US/docs/Web/API/MediaStream

Extends

Transcriber

Extended by

Constructors

new StreamTranscriber()

new StreamTranscriber(
   modelURL, 
   callbacks, 
   useVAD): StreamTranscriber

Defined in: streamTranscriber.ts:72

Creates a transcriber for transcribing a MediaStream from any source. After creating the StreamTranscriber, you must invoke StreamTranscriber.attachStream to provide a MediaStream that you want to transcribe.

Parameters

Parameter	Type	Default value	Description
`modelURL`	`string`	`undefined`	The URL that the underlying MoonshineModel weights should be loaded from, relative to Settings.BASE_ASSET_PATH.
`callbacks`	`Partial`<`TranscriberCallbacks`>	`{}`	A set of TranscriberCallbacks used to trigger behavior at different steps of the transcription lifecycle. For transcription-only use cases, you should define the TranscriberCallbacks yourself; when using the transcriber for voice control, you should create a VoiceController and pass it in.
`useVAD`	`boolean`	`false`	A boolean specifying whether or not to use Voice Activity Detection (VAD) on audio processed by the transcriber. When set to `true`, the transcriber will only process speech at the end of each chunk of voice activity.

Returns

StreamTranscriber

Example

This basic example demonstrates the use of the transcriber with custom callbacks:

import StreamTranscriber from "@usefulsensors/moonshine-js";

var transcriber = new StreamTranscriber(
     "model/tiny",
     {
         onModelLoadStarted() {
             console.log("onModelLoadStarted()");
         },
         onTranscribeStarted() {
             console.log("onTranscribeStarted()");
         },
         onTranscribeStopped() {
             console.log("onTranscribeStopped()");
         },
         onTranscriptionUpdated(text: string | undefined) {
             console.log(
                 "onTranscriptionUpdated(" + text + ")"
             );
         },
         onTranscriptionCommitted(text: string | undefined) {
             console.log(
                 "onTranscriptionCommitted(" + text + ")"
             );
         },
     }
);

// Get a MediaStream from somewhere (user mic, active tab, an <audio> element, WebRTC source, etc.)
...

transcriber.attachStream(stream);
transcriber.start();

Overrides

Transcriber.constructor

Properties

Property	Modifier	Type	Default value	Inherited from	Defined in
`audioContext`	`protected`	`AudioContext`	`undefined`	-	streamTranscriber.ts:14
`callbacks`	`public`	`TranscriberCallbacks`	`undefined`	`Transcriber.callbacks`	transcriber.ts:81
`isActive`	`public`	`boolean`	`false`	-	streamTranscriber.ts:18
`mediaRecorder`	`protected`	`MediaRecorder`	`undefined`	-	streamTranscriber.ts:13
`model`	`static`	`MoonshineModel`	`undefined`	`Transcriber.model`	transcriber.ts:80

Methods

attachStream()

attachStream(stream): void

Defined in: streamTranscriber.ts:116

Attaches a MediaStream to this StreamTranscriber for transcription. A MediaStream must be attached before starting transcription.

Parameters

Parameter	Type	Description
`stream`	`MediaStream`	A MediaStream to transcribe

Returns

void

detachStream()

detachStream(): void

Defined in: streamTranscriber.ts:129

Detaches the MediaStream used for transcription.

Returns

void

getAudioBuffer()

getAudioBuffer(): AudioBuffer

Defined in: streamTranscriber.ts:143

Returns the most recent AudioBuffer that was input to the underlying model for text generation. This is useful in cases where we want to double-check the audio being input to the model while debugging.

Returns

AudioBuffer

An AudioBuffer

loadModel()

loadModel(): Promise<void>

Defined in: transcriber.ts:93

Returns

Promise<void>

Inherited from

Transcriber.loadModel

start()

start(): Promise<void>

Defined in: streamTranscriber.ts:158

Starts transcription.

if useVAD === true: generate an updated transcription at the end of every chunk of detected voice activity. else if useVAD === false: generate an updated transcription every Settings.FRAME_SIZE milliseconds.

Transcription will stop when stop is called, or when Settings.MAX_RECORD_MS is passed (whichever comes first).

Note that the StreamTranscriber must have a MediaStream attached via StreamTranscriber.attachStream before starting transcription.

Returns

Promise<void>

Overrides

Transcriber.start

stop()

stop(): void

Defined in: streamTranscriber.ts:257

Stops transcription.

Returns

void

Overrides

Transcriber.stop

Moonshine Model Voice Controller