Docs/Voice Conversations

Voice Conversations

Push-to-talk NPC conversations with voice detection and automatic reply playback.

Voice runs over the same connection as text. The player talks, the SDK detects when they stop, the NPC replies out loud through the AudioSource that was added next to the component. You start and stop the conversation; everything in between is handled.

The conversation loop

Call ToggleConversation() (or StartConversation() / EndConversation() for explicit control) when the player engages the NPC. While the conversation is active the SDK listens to the microphone, uses voice activity detection to decide when the player has finished a sentence (a short stretch of silence ends the turn), sends the speech, and queues the spoken reply for playback. The reply text also arrives via OnNPCResponseReceived, and the player's own transcript via OnPlayerSpeechRecognized, so a subtitle log costs nothing extra.

VoiceInteraction.cs

using Mistscale.SDK;
using UnityEngine;

public class VoiceInteraction : MonoBehaviour
{
    [SerializeField] private MistscaleNPC mira;

    void Update()
    {
        // Talk to the NPC while looking at it and holding V
        if (Input.GetKeyDown(KeyCode.V) && PlayerIsFacing(mira))
            mira.ToggleConversation();
    }
}

Tuning voice detection

The voice component added next to MistscaleNPC exposes the detection knobs in the Inspector:

VoiceThreshold

How loud the microphone input must be to count as speech. Raise it in noisy environments, lower it for quiet talkers. Default 0.02.

SilenceDurationSecs

How long the player must pause before the turn is considered finished and sent. Default 1.5 seconds.

SampleRate

Microphone capture rate. The default of 16000 is what the service expects; change only with a reason.

Reacting to voice state in your UI

The voice component raises events for every state change, which is everything you need for talk indicators, subtitles, or an animated microphone meter:

OnListeningStarted

The microphone is live and waiting for the player.

OnListeningStopped

The microphone has been released.

OnVoiceStartedSpeaking

The player began talking. Good moment to show a recording indicator.

OnVoiceStoppedSpeaking

The player went quiet; the turn is about to be sent.

OnVoiceResponseReceived

A chunk of the NPC's spoken reply arrived. Playback is automatic.

OnVoiceResponseComplete

The NPC finished speaking. Re-enable input or return camera focus here.

GetCurrentAudioLevel()

Live microphone level, 0 to 1. Poll it to drive a mic meter.

Microphone permission

Desktop builds generally have microphone access, but mobile, WebGL, and console platforms each have their own permission flow. Request microphone access before the first conversation, and consider a text-only fallback when permission is denied; text and voice can be mixed freely in the same session.

Your First NPC

Spatial Context in Unity