Voice
Stream captured audio to an NPC and play back its spoken reply.
Voice runs over the same connection as text. The plugin gives you the wire methods — streaming audio chunks in, receiving synthesized speech back — but capturing the player's microphone and deciding when they've stopped talking is up to you, using Unreal's own audio capture systems.
AudioCapture / SubmixListener APIs are the usual approach) and pass the plugin raw PCM bytes.Sending audio
Stream chunks as you capture them, then send one final chunk with bEnd = true — or let the server auto-finalize once it has buffered about 5MB, whichever comes first:
// PcmChunk: TArray<uint8> of raw audio you've already captured
Connection->SendVoiceChunk(PcmChunk, false);
// ...more chunks as they're captured...
Connection->SendVoiceChunk(FinalChunk, true); // bEnd: trueDatabEndSenderIdOnce the server finishes transcribing, an OnTranscript delegate fires with what it heard, followed by the normal chat delegates — voice replies arrive as one complete OnChatChunk rather than streamed token by token, since grounding runs before anything is sent back.
Playing the reply
Synthesized speech arrives as raw binary WebSocket frames — no JSON envelope, delivered per sentence, in playback order:
Connection->OnAudio.AddDynamic(this, &AMyGameMode::OnAudio);
UFUNCTION()
void AMyGameMode::OnAudio(const TArray<uint8>& Bytes)
{
// Raw synthesized speech for one sentence. Decode/queue it with
// your own USoundWaveProcedural playback pipeline.
PlayAudioChunk(Bytes);
}