Voice AI has a grimy secret: most of it was by no means designed for dialog. The dominant paradigm — feed textual content in, get audio out — traces its lineage to audiobook narration and voiceover manufacturing, the place the mannequin by no means hears the particular person on the opposite finish. That’s high quality whenever you’re producing a podcast intro. It’s not high quality when a pissed off consumer is attempting to get assist from an AI agent at 11pm.
Inworld AI is looking that out straight with the launch of Realtime TTS-2, a brand new voice mannequin launched as a analysis preview by way of its Inworld API and Inworld Realtime API. The mannequin hears the complete audio of the alternate, picks up the consumer’s tone, pacing and emotional state, then takes voice path in plain English the way in which builders immediate an LLM.
What’s Truly Totally different Right here
The significant architectural distinction with TTS-2 is that it operates as a closed-loop system. The mannequin takes the precise audio of the prior turns of the alternate as enter, not only a transcript — it hears how the consumer truly sounded. That’s a non-trivial distinction. A transcript of “okay, high quality” provides you the phrases. The audio of “okay, high quality” tells you whether or not the particular person is relieved, resigned, or sarcastic. TTS-2 is designed to make use of that sign.
The identical line lands in another way after a joke than after unhealthy information, and the mannequin is aware of the distinction as a result of it heard the prior flip. Tone, pacing, and emotional state carry ahead robotically. Virtually talking, audio context flows throughout turns inside a Realtime session with out builders needing to cross express prior_audio fields or construct further plumbing.
4 Capabilities, One Mannequin
Inworld staff is delivery TTS-2 with 4 key options, positioning the mixture and never any particular person piece, because the differentiation.
- Voice Course: It lets builders steer supply utilizing plain-language prompts inline at inference time. As an alternative of choosing from a set emotion enum like
[sad]or[excited], builders cross a bracket tag like[speak sadly, as if something bad just happened]straight within the textual content. Lengthy, descriptive prompts beat quick labels — the mannequin responds much better to full context than single-word labels. Inline non-verbal markers like[laugh],[sigh],[breathe],[clear_throat], and[cough]could be dropped anyplace within the textual content the place the second ought to happen, and the mannequin locations them as audio occasions, not pronounced phrases. - Conversational Consciousness: It’s the closed-loop structure described above — the architectural shift that separates TTS-2 from prior-generation fashions that deal with every sentence as a stateless technology name.
- Crosslingual assist: One voice id is preserved throughout over 100 languages, together with mid-utterance language switches inside a single technology. No language flag is required — the mannequin handles transitions robotically, maintaining timbre, pitch, and character fixed throughout the swap. The highest-tier languages ship at native-speaker high quality, whereas the lengthy tail is described as launch-window experimental, in step with the mannequin releasing as a analysis preview.
- Superior Voice Design: It generates a saved voice from a written immediate and no reference audio required. Builders can describe an individual in prose, save the end result as a reusable voice, and name it like some other voice within the app. Voice Design ships with three stability modes: Expressive (for reside shopper dialog and companions), Balanced (the default for many agent workloads), and Steady (for IVR {and professional} deployments the place pitch drift is unacceptable).
The Conversational Layer Beneath
Past the 4 key options, it calls out a set of behaviors that push speech additional into what it describes as “particular person paying consideration” territory. Probably the most technically attention-grabbing is disfluencies: the mannequin generates pure uh and um, self-corrections, mid-noun-phrase pauses, and trailing ideas that sign heat and recall reasonably than malfunction. Critically, completely different speaker profiles cluster fillers in another way, and the mannequin follows the rhythm — filler-as-energy sounds completely different from filler-as-hesitation. Voice cloning can also be supported by way of a two-step API: add a reference pattern (5–15 seconds, clear, single speaker) to /voices/v1/voices:clone, get a voice ID, and use it like some other voice.
The place It Suits within the Stack
TTS-2 is one layer in Inworld’s broader Realtime API pipeline. The total stack contains Realtime STT, which transcribes and profiles the speaker in a single cross — capturing age, accent, pitch, vocal model, emotional tone, and pacing as structured alerts on the identical connection. A Realtime Router that routes throughout 200+ fashions, choosing the suitable mannequin and instruments based mostly on the consumer’s state and dialog context. And TTS-2 on the output layer. The pipeline runs over a single persistent WebSocket connection, with sub-200ms median time-to-first-audio for the TTS layer.
The Broader Context
Realtime TTS 1.5 already ranks #1 on the Artificial Analysis Speech Arena (as of Might 5, 2026), forward of Google (#2) and ElevenLabs (#3). The launch of TTS-2 alerts that Inworld considers uncooked audio high quality a solved drawback — and is now competing on the behavioral layer: context-awareness, steerability, and id consistency throughout languages.
Try the Docs and Technical details. Additionally, be at liberty to comply with us on Twitter and don’t overlook to affix our 130k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Must companion with us for selling your GitHub Repo OR Hugging Face Web page OR Product Launch OR Webinar and so forth.? Connect with us
