VibeVoice-ASR-Streaming: the VASR streaming+diarization model, planned as a metered API

What it is

VibeVoice-ASR-Streaming is Microsoft's streaming speech-to-text model, released on 3 September 2026 under the MIT license. Its model card describes it in one line. It is a unified streaming ASR model that transcribes who the speaker is and what they said, as the speech arrives, in 10 languages. Customized hotwords are supported too. It ships in two sizes, named 1.5B and 7B, and a technical report accompanies the weights.

The model belongs to a family, and the family split matters because the members get confused in search results:

The streaming model is the newest and the smallest in reach. It trades two things against its batch sibling: languages (10 against 50+) and accuracy. The report states the cost of streaming plainly. Word and character error rates rise by 0.75 to 3.53 points and speaker-attributed error by 5.13 to 6.67 points. That holds on every benchmark tested, relative to the non-streaming model. The released streaming checkpoints also target recordings up to eight minutes, not the batch model's hour.

Two naming traps are worth clearing up. First, the size names understate the models. The 1.5B checkpoint holds about 2.81 billion parameters and the 7B about 8.67 billion, per the Hugging Face metadata on the 7B card and its 1.5B sibling. Second, the repository is released as research code, and it says so plainly. The authors do not recommend using VibeVoice in commercial or real-world applications. Further testing and development come first. That is not a reason to walk away. It is the reason a hosting platform has to run its own measurement program before it sells the model. Which is the posture of this page.

This platform has built and measured a lane for the 1.5B checkpoint on real GPU hardware. The measurements are published in the benchmarks section below with their dates. The model itself is not in the platform's catalog today. The gateway route exists and answers with a model_not_found error for this model id until the catalog row is restored. The return is gated on three verifiable conditions. A checkpoint must emit per-speaker output fields rather than in-text markers. A correct-attribution pass must hold on a multi-speaker conversation the whole way through. And a session-length solution is needed for long streams. No date is attached to any of that on this page. What is published here is the model's capability record, this platform's own numbers, and the exact call shape as it is designed to work. The integration can be written before the listing returns.

Use cases

The speaker-label truth, before anything else

The claim that travels with this model is who-said-what transcription in one pass. There is no separate diarization stage bolted on. The design is real and it is the model's main idea. Instead of transcribing first and assigning speakers after, the model interleaves labeled text with the audio chunks. It commits a label about 1.5 to 2 seconds after the speech. The report contrasts this with Google Cloud Speech-to-Text, which revises speaker labels retroactively. The labels it emits first carry 27.0 to 32.2 points more error than its revised ones. So a live consumer either waits for the revision or eats the gap. This model commits early and does not revise.

Now the part every builder needs to know and no model card states. On the wire, the labels are plain text inside the transcript. A chunk comes back with a line reading Speaker 0: followed by the words. There is no structured per-speaker field, and no timestamps are emitted. If your pipeline wants speaker as data rather than as text, you are parsing the text or adding a pass.

Then there is this platform's own finding, stated plainly because it is the reason this model is not listed. In this platform's testing of the 1.5B checkpoint, the in-text speaker markers were misattributed. The test audio was a three-speaker conversation. The transcript named the wrong speaker where the ground truth said otherwise. The event stream itself also carried a single constant label rather than per-speaker fields. Until a checkpoint clears the bar above, plan around streaming text without speaker attribution through this route. What was said, live: yes, with the caveats in the benchmarks section. Who said it: not promised.

With that settled, the fits:

• Live agents and voice assistants. The report's own motivation: existing unified models mainly support offline recognition, which cannot meet the latency needs of a real-time assistant. Chunked reading with a half second of lookahead is built for this, and hotwords let you seed the vocabulary with the names your assistant will actually hear.
• Live captions in ten languages. Meetings, calls, and streams in Chinese, English, French, German, Italian, Japanese, Korean, Portuguese, Russian, or Spanish, with text landing per chunk while the speaker talks.
• Domain vocabulary that generic ASR mangles. Hotwords are a first-class input, set once and effective for the session. Community testing of the family on long Chinese audio found name errors largely resolved by supplying the names as hints; the mechanism is officially documented, not folklore.
• Transcription pipelines moving off batch. If you currently batch your audio and want incremental text during capture, this is the family's answer, and the report's accuracy table tells you exactly what the move costs.

Where not to use it, from the report's own limitations and the platform's measurements. First, long meetings with heavy overlapped speech. The report says performance degrades because the output is a single serialized stream. Second, anything past eight minutes per recording, which the released checkpoints do not target; use the batch sibling or chunk. Third, clean single-speaker audio where the last fraction of a point matters. On LibriSpeech test-clean the streaming 7B measured 2.33 percent error against Voxtral-Mini-4B-Realtime at 1.88. Fourth, any language outside the ten, which is the batch model's territory.

API usage

This model takes a WebSocket streaming face, not a chat-completions shape and not a file-upload batch shape. The route is GET /v1/audio/transcriptions/stream on https://api.ironstratum.com/v1: a plain HTTP request that upgrades to a WebSocket. The bearer key check and the wallet balance check run on the upgrade request itself, before any socket opens. So a bad key answers 401, a revoked key 403, and an empty wallet 402 as ordinary HTTP responses.

The frame law, exactly as the gateway enforces it:

1. First frame out is the config, as JSON text. The accepted set at this face is model, format, and optionally sample_rate and channels. The model id is the platform alias stt-stream-vibevoice. Format must be pcm_s16le and the audio is 16 kHz mono; sample_rate and channels default to 16000 and 1, so sending them is optional but sending anything else is a loud refusal, never a silent resample.
2. Then binary frames of PCM audio, sent as you have them. Raw 16 kHz 16-bit mono samples, 32,000 bytes per second. Frames can be any size up to one mebibyte each; the lane buffers to its own chunk geometry, which it reads from the checkpoint rather than assuming.
3. Text frames come back as events: transcription.partial while a chunk is still settling and transcription.final when it lands, each carrying the text and the final carrying its duration. Partials flow from the first chunk.
4. End of audio is a half-close. Send your close frame, then keep reading. The gateway converts your close into a flush marker on the model side. It drains every remaining event to you, and only then closes from its side. If you close and stop reading, you lose the tail.
5. Failures are one error frame, then close: {"error":{"code":"...","message":"...","retryable":false}} for request problems, retryable: true for a mid-stream failure on the model side. A mid-session failure is terminal. A live stream is not replayable, so there is no idempotent retry here. Retry means a new session.

curl

curl cannot hold a WebSocket session, and pretending otherwise would waste your time. What curl does show is the HTTP face underneath the upgrade, which is where auth and wallet problems surface:

curl -i "https://api.ironstratum.com/v1/audio/transcriptions/stream"

That returns a plain 401 with the standard error envelope, because the bearer key is missing. With a key but an empty wallet it returns 402. Those statuses never open a socket.

The one-line tool for a real session is not available in a shell, and here is why. This face requires a text config frame followed by binary audio frames, and websocat applies one frame type to the whole pipe. With -b the config frame goes out as binary and the face refuses it as invalid_json. Without -b the audio goes out as text and the gateway drops it. The python tab below is the minimal full client.

What a shell tool can carry is the dial check: one text-mode websocat call carrying only the config frame. That one call proves the route, the key, and the config validation. Today it answers with the model_not_found error frame, which is exactly the dropped-row posture. When the row is restored, the same check opens and closes a clean empty session:

echo '{"model":"stt-stream-vibevoice","format":"pcm_s16le"}' | \
  websocat "wss://api.ironstratum.com/v1/audio/transcriptions/stream" \
  -H "Authorization: Bearer $KEY"

python

The canonical client, using the websockets package (14.x or newer) and a WAV file as the audio source. In a live application you would run the reader loop as a task alongside the sender instead of draining at the end. That way partials render while audio still flows:

import asyncio
import json
import os
import wave

import websockets

URL = "wss://api.ironstratum.com/v1/audio/transcriptions/stream"
CHUNK = 3200  # bytes: 100 ms of 16 kHz 16-bit mono PCM

async def main():
    headers = {"Authorization": "Bearer " + os.environ["KEY"]}
    async with websockets.connect(URL, additional_headers=headers) as ws:
        await ws.send(json.dumps({
            "model": "stt-stream-vibevoice",
            "format": "pcm_s16le",
            "sample_rate": 16000,
            "channels": 1,
        }))

        with wave.open("meeting.wav", "rb") as wav:
            assert wav.getframerate() == 16000 and wav.getnchannels() == 1
            while True:
                pcm = wav.readframes(CHUNK // 2)
                if not pcm:
                    break
                await ws.send(pcm)

        # End of audio is the half-close: send the close frame, then
        # keep reading until the server closes from its side. Events
        # that arrive during the close handshake are buffered and
        # delivered before the connection-closed exception.
        await ws.close()
        try:
            async for message in ws:
                print(message)
        except websockets.ConnectionClosed:
            pass

asyncio.run(main())

openai-sdk

There is no honest OpenAI SDK snippet for this route, and this tab will not fake one. The SDK's transcription method, client.audio.transcriptions.create, is a REST multipart upload for finished files. Its realtime client, client.realtime.connect, speaks OpenAI's own Realtime protocol and is built for OpenAI's Realtime service. A face that speaks a different WebSocket protocol, like this one, is not reachable through it. So no SDK call demonstrates this route. Changing a base URL migrates the SDK to REST-shaped endpoints such as the platform's batch transcription models, not to this one.

What an SDK migration grafts in instead is a small dial function next to the existing client. This is the minimal session, config frame to first event:

import asyncio
import json
import os

import websockets

async def dial():
    ws = await websockets.connect(
        "wss://api.ironstratum.com/v1/audio/transcriptions/stream",
        additional_headers={"Authorization": "Bearer " + os.environ["KEY"]},
    )
    await ws.send(json.dumps({
        "model": "stt-stream-vibevoice",
        "format": "pcm_s16le",
        # sample_rate and channels default to 16000 and 1
    }))
    first = await ws.recv()
    print(first)
    await ws.close()

asyncio.run(dial())

The failure modes to tell apart before you add reconnect logic:

Billing follows the audio, not your wall clock. The unit is the audio second. It is the sum of the durations that final events carry when any final carried one, else the stream wall clock. A client that disconnects after finals were delivered is billed for them, because the transcription was delivered. A stream that dies on the provider side mid-session is not billed, because finals from a dead stream may be incomplete. Refusals at the door bill nothing. This face accepts the Idempotency-Key header and logs it, with no replay semantics. A live socket cannot be replayed, so a retrying client opens a new session and bills anew. Build that into any automatic reconnect logic before it bills you twice.

Benchmarks

Two layers of evidence follow. One is the model's authors' numbers from the technical report (revision of 10 September 2026). The other is this platform's own hosted-lane measurements. The two layers answer different questions, and each carries its date.

The design numbers first. The model reads chunks of 15 or 22 latent frames, 2.0 or 2.9 seconds of audio, with a fixed 4-frame (half second) lookahead:

Decoding one chunk took 146 to 208 ms on a single A100 80GB under vLLM in bf16 (7B model, 15-frame chunks), a real-time factor at or below 0.104.

Speaker-attributed accuracy, five-set mean, lower is better. The numbers come from the report's comparison of streaming systems, all measured by the authors under one protocol:

The 7B model is best on AISHELL-4 (22.76), AliMeeting, and AMI-IHM (19.83). Gemini 3.5 Transcribe Live leads AMI-SDM there (27.18 against 29.81). On speaker attribution specifically (cpWER/cpCER), the report has it best or tied-best on 12 of 13 settings. The set includes AISHELL-4 at 28.70, AliMeeting 39.80, AMI-IHM 27.48, AMI-SDM 39.01, and the MLC-Challenge average 22.75. That is ahead of Microsoft's own Azure ConversationTranscriber by 2.39 to 12.45 points on the meeting sets. One result the report includes and most summaries drop sits on clean single-speaker speech (LibriSpeech test-clean). There the streaming 7B measured 2.33 against Voxtral-Mini-4B-Realtime at 1.88, with Nemotron-3.5-ASR 0.6B at 3.02 and X-ASR at 2.96. This model's edge is conversations, not clean read speech.

This platform's own measurements, taken on the hosted lane (an RTX 3060-class GPU with 12 GB, the 1.5B checkpoint in bf16, task-19 measurement series):

What those numbers do and do not prove. The hosted-lane figures show the pipeline works and meters exactly, on hardware one size class down from the report's A100. They are not a production latency guarantee. The 220-second cap is the headline. The platform's edge resets any WebSocket connection at about 220 seconds of wall time, regardless of traffic. That is why a session-length solution is one of the three listing conditions above. The 60-minute figure predates the cap and bypassed the gateway entirely. That is why its row carries that qualifier. Long sessions through the platform need either a model-side fix or chunked sessions stitched client-side. Until one of those exists, plan your session lengths accordingly.

Getting started

The sequence for a model in this posture has two tracks.

If you are evaluating the model today, read the model card and the technical report. Try the official demo, and run the vLLM serving recipe on your own GPU if you have one. Write your integration against the frame law above. It is stable design, and when the row is restored your client works against this face by changing nothing.

When the listing lands, sign up on the console (it runs as an invite-only beta in this phase). Create an API key, fund the prepaid wallet, and run the python tab. Spend stops when the wallet hits zero, and that cap applies to every key on the account. Any single key can be revoked on the spot, and the others keep working. Rates are not written on this page. The pricing page carries them, drawn from the catalog the meter reads. To situate this model among the other speech models, see the speech to text category. The models index lists the whole catalog.

Last verified: 2026-09-25