hayamimi (早耳) STT: real-time multilingual speech to text

What it is

hayamimi (早耳, Japanese for "quick ear") is an open-source project for real-time, multilingual speech to text that runs on CPU hardware only. No GPU, no PyTorch, no CUDA, under 2 GB of RAM. It is published by its author under the GitHub handle oboroge0 (hayamimi repository). The design goal fits in one line. Draft subtitle text appears while you are still talking. The finalized line lands roughly 100 ms after you stop.

If you searched the bare name and arrived amid Gundam plastic-model news, you have met the name collision. A long-running Japanese gunpla tracking site and its hobby accounts own most of the bare-name results. An Instagram account and a voice actress own two more rows. And "haya mimi" is also a common Swahili phrase. None of those have anything to do with speech recognition. Adding "stt" or "speech recognition api" to the query is what surfaces the model today. This page is about the model.

The engineering idea is routing. The usual CPU setup falls back to one general-purpose model (Whisper) and accepts its accuracy ceiling and its speed. hayamimi instead picks a specialist model per utterance:

Every model runs as an INT8-quantized ONNX file through the sherpa-onnx runtime. A whisper-tiny spoken-language identifier plus character-set arbitration decides the route per utterance. An LRU cache keeps resident models inside the memory cap. That matters when a session wanders across languages. On the project's own scorecard, the Japanese route reaches 3.8% character error rate on real TV broadcast audio (remeasured 2026-09-01). Every route holds a mean real-time factor between 0.043 and 0.102 on a 6-core desktop CPU. That is comfortably faster than realtime (scorecard).

Japanese output gets its punctuation restored by a small BERT model. The same author publishes a mobile-grade fp16 ONNX build of that punctuation model separately (hayamimi-punct-ja-fp16 on Hugging Face). That is a fair signal of where the project is heading. The v0.4.0 release ships an embedding API and a Flutter package for on-device use.

The project is young and moving fast, which is worth knowing before you build on it. Created on 2026-08-25, the repository shipped seven releases in its first twelve days, the latest on 2026-09-06. It holds 337 stars at this writing. It was last pushed on 2026-09-10, the day before the final verification pass for this page. Source code carries an MIT license. Model weights are fetched from their original publishers at install time, under their own licenses. Development and testing happen on Windows 11. The runtimes are cross-platform and macOS and Linux are expected to work. But the author states plainly that they are not yet covered by end-to-end CI runs.

Running it yourself is a real option and the repo makes that straightforward. You need Python 3.10 or newer, ffmpeg on PATH, and a one-time model download of about 3.1 GB. A Japanese-and-English-only minimal install is about 1.1 GB. What you take on with the local route is the upkeep. You own the process, the model storage, the machine it runs on, and every update. The hosted route on this platform trades that for a metered bill. The model is served, patched, and metered per audio second behind one endpoint.

This platform serves the model under the alias hayamimi-stt as a metered transcription API: one endpoint, batch and streaming shapes. The route answers calls today, metered per audio second. The hosted contract covers the 29 dedicated language codes. They span Japanese, Mandarin, Korean, Cantonese, English, and 24 European languages. The about-1600-language Omnilingual fallback belongs to the local install, not to the hosted endpoints.

Use cases

Live captions where the machine has no GPU. Meetings, streams, and broadcast monitoring on ordinary laptops or desktops. The local application ships an OBS browser-source overlay and a dashboard. The hosted streaming face delivers the same partial/final event stream over SSE. Your own caption UI can consume it directly. Japanese is the strongest route and the best-measured one. For readable Japanese output specifically, punctuation restoration matters as much as raw accuracy. Subtitles and meeting minutes without 、。 are hard to scan. That step is built into the pipeline rather than left to a post-processor you write.

Multilingual sessions where speakers alternate languages. An interpreter switching between Japanese and English sentence by sentence is the stress case. Per-utterance routing transcribes both sides correctly. The session sticks to a confirmed language until a new one is detected consistently. That trades a short startup delay for stability. The stability holds against jingles and background music that fool weaker detectors. Mid-sentence code-switching does not work. The router picks one language per utterance. A sentence that mixes two languages inside itself will mangle the minority portion. The project's own limitations list says so plainly. Our position is the same.

Batch transcription of recorded clips. WAV, FLAC, and Ogg Opus uploads, up to 25 MiB and roughly 240 seconds per request. Client-side chunking handles longer recordings. Useful for captioning archives, voicemail backlogs, and podcast episodes cut into segments. The batch face is the codec-tolerant one. It decodes the three containers above. The streaming face is strict about raw 16 kHz mono PCM or WAV.

Voice apps that need draft text fast. Our hosted measurements (below) put the first partial at 342 ms after audio start when audio is pushed ahead of realtime. Partials then land about every 229 ms. That is fast enough for reactive UI: command hints, live search-as-you-speak, assistive captioning. The project's design target was a finalized line within about a second of the speaker stopping. The shipped project reports roughly 100 ms on Japanese with punctuation included.

Transcripts that improve after the fact. The local pipeline has a two-pass refine step. After two seconds of silence it batch re-decodes recent utterances for a cleaner transcript. The project measured that improving real-broadcast Japanese from 15.5% to 12.0% CER. That was before the 2026-09-01 remeasure that took the headline figure to 3.8%. On the hosted streaming face the same lineage shows up as the refine event. It re-emits a finalized segment with the auto-detected language attached. Treat finals as the transcript and refine events as the polished layer.

Where not to use it. Three honest cases. First, if your audio is English, Mandarin, or Korean, one honest case applies. If turbo-class Whisper accuracy on that specific language matters more to you than CPU realtime, the project's own comparison data says turbo keeps a small edge on those three. Second, overlapping speakers: the model labels speaker turns. Its second pass cut diarization error from 25.7% to 13.9% on five AMI meetings. But it does not separate two people talking at once, and its speaker count is an overestimate. Read the labels as turn markers rather than a headcount. Third, translation. The local app can translate Japanese lines live, but the hosted API returns transcription only.

API usage

Endpoint: POST /v1/audio/transcriptions on https://api.ironstratum.com/v1, authenticated with a bearer API key from the console. The model id is hayamimi-stt. On the batch face, language takes one of the 29 dedicated codes. A missing or non-dedicated value reaches the model's default path and returns a terminal 502. Pick a dedicated code and do not retry the same value after that error. Billing is per audio second, taken from the duration the response reports.

Streaming or batch

The category search keeps asking one question, realtime voice bot or recorded files. The answer picks the face. Both live on the same endpoint:

• Stream when the audio is happening now. Live captions, turn detection in a voice app, search-as-you-speak: the value is draft text while the speaker is still talking. Partials arrive as the audio streams, finals confirm each line, refine events polish it, and the contract is strict on purpose, raw 16 kHz 16-bit mono PCM or WAV only, because the stream path does no container decoding.
• Batch when the audio already exists on disk. Recorded meetings, voicemail backlogs, podcast segments: upload the file, get one JSON answer back. This is the codec-tolerant face, WAV, FLAC, and Ogg Opus, and the simpler client to write, bounded at about 240 seconds per request with client-side chunking beyond that.

Both faces bill per audio second rather than per wall-clock minute of waiting. Both answer to the same alias and key.

curl

Batch, the common case:

curl https://api.ironstratum.com/v1/audio/transcriptions \
  -H "Authorization: Bearer $KEY" \
  -F model=hayamimi-stt \
  -F language=ja \
  -F file=@meeting.wav

The response is JSON with the transcript, segment list, and duration:

{
  "text": "本日の会議を始めます。",
  "segments": [],
  "duration": 63.4
}

Streaming adds stream=true and switches the response to server-sent events. Use --no-buffer to see events as they arrive:

curl --no-buffer https://api.ironstratum.com/v1/audio/transcriptions \
  -H "Authorization: Bearer $KEY" \
  -F model=hayamimi-stt \
  -F stream=true \
  -F file=@clip.wav

data: {"type":"transcription.partial","text":"本日の会議を"}
data: {"type":"transcription.final","text":"本日の会議を始めます。"}
data: {"type":"transcription.refine","text":"本日の会議を始めます。","lang":"ja"}
data: [DONE]

The stream face auto-detects language and reports the detected code on the refine event. It accepts raw 16 kHz 16-bit mono PCM or WAV only. Anything else is a 400 before the model is reached. The batch face is codec-agnostic within its decode set (WAV, FLAC, Ogg Opus).

python

Batch with requests:

import os
import requests

resp = requests.post(
    "https://api.ironstratum.com/v1/audio/transcriptions",
    headers={"Authorization": f"Bearer {os.environ['KEY']}"},
    files={"file": open("meeting.wav", "rb")},
    data={"model": "hayamimi-stt", "language": "ja"},
    timeout=300,
)
print(resp.json()["text"])

Consuming the SSE stream line by line:

import os
import requests

with requests.post(
    "https://api.ironstratum.com/v1/audio/transcriptions",
    headers={"Authorization": f"Bearer {os.environ['KEY']}"},
    files={"file": open("clip.wav", "rb")},
    data={"model": "hayamimi-stt", "stream": "true"},
    stream=True,
    timeout=300,
) as resp:
    for line in resp.iter_lines():
        if line.startswith(b"data: "):
            print(line[6:].decode())

openai-sdk

The endpoint follows the OpenAI audio-transcription shape. The standard client works with a base URL change:

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["KEY"],
    base_url="https://api.ironstratum.com/v1",
)

with open("meeting.wav", "rb") as f:
    transcript = client.audio.transcriptions.create(
        model="hayamimi-stt",
        file=f,
        language="ja",
    )
print(transcript.text)

One shape note for both faces. The effective ceiling is about 240 seconds of audio per request. Chunk longer recordings into separate requests and join the transcripts client side.

Errors worth telling apart before you set a retry policy:

Metering follows the audio, not the wall clock you experience. The batch face bills the ceiling of the duration the response reports. Wall clock is the documented fallback when a container omits duration. The stream face sums the durations carried by final events, with the uploaded PCM byte count as its fallback. A slow network upload does not inflate the bill.

Both faces accept an Idempotency-Key header. On this multipart route it carries no replay semantics. A retried request executes and bills again. Keep that in mind when you add automatic retries around flaky networks. Chunk boundaries and deduplication are the client's job here.

Benchmarks

Two measurement layers matter here: the project's own published numbers, and ours from hosting the model. They measure different things. Both are shown with their dates and conditions.

Project scorecard, on real speech, per the project's evaluation docs. It covers language identification, routing, decode, and Japanese punctuation:

Japanese was remeasured on 2026-09-01 after a head-dropout fix and CJK number normalization landed. Earlier third-party write-ups still quote the older 5.8% figure. The Mandarin number includes about 1.3 points of numeral-notation mismatch. The pipeline writes arabic digits where some references spell them in kanji. That is a scoring convention, not a recognition error (scorecard).

Against Whisper on the same clips, same scoring, same CPU (a Ryzen 5 5600), the project measured itself against whisper-large-v3-turbo through faster-whisper INT8. Turbo was given the correct language while hayamimi detected it automatically. hayamimi ran 11 to 19 times faster on every language. Turbo did not reach realtime on any (RTF above 1 across the board). Accuracy split: Japanese 7.5% versus 13.75% CER and Cantonese 6.1% versus 15.5% both favor hayamimi by roughly half. English, Mandarin, and Korean sit 0.1 to 1.3 points in turbo's favor (comparison document, dated 2026-08-27).

Where it wins and where it trades

Read as a language table, the same project comparison is the honest coverage answer:

Japanese and Cantonese are the wins, at roughly half turbo's error rate. Japanese is the route the project is built around. ReazonSpeech (Zipformer) runs under the hood, scored on real TV broadcast audio rather than read speech. English, Mandarin, and Korean are the tradeoffs. A turbo-class Whisper keeps a small edge there if one language's accuracy is the whole requirement. The source for every number is the project's comparison doc named above, dated 2026-08-27. The conditions and the speed result sit in the paragraph before the table.

Our hosted measurements, from the launch catalog records:

Read those honestly: our reference set is small and clean. A 0.0000 round-trip proves the hosted pipeline is exact on easy audio. It does not prove it beats the 3.8% broadcast number. The broadcast figure is the accuracy basis for real-world Japanese. Ours are the operational numbers for what a hosted call feels like.

Two more project numbers complete the latency picture. Across a five-language soak test with every feature enabled, final latency averaged 236 ms with a 552 ms worst case. The roughly 100 ms figure is Japanese-only. And the whole stack keeps resident memory under 2 GB by default. That constraint is what makes it deployable on small machines in the first place. Those are the project's own measurements from its README and goals document, not ours.

Getting started

Sign up on the console and create an API key on the keys page. Then fund the prepaid wallet. The wallet balance is what bounds spend on every key. Then run the curl tab above with your key. That first call is the whole setup: no Python environment, no ffmpeg, no model download.

For cost, see pricing. Every rate there renders from the same catalog the gateway meters against. To compare this model against the other speech models, see the speech to text category. For the full model list, see all models.

Last verified: 2026-09-25