Hayamimi Japanese speech-to-text: the Whisper duel on Japanese audio

For Japanese audio, the general-purpose default loses to a specialist. On real TV broadcast Japanese, the hayamimi project scores 3.8% character error rate. Whisper-large-v3-turbo reads 13.8% on the same clips. It runs 11 to 19 times faster on a plain CPU. This page lays out that duel with its dates. It shows where Whisper still wins. It walks the fine-tune lane between them.

The name comes first. Hayamimi writes itself 早耳, Japanese for "quick ear". It is a young open-source project, first published on 2026-08-25. It runs real-time speech to text on CPU hardware only. No GPU, no PyTorch, under 2 GB of RAM. Whisper needs less introduction. It is the OpenAI family most of the market defaults to. Its fastest hosted cut is whisper-large-v3-turbo, or turbo here. Your audio is Japanese? This page is the comparison the search results never gave you. Which model, on whose numbers, under what conditions.

What wins on Japanese audio, in three sentences

One metric note before the numbers. Japanese scores here are character error rate, written CER. The next section explains why.

Hayamimi wins Japanese on both axes the duel can measure. Its Japanese route reaches 3.8% CER on real TV broadcast audio, remeasured 2026-09-01. Turbo reads 13.8% on the same clips, per the project's own scorecard. On speed, hayamimi decodes that audio 11 to 19 times faster than turbo on the same desktop CPU. A final Japanese line lands about 100 ms after you stop talking.

Whisper still wins three things, and they are real. Breadth: turbo lists 99 languages. The hosted hayamimi route carries 29 dedicated codes. Ecosystem: a large toolbox of runtimes, wrappers, and fine-tunes surrounds the Whisper family. Translation: turbo was not trained for it. Its own repository says so. The wider Whisper family was. A Japanese-to-English translation job still runs on large or medium Whisper. Hayamimi's local app also translates Japanese live. The quality limits are documented plainly by its author.

The rest of this page is the evidence for those two paragraphs.

Why Japanese punishes general-purpose models

Three facts about the language decide this comparison before any benchmark does.

First, Japanese has no spaces. Word error rate, the standard ASR metric, is close to meaningless for it. Read CER instead. The gap is not subtle. The Japanese fine-tuned kotoba-whisper v2.0 reports both metrics on the same test sets. Its model card shows word error rates from 55.6% to 63.7% beside CERs from 8.4% to 11.6%. A Japanese model is not broken because its WER reads 60%. A WER number quoted alone, for Japanese, is a number to ignore.

Second, raw decoder output arrives unpunctuated. Usable Japanese text needs the comma 、 and the full stop 。. Subtitles without them are hard to scan. Hayamimi restores Japanese punctuation inline. A small BERT model does it inside its own pipeline. On the Whisper side, that work is left to surrounding tooling. That is one reason the subtitling scene around Whisper grew so many helpers.

Third, the audio domain swings results. Japanese test sets range from clean read-aloud speech to messy TV broadcasts. The messy end brings jingles, overlaps, and captions. OpenAI's release notes for turbo state it performs better on FLEURS. FLEURS is a set of cleaner recordings. The same notes name larger degradation on some languages. A model can look strong on clean read speech. The same model can sag on broadcast Japanese. That domain gap is where the specialist route below earns its numbers.

The direct duel, with dates and conditions

The hayamimi comparison document, dated 2026-08-27, runs the direct fight. Same clips, same scoring function, same CPU: a Ryzen 5 5600 desktop. Turbo ran through faster-whisper at INT8. It was handed the correct language, a small advantage. Hayamimi detected the language itself.

Three reads. Speed: hayamimi is 11 to 19 times faster on every language. Turbo never reaches realtime on this CPU. Its real-time factor sits above 1 across the board. Accuracy: Japanese and Cantonese favor hayamimi by roughly half the error. Turbo keeps a 0.1 to 1.3 point edge on English, Mandarin, and Korean. Scope: the Japanese row predates a 2026-09-01 fix, described next. Treat it as the conservative read.

A caution that applies to every number on this page, both projects included. These are the model authors' own published measurements. They carry dates and documented conditions, and that is what makes them usable. They are still self-reported. Run your own clips before you commit a pipeline. Both hosted lanes here make that test a one-line call. It is shown near the end.

The broadcast number and its fine print

The headline Japanese figure, 3.8% CER, comes from the scorecard linked above. Its context matters. The test set is the ReazonSpeech test split. That means real Japanese TV broadcast audio, not read-aloud speech. The score was remeasured on 2026-09-01 after two pipeline fixes. One repaired a bug that dropped the opening of an utterance. The other normalized Japanese numbers. Before those fixes the figure was 5.8%. Older write-ups still quote 5.8% today. See it cited anywhere? That is a snapshot from before the remeasure.

The same honesty cuts the other way. Hayamimi's Japanese number is a broadcast test. Its English number comes from audiobook narration. Its Mandarin, Korean, and Cantonese numbers come from read speech. Different domains, different difficulty. The scorecard says so itself. That is why its table is trustworthy enough to quote.

One more number for scale. A five-language soak test ran with every feature on. It measured final lines at 236 ms on average after speech ended. The worst case was 552 ms. The roughly 100 ms figure is the Japanese-only, tuned-path number. Both come from the project's goals document, not from us.

Where Whisper still wins

Breadth. The turbo model card lists 99 language codes, Japanese among them. The hosted hayamimi route carries 29. Does your archive span 70 languages no hayamimi route covers? Then breadth decides before accuracy does.

Throughput on a GPU. Turbo is the pruned cut of large-v3. The decoder dropped from 32 layers to 4. Parameters dropped from 1.55 billion to 809 million. OpenAI's release post puts its speed above what the tiny model managed a generation back. On GPU hardware it chews through batch work. The card is equally plain the other way. The family "cannot be used for real-time transcription out of the box". On the duel's desktop CPU it did not reach realtime. GPU batch: Whisper. CPU realtime: hayamimi.

Ecosystem. The faster-whisper port runs "up to 4 times faster" than the reference code. Same accuracy, with INT8 support on CPU. Wrappers for timestamps, alignment, and subtitling fill the space around it. The Japanese corner of that ecosystem is thinner than it looks. N46Whisper is the Colab notebook Japanese subtitlers passed around for years. Its repository now carries a "no longer maintained" notice, read 2026-10-28.

Honesty notes from the card itself. The Whisper limitations section is blunt here. Predictions "may include texts that are not actually spoken". It states the model is prone to repetition. Beam search and temperature scheduling mitigate this. They do not eliminate it. Plan for silent-audio hallucination and repetition loops in any Japanese deployment. Hayamimi's documented weak spots differ. It does not separate two overlapping speakers. It mangles mid-sentence code-switching. Its Japanese-to-Chinese or Korean translations can garble numbers.

The Japanese fine-tune lane, and why it converges

The query that brings many readers here is the "whisper japanese finetune" tail. Can Whisper itself be fixed for Japanese? Yes, partly. The fix tells the whole story of this page.

The best-known answer is kotoba-whisper. It distills whisper-large-v3, with the decoder cut to two layers. Its v2.0 release trained on ReazonSpeech. Its card describes that corpus as the largest Japanese speech-transcription dataset in existence. The source is Japanese TV audio. It is 6.3 times faster than its teacher. It beats large-v3 on the ReazonSpeech held-out set. The CERs read 11.6% there and 9.2% on Common Voice 8 Japanese.

Notice the convergence. The strongest Whisper-based Japanese move trains on real broadcast audio. That is the corpus hayamimi's Japanese route is built from. Hayamimi's route is a ReazonSpeech-trained Zipformer. It is a 159M-parameter, character-based recognizer from the Reazon team, Apache 2.0. The model repository names it. Every serious road to Japanese accuracy passes through real Japanese broadcast audio. The specialist starts there. It skips the detour. No 809M-parameter weights to load. No 30-second windowing, which long-form Whisper requires.

For scale, one number. The original Whisper trained on 680,000 hours of multilingual audio, per the paper. That scale explains its breadth. It does not overturn the domain result above.

What each side costs to run yourself

Hayamimi local needs Python 3.10 or newer and ffmpeg on PATH. A one-time model download runs about 3.1 GB. A Japanese-and-English-only install is 1.1 GB. The project was built and tested on Windows 11. macOS and Linux are expected to work but lack automated test coverage there. Total memory stays under 2 GB by design. Electricity and an ordinary machine are the whole running cost.

Whisper local: turbo carries 809M parameters. It runs comfortably on modest GPUs. On CPU, INT8 runtimes cut memory and lift speed. The duel's numbers above are the honest CPU picture: below realtime. Running Whisper yourself, against a metered audio hour? Two guides cover the wallet math. The Whisper local vs API break-even guide. The cost per hour calculator.

Both models also sit behind one hosted endpoint.

Both models behind one endpoint

As of 2026-10-28, this platform serves both lanes on the same route. The hayamimi-stt model page carries the batch and streaming faces. It lists 29 language codes. The whisper model page carries turbo on the batch face. Both bill per audio second of the content itself. Never the wall clock. The wallet is prepaid. A refused request bills nothing.

A first Japanese call uses the batch face and the model id hayamimi-stt.

curl https://api.ironstratum.com/v1/audio/transcriptions \
  -H "Authorization: Bearer $KEY" \
  -F model=hayamimi-stt \
  -F language=ja \
  -F file=@meeting.wav

The response is JSON. It carries the transcript, a segment list, and the duration the billing uses.

{
  "text": "本日の会議を始めます。",
  "segments": [],
  "duration": 63.4
}

Contract facts that matter before the first call. The language field takes one of the 29 dedicated codes. A non-dedicated value is a terminal 502. Do not retry it unchanged. Uploads cap at 25 MiB and about 240 seconds per batch request. The streaming face accepts raw 16 kHz mono PCM or WAV only. It closes sessions at a 230-second cap with a named terminal event. It is the wrong face for anything that needs container decoding. The deep dive on chunking and joining is the long audio guide.

Rates are never typed into pages on this site. They render from the live catalog at build time.

The speech to text lane page lists every transcription model on the platform. Current rates sit on the pricing page. Sign up, create a key, and send one Japanese file to each lane. An hour of your own audio settles the accuracy question. Faster than any benchmark table, including this one.

Which model gets your Japanese audio

Two lanes, one endpoint, one wallet. The models index shows the full catalog. The decisive comparison is a test file, not a spec sheet.

Common questions

Is hayamimi better than Whisper for Japanese?

On the published numbers, yes, for Japanese. The direct duel ran the same clips on the same CPU. It puts hayamimi at 7.5% CER against turbo's 13.75%, dated 2026-08-27. The post-fix broadcast figure is 3.8% against 13.8%. Hayamimi is also 11 to 19 times faster on that CPU. The reverse holds for English, Mandarin, and Korean. There turbo keeps a small edge. All of it comes from the hayamimi project's own dated documents. Test your own audio.

Why do Japanese speech to text scores look so bad?

Because word error rate is the wrong ruler. Japanese has no spaces. One missed word boundary cascades the whole sentence's word matches. CER counts per character and is the metric Japanese results belong in. The kotoba-whisper card shows both on the same sets. WER ran 55.6% to 63.7%, beside CER at 8.4% to 11.6%.

Can Whisper be fine-tuned for Japanese?

Yes, and it has been. Kotoba-whisper v2.0 distills whisper-large-v3 and trains on ReazonSpeech, the large Japanese TV-audio corpus. It beats its teacher on the held-out broadcast set and runs 6.3 times faster. Note where its training data comes from: real Japanese broadcast audio. That is the same corpus family the hayamimi Japanese route is built from.

Does whisper-large-v3-turbo translate Japanese to English?

No. OpenAI's repository and release post state that turbo was fine-tuned on transcription data only. Translation data was excluded. Asked to translate, it returns the original language. Use the large or medium cuts for Japanese-to-English translation.

How fast is hayamimi on CPU?

Fast enough for realtime with room to spare. Real-time factors run 0.061 to 0.109 in the duel. A six-core desktop CPU decodes an hour of audio in roughly four to seven minutes. A finalized Japanese line lands about 100 ms after speech ends. Partial drafts update about twice a second while you talk.

What does it cost to run each model here?

Both lanes bill per audio second of the audio itself. The wallet is prepaid and stops hard at zero. A refused or malformed request bills nothing. Current rates render on the pricing page. The metering reads the same catalog. The numbers there are always the live ones.

The speech to text lane page lists both models. The pricing page carries the live rates. To settle the duel on your own audio, three steps. Sign up. Create a key. Send one Japanese file down each lane.

Last verified: 2026-10-28