I Built a Cross-Platform Live Subtitle Tool

mimi started as a Mac app. It is now a cross-platform live subtitle tool. It listens directly to system audio, recognises Japanese, English, or Korean, and translates it into Simplified Chinese, English, or Japanese in an overlay above the original app.

I built it because Japanese videos without subtitles had always been hard for me to follow. As a teenager, I could read expressions and hear the tone; the rest was guesswork. The same problem later came up with dramas, games, and livestreams, so I turned the fix into a general-purpose tool.

The name comes from the Japanese word for “ear,” 耳 (みみ), romanised as mimi. There is no deeper meaning, and it is not anyone's name.

Two seconds late is wrong, even when the translation is right

The first version completed the whole route: it captured system audio, recognised Japanese, and returned Chinese. I was pleased for a moment.

Then I watched it for several minutes and could not stand it.

The voice finished while the subtitle was still on its way. The next line began as the previous translation finally appeared. A translation can be correct and still ruin the experience; two or three seconds of delay is enough to cut the viewing rhythm into pieces.

What mimi does is short: listen to system audio, recognise it, translate it, and place subtitles over the original app. The problem is that if every step waits for the previous one to finish completely, the whole chain is late. If mimi waits for a perfect sentence and a finished translation before showing anything, the subtitle can only chase the voice from behind.

The same sentence takes a draft route and a final route

I stopped waiting for one perfect answer and split each utterance into a draft and a final result. A draft appears quickly and can be replaced whenever the speech changes. A final result keeps the confirmed sentence, and its order cannot move.

Speech recognition keeps revising a draft. It may first hear 「今日は」 and then extend it to 「今日は帰らない」. If every draft translation runs to completion, the old request is already obsolete before its answer arrives, and the screen becomes a queue of expired subtitles.

mimi therefore keeps only the newest draft. When new speech arrives and the old translation is still running, the stale work gives way to the newer sentence.

Final lines do the opposite. If the next one appears before the previous sentence has finished, the conversation is reordered. Confirmed sentences therefore land one by one.

That is the awkward heart of live subtitles: fast results are allowed to be unstable; stable results have to stay in order. Force both jobs through one route and the subtitle either arrives late or jumps around.

Subtitles cannot keep piling up

Once recognition and translation caught up, the screen became the problem. During a long speech, the entire block kept reflowing and my eyes lost their place. Cutting too aggressively was no better; one sentence became several pieces of chat.

mimi tries to split where a sentence naturally ends. Confirmed segments stay above and fade with time; only the active tail keeps changing, so unfinished words do not take over the window.

That sounds like a display detail. It still decides whether the screen feels like subtitles or a terminal log.

What it can do now

• Captures system audio without opening the microphone, so browsers, players, and desktop apps all work
• Recognises Chinese, English, Japanese, or Korean; output can stay in the source language or translate to Chinese, English, or Japanese
• The overlay can be dragged, resized, and locked for click-through
• Confirmed subtitles fade while only the active tail continues to update
• No account, audio archive, or persistent subtitle history

“Live” does not mean zero latency. The network, recognition, and translation all take time, and the first lines may need a moment to catch up. Zero latency is out of reach; keeping stale work from forming a queue is manageable.

mimi is not meant to be a meeting transcription product. There is no account or cloud history, and it will not turn an evening into a document. A little recent dialogue remains in memory for display and translation context, but it is not written to disk.

The next three images are anime-style usage illustrations, not product screenshots:

Drama: the overlay sits above the player, with no subtitle file to hunt down.

Game: lock the overlay and the mouse passes through to the controls underneath.

Livestream: mimi listens to system audio even when no subtitle track exists.

The source and latest download are on GitHub (opens in a new tab). First use can configure the default Alibaba Cloud Model Studio profile or add an OpenAI Realtime profile. Model calls may cost money—that part is worth saying plainly.

After becoming cross-platform, first-launch permissions and security prompts differ by system. Names, titles, and very fast dialogue still come out wrong. But when a video has no subtitles now, I open mimi first instead of hunting for a subtitle file.