Why do auto-translated subtitles read fluently but still get things wrong?

Fluency and accuracy are produced by different stages. The translation stage optimises for natural output text, so a misheard word arrives already corrupted and gets rendered as a confident, grammatical English sentence rather than flagged as uncertain.

Do auto-translated subtitles get better on higher-quality video?

Only if the audio improves with it. Speech recognition works from the audio track, so resolution is irrelevant — a clean single-microphone recording produces better results than a 4K file with noisy, overlapping audio.

Should I turn auto-translated subtitles off?

Usually not. They carry situational information that silence does not, and the failure modes are recognisable once you know them. Distrust specific lines rather than discarding the whole track.

How Good Are Auto-Translated Subtitles Really?

Auto-translated subtitles are reliably good enough to follow who is doing what and why, and unreliable for wordplay, honorifics, names, and any line where a mistranslation changes the meaning. Quality is limited less by the translation stage than by the speech recognition feeding it.

Auto-translated subtitles are good enough for the thing most viewers actually want — following the situation, the roles, and the general drift of a conversation. They are not good enough for anything that turns on the precise wording of a single line.

That split is not a hedge. It is a direct consequence of how the pipeline is built: two automated stages in series, where the first stage's mistakes become the second stage's input, and the second stage has no way to know it is translating something that was never said.

What is actually happening when subtitles are auto-generated?

Two separate systems run in sequence, and they do not talk back to each other.

Stage one is automatic speech recognition (ASR). It converts the audio track into Japanese text with timings. This is the hard stage for this material, and it is where most of the damage happens.

Stage two is machine translation (MT). It takes that Japanese text and produces English. Modern MT is genuinely strong at this — on clean, correct input.

The problem is the interface between them. Stage two receives text, not audio. It cannot hear that stage one guessed. A misheard word arrives looking exactly like a correctly heard word, and MT does what it is built to do: produce fluent, grammatical output. The result is a sentence that reads perfectly and means something that was never said.

Why is this material harder than the audio these systems are tuned on?

Because speech recognition is trained and benchmarked largely on broadcast, lecture and meeting audio, and adult video violates nearly every assumption that material encodes.

Speakers overlap. ASR is built around turn-taking. Simultaneous speech either collapses into one speaker's line or produces a blend of both.
Much of the audio is non-verbal. Breath and vocalisation occupy the acoustic space where a system expects words, and a system that is biased toward finding speech will find speech.
Recording conditions vary wildly. Handheld and single-microphone setups produce a speech-to-noise ratio far below broadcast baseline.
Context windows are sparse. MT resolves ambiguity using surrounding sentences. Dialogue here is often short, isolated exclamations with almost no surrounding text to disambiguate against.
Japanese drops subjects. A grammatical Japanese sentence frequently omits who is speaking, who is being addressed, and whether the referent is singular. English requires a pronoun, so the translator must invent one — and it will pick a plausible default and state it with full confidence.
Honorifics and register have no English slot. The relational information carried by speech level is one of the first things lost.

None of these are solved by a better translation model, because none of them are translation problems.

What kinds of errors should you expect, and how bad is each?

Errors sort cleanly by which stage produced them, and that determines how recoverable they are.

Error type	Originates in	How it looks on screen	Impact on comprehension
Invented dialogue over non-speech audio	ASR	Lines appear where nobody is talking	Low — obvious, easy to ignore
Misheard word rendered fluently	ASR	A grammatical sentence that doesn't fit the scene	High — invisible, and plausible
Names turned into ordinary words	ASR	Person's name replaced by a similar-sounding noun	Medium — confusing but detectable
Wrong pronoun gender or number	MT	"He" and "she" swap for the same person	Medium — jarring, usually inferable
Honorifics and register flattened	MT	Everyone speaks in the same neutral tone	Medium — relational nuance lost entirely
Idiom translated literally	MT	Word-by-word rendering that means nothing	Low — recognisable as nonsense
Wordplay and puns	MT	Either literal, or replaced by something unrelated	High — the joke is simply gone
Overlapping speech reduced to one line	ASR	Only the louder speaker appears	Medium — you lose the exchange

The two high-impact rows share a property: you cannot detect them from the subtitle alone. Everything else announces itself. A misheard word rendered into fluent English does not, which is exactly why it matters most.

When can you trust them, and when should you not?

Trust them for continuity and situation; distrust them at any point where the meaning is load-bearing.

Reasonable to rely on:

Following who is present, what is happening, and how the scene progresses
Broad emotional register — agreement, refusal, hesitation
Repeated, formulaic exchanges where context makes the meaning redundant
Deciding whether a scene is what you were looking for

Worth distrusting:

Any line that seems to reveal a plot point, relationship or premise
Names, places, brands and label references
Anything presented as a joke or a pun
Lines during overlapping dialogue or heavy background noise
Politeness and register cues, if the relationship between people matters
Numbers and times, which ASR misrecognises readily

A practical rule: if a single subtitle line changes your understanding of the whole scene, treat it as unverified. Genuine plot information tends to be restated or shown visually. A revelation that appears exactly once, in one line, and never recurs is more likely a pipeline artefact than a script point.

How do you judge a track quickly?

Watch the first few minutes with these five checks, and the track's grade will be obvious.

Check	What a weak track does
Do lines appear during silence or non-verbal audio?	Yes, frequently
Is a person's name spelled the same way each time?	No — it drifts or becomes a common noun
Do pronouns stay consistent for one person?	No — they flip between lines
Are lines uniformly short and similarly sized?	Yes — automatic segmentation
Do idioms read as literal nonsense?	Yes — no idiomatic rewriting

Two or more of these means you are watching an automated pipeline's output, and quality will not improve later in the runtime. That is worth knowing up front, because it tells you how much weight to put on any individual line for the remaining hour or two.