The lips stop matching the voice
Nobody notices it in the first ten minutes. Two hours into a live broadcast, the news anchor's mouth is very slightly ahead of their voice — or behind it — and the viewer can't say why the channel feels off, but they feel it. Lip-sync error is the uncanny valley of video playback: below a threshold it's invisible, above it the whole picture feels cheap, and on a long-running live stream you drift across that line so slowly you never catch it on a short test.
Video-on-demand rarely has this problem — a two-hour film decodes from one clean timeline and stops. Live is different. It runs for hours, it splices in ads, it comes off encoders and packagers that were never perfectly in step, and it plays on TV SoCs whose clocks are a few parts-per-million off from everyone else's. Drift is the default; staying in sync is the thing you have to engineer.
Why audio and video pull apart
Inside the player, audio and video are two pipelines with two decoders, and they don't advance on the same clock unless something makes them.
Every frame and audio sample carries timestamps — PTS for presentation, DTS for decode — that say when it should be shown or heard relative to a shared timeline. In theory both pipelines obey those stamps and stay locked. In practice the audio clock and the video clock are physically different oscillators, and on cheap hardware the audio DAC and the video output tick at very slightly different rates. A few parts-per-million sounds like nothing; across three hours a few ppm is tens of milliseconds, and it accumulates in one direction. That's the slow creep.
Then there's the source. Live segments don't always have tidy, identical durations. Ad breaks arrive on discontinuity boundaries — SCTE-35 markers that cut from the program to an ad and back — and each splice is a place where the audio and video timelines can be stitched with a small offset. Timestamp gaps, a segment that's a hair short or long, a PTS that wraps or jumps: every one is a chance to gain or lose a few milliseconds.
Buffer underruns are the violent version. When the network stalls and the buffer runs dry, something has to give. If the video pipeline drops or skips frames to catch up while the audio keeps playing — or the reverse — you don't drift by a few milliseconds, you step out of sync all at once.
Detecting it before the viewer does
You can't fix drift you aren't measuring, and the measurement is comparing where audio thinks it is against where video thinks it is. In the browser you have less direct access than a native player, but you can compare the audio and video positions the pipeline exposes and watch the delta grow over time. On a native stack you can read the actual render timestamps of the two pipelines. Either way the signal is the same: a widening difference between the audio clock and the video clock.
The number that matters is the perception threshold, and it isn't symmetric. Humans tolerate audio lagging video far better than audio leading it — we're used to sound arriving after the thing that made it, never before. The broadcast rule of thumb, from the ITU and EBU work on lip-sync, is roughly audio no more than +45ms ahead of video and no more than −125ms behind before it becomes detectable. So your correction thresholds should be asymmetric too: react sooner when audio gets ahead of the picture than when it falls behind, because the eye is far less forgiving of sound that arrives early.
Fixing it: a nudge or a jump
Once you've decided the drift is past threshold, you have two tools, and the whole craft is choosing between them.
The gentle tool is a playback-rate nudge. Speed up or slow down the leading pipeline by a fraction of a percent — playbackRate = 1.02 for a moment — and let it close the gap over a second or two. Kept small enough, tens of milliseconds of pitch-shift and frame pacing sit below anyone's notice. This is the right tool for the slow ppm creep, where the error is small and you have time to bleed it off invisibly.
The blunt tool is a hard reseek: jump the player to a corrected position and re-align both pipelines to the same timeline point. It fixes any size of error instantly, but the user sees and hears it — a stutter, a repeated or dropped moment. You reserve it for the big desyncs, the step-changes a nudge would take too long to absorb.
The structural move is to resync on the boundaries the stream already gives you. A discontinuity — an ad splice, an SCTE-35 marker — is a place where a small correction hides inside a cut the viewer already expects to be a little seamless-ish. Re-anchoring both pipelines to the media timeline at a discontinuity is nearly free perceptually, where doing the same jump mid-sentence is jarring. Correct where the seam already is.
The tradeoff you're actually making
Every correction is a choice between two not-quite-free evils: a moment of audible or visible fix-up now, or letting the drift ride and hoping it stays under threshold. Neither is invisible.
The judgment is proportional. A small, slow drift below the perception threshold — leave it; the cure is more noticeable than the disease. A drift heading past +45ms of audio-ahead — nudge it now, before the viewer's eye catches it. A sudden step out of an underrun — take the hard reseek and accept the one stutter, because the alternative is hours of everything being subtly wrong. On a stream that runs all day, the goal isn't perfect sync at every instant. It's never letting the error grow big enough, or last long enough, for a human to name what's bothering them.