Prime Video Uses AI to Sync Lips to Dubbed Audio

Amazon’s Prime Video has introduced an AI feature that aligns actors’ mouth movements with dubbed audio. It applies to the English-dubbed version of the German-origin series Maxton Hall. It is being offered to subscribers worldwide. Reporting by The Verge’s Emma Roth summed up the key point of the feature as follows.

lines up an actor’s mouth with “human-dubbed” audio

The distinctive feature is that it starts from human-recorded dubbing and retroactively adapts the footage around the mouth. For now, it is limited to the English version of the series. The company says it plans to expand to more titles in the future.

Starting With Limited Availability for the

English-Dubbed Version

The rollout covers Seasons 1 and 2 of Maxton Hall. Lip sync is applied when watching with the English dub. The eligible region is worldwide. Subscribers in all countries, including Japan, can use it.

Season 3 is scheduled for release on December 9, 2026. The company says the feature will also be available for that season. This is a stage of verifying effectiveness through operation on a single title. It can be seen as a decision to opt for gradual expansion.

Video processing on distribution infrastructure is a high-load area. Optimizing encoding and decoding remains an ongoing challenge. As Vulkan Video Encode Returns on Intel Alchemist GPUs shows, the options for handling video processing are diversifying. This lip-sync effort also relies on a processing foundation for video generation and compositing.

System Combining AI and Visual Effects

According to Prime Video, it combines AI and visual effects technologies. The goal is to match mouth movements to translated dialogue. It does not create dubbing through voice synthesis. Instead, it adapts the video to human-recorded dubbed audio.

It adjusts mouth opening and closing, jaw movement, and cheek changes on the video side. The processing must match speech breaks and variations in sound intensity. It is not a simple face replacement. It is designed to reduce unnaturalness while preserving the original performance and expressions.

In generative video editing, preserving a person’s identity is important. If the face looks like someone else, it undermines the value of the work. It must also follow lighting, camera angles, and the speaker’s orientation. Stability without breakdowns during long playback of a series is a requirement.

Aim to Eliminate the Awkwardness of Dubbing

In dubbed works, a mismatch between audio and mouth movements remains. Viewers unconsciously check the mouth area. If the mismatch is large, it disrupts concentration on the content. It can also motivate switching to subtitles.

Lip sync is a measure aimed at maintaining immersion. It retains the original-language performance while broadening the audience in other languages. It is effective in bringing a German work to English-speaking audiences. It can lower language barriers and expand distribution of the work.

Consideration for the employment of dubbing performers can also be seen. The audio itself remains human-recorded. It is not replaced by automated voices. By using AI for correction on the video side, it clarifies the division of labor between human work and technology.

Differences From Rivals’

Automatic Dubbing Features

Meta and YouTube have introduced automatic dubbing features for creators. They automate audio translation and generation. They also offer a lip-sync option to make it look as if the person is speaking the viewer’s language. The aim is to facilitate multilingual expansion of user-posted videos.

Prime Video’s approach takes a different direction. It does not foreground automatic voice generation. It starts from human dubbing and refines the video. It is a choice that emphasizes quality control as a work. It is a process suited to commercial distribution of films and series.

User-posted videos and commercial works require different standards. For the former, speed and volume are important. For the latter, acting and consistency of the worldview are important. Even with the same lip sync, the application area and quality standards differ. The technology could be repurposed, but the operational philosophy is different.

Rollout Following a Pilot Across 12 Titles

Last year, Prime Video began a pilot of AI-assisted dubbing. It covered English and Spanish. It involved 12 films and series. They included El Cid: La Leyenda, Mi Mamá Lora, and Long Lost.

That effort focused mainly on supporting the audio side. This lip sync moves into correction on the video side. It appears to build on language processing and production workflow insights gained from the pilot. Rather than a sudden full-scale rollout, it is a step-by-step expansion.

Handling of video and audio is also important in Amazon’s devices and application layer. For example, Amazon Fire Max 11 Root Shell Hack Enables Disabling Home Screen highlighted display control on the device side. Video processing on the distribution side and the playback environment on the device side affect each other. Optimizing the whole chain from production to playback is a challenge.

Impact on the Battle for Streaming Leadership

Streaming services are strengthening procurement of works from outside their home countries. Global hits of non-English works continue. The quality of dubbing and subtitles directly affects continued viewing. Lip sync could become a differentiating factor.

Rising production costs are also a factor. There is a need to leverage existing assets rather than rely on new blockbusters. Increasing revenue per title through multilingual expansion is an effective approach. Value can be increased without additional shooting.

Rights clearance and handling of likenesses will also become issues. It involves altering actors’ faces and mouths with AI. Prior agreement and contract arrangements are prerequisites. Consultations with labor unions and how to disclose the use will also be questioned. As the technology spreads, rule-making on production sets is expected to advance.

Handling of personal information and training materials is also a common industry challenge. As OpenAI Announces Private Safety Processing with Zero Data Retention shows, interest is high in designs that do not retain information. In video generation as well, clarification of material management and retention policies is required.

Editorial Opinion

On the short-term impact. In the next three to six months, attention will focus on the operational results for Maxton Hall. Watch time, dubbing selection rates, and effects on deterring cancellations will be subject to verification. Other streaming operators may also reveal follow-up measures. On production sets, contract terms for likeness use are expected to be reviewed.

On the long-term perspective. In one to three years, simultaneous multilingual release is expected to be incorporated into the standard workflow. Lip sync is assessed to raise the quality standard for dubbed versions. The division of labor between human recording and video correction may become established. It is seen as becoming a foundation supporting the global distribution of regional works.

Questions from the editorial team. The issue of who decides the acceptable scope of alteration remains. Is the consent of the actors themselves sufficient. Is disclosure to viewers necessary. There is also the challenge of how to prevent repurposing for archival footage or news footage. The setting of boundaries between convenience and authenticity will be questioned.

References

• “Amazon Prime Video’s new AI tech matches lips to dubbed audio”, by Emma Roth — The Verge, 2026-09-09T13:00:00.000Z (ARR)
• Source URL: https://www.theverge.com/tech/991809/amazon-prime-video-ai-lip-sync-dubbing