Meta Connect 2024: Meta AI gets lip-synced translation tool on Facebook, Instagram

Meta AI update: Mark Zuckerberg-owned Meta Platform announced a major update to its generative artificial intelligence (AI)-based assistant, Meta AI, at Meta Connect 2024. In addition to being powered by the company's newest Large Language Model (LLM) Llama 3.2, Meta announced new tools for Meta AI on Instagram and Facebook that are aimed at making it easier for the creators to create content and share it on Meta's apps.

Here are all the new features coming to Meta AI on Facebook and Instagram:

Meta AI lip-sync and dubbing tool

Meta has announced that it is testing a new AI translation tool that will enable creators to automatically translate the audio of Reels to different languages. It will also lip-sync the translation for coherence. "With automatic dubbing and lip syncing, Meta AI will simulate the speaker’s voice in another language and sync their lips to match," Meta wrote in its blog post.

As far as availability is concerned, Meta said that it has started testing the auto dubbing and lip-syncing feature to a small group of creators from Latin America and the US in English and Spanish. It will expand it to more creators and languages in the future.

Background creation tool in Stories

Meta also announced that now Meta AI will be able to generate new backgrounds for the images that users reshare from their feeds to their Instagram Stories.

Meta AI's Imagine feature comes to Facebook, Instagram

Additionally, Meta said it is expanding the availability of Meta AI's Imagine feature to Instagram and Facebook. With this, users will be able to share their AI-generated images in Feed and Stories on Facebook and Instagram. Meta, in its blog, also said that Meta AI will also be able to suggest captions for Stories on Facebook and Instagram.

New AI-generated background

Lastly, the company is rolling out a new feature that will enable users to make personalised AI generated backgrounds for their chats on Instagram.