Google attempts to infuse Lip Syncing into its Artificial Intelligence System

168

Google introduces the lip-sync challenge for its reputed AI system with the goal to induce reading ability into its AI system to ease the feasibility of its operations for individuals with speaking disabilities. Applications will be crafted for such individuals after the successful imbibition of the art of lip reading into the Google AI system. Run by the Google AI Experiments Group, the challenge is open to all to test their skills. The experiment uses Google’s AI technology TensorFlow.js to detect facial movements while the user is lip syncing with the song. The FaceMesh model is used in this experiment to compare the facial movements to an existing baseline.

How to Begin?

With the sole aid of a webcam, you can successfully participate in the challenge on the page entitled Experiments with Google-LipSync by YouTube. After clicking on the “launch experiment’ button, you will be redirected to another screen that automatically configures the AI system to sync with the equipment you are using. Subsequently, you can now see yourself in a bubble on the screen, preparing to click on the ‘I’m ready’ button. Clicking this button releases the song ‘Dance Monkeys’ by Tones and I, beginning the lip sync challenge. You now have to sing along to get a rating out of five stars- upon the song ending. This score card is produced in the next screen.

Future Upgrades?

The future of LipSync remains uncertain in terms of Google adding more songs or additional features. As of today, it is based solely on the optimization of facial expressions and lip movements and does not store any additional user audio or data.

A Testament to Innovations:

LipSync is an exceptional innovation to enhance the operative accuracy of AIs because it does not rely on any additional hardware equipment, apart from the webcam. If an AI revolution is imminent, this is a step forward in the creation of smarter AI systems. Google’s attempts at developing a similar project in 2016, in collaboration with the Oxford University- entitled DeepMind- was equally significant. However, what sets LipSync apart is the elevated innovative capacity it espouses that uses simply a video camera to accurately decipher facial movements. If groomed to reach its full potential, LipSync will not only revolutionize virtual speech recognition but also aid in creating a better virtual experience for people with speech disorders like, ALS.