The painted smile of the “Mona Lisa” is known around the world, but that famous face recently displayed a startling new range of expressions, courtesy of artificial intelligence (AI).
In a video shared to YouTube on May 21, three video clips show disconcerting examples of the Mona Lisa as she moves her lips and turns her head.
She was created by a convolutional neural network a type of AI that processes information much as a human brain does, to analyze and process images.
Researchers trained the algorithm to understand facial features’ general shapes and how they behave relative to each other, and then to apply that information to still images. The result was a realistic video sequence of new facial expressions from a single frame.
For the Mona Lisa videos, the AI “learned” facial movement from datasets of three human subjects, producing three very different animations.
While each of the three clips was still recognizable as the Mona Lisa, variations in the training models’ looks and behavior lent distinct “personalities” to the “living portraits.”
The engineers introduced the AI to a very large dataset of reference videos showing human faces in action. The scientists established facial landmarks that would apply to any face, to teach the neural network how faces behave in general.
Then, they trained the AI to use the reference expressions to map movement of the source’s features. This enabled the AI to create a deepfake even when it had just one image to work from, the researchers reported.