“Deepfake” isn’t a new type of beauty cream, an underwater virtual tour or even a sternly worded insult. It is in fact, the culmination of decades worth of audio and video editing methods combined with cutting-edge artificial intelligence and machine learning.
Despite the fact that most people have come across it on YouTube, Facebook and other places, most actually don’t understand what is involved in it. Why is it so revolutionary and terrifying? What makes it that much different than existing techniques? Why does it have the potential to allow actors, actresses and more to create new content years after they are gone? Let’s find out.
What is a deepfake?
Deepfakes first entered the public consciousness in 2018, with a demonstration video with President Obama in conjunction with comedian Jordan Peele. We’ll address why that particular video was so effective later, but the core idea of a deepfake is essentially taking media of one person and using it as a template to create a new video of another.
Now some could argue that this has been in place conceptually for years and years now. Take, for instance, “Forrest Gump,” which showcases Tom Hanks meeting historical figures such as President John F. Kennedy. These scenes obviously did not occur — yet look like they did. These examples are forms of video editing where it’s possible to overlay a person into existing film footage, as shown in the behind-the-scenes video from Dateline in 1994. The key limitation with this technique is how well you can match up to the source material.
On the flip side, there are also imitators of different celebrities and other famous people. Elvis Presley impersonators are a common sight across the United States and are often judged on just how close a resemblance they can create: movements, body type, voice and vocal patterns all play into the accuracy of their presentation. However, this particular style does not work for long, as it is incredibly hard to keep up for an extended period of time.
Deepfakes, on the other hand, are able to take high-definition digital recordings — videos and still images — and combine them with various software solutions to create something new. DeepFaceLab is one of the primary pieces of software able to perform facial recognition and begin calculating out the differences between two different sources and what it would look like if one person was performing the actions that the other was doing.
One great example of this software in action is a simulation of what it would look like if Robert Downey Jr. and Tom Holland acted together in a particular scene from “Back to the Future.” Only the faces themselves have been modified, leaving the audio and bodies from Christopher Lloyd and Michael J. Fox intact.
Why is this not always enough?
Just doing face replacement does not change the nature of a scene on its own, and even when it comes to that, clean source data is critical. In an example video from Corridor Digital, they showcase the problems they ran into when working on a music video for Snoop Dogg. If you were trying to bring somebody back that only limited source data exists for, the system would run into tremendous problems trying to calculate everything from scratch. This means more than just facial features need to be considered: elements such as lighting, voices and mannerisms, the small touches that can take something from a “computer graphic” to “that person.” As such, trying to create “found footage” in this way from someone like Abraham Lincoln, a person that existed before the advent of high-quality imagery, would be significantly harder than just finding someone close enough to imitate them.
In this instance and in the President Obama video listed above, they did just that: they were able to imitate the person they were trying to create the deepfake of and got as close as they could before they began the deepfake calculations. Once combined, this creates an output that is as polished as possible, given the technology currently available.
What does the future hold for this technology?
On the positive side, video games and similar media have been creating new problems and possibilities for game developers, movie studios, sports athletes and actors/actresses for years now. Being able to use likenesses and digital doubles of a person years after they are unable to perform what they are famous for can make a product significantly more appealing, as well as generating a steady revenue stream for years to come.
On the negative side, actors and actresses have already had their identities stolen in this way and used without their consent in malicious ways. Videos have the potential to be released for political and financial gain that have no basis in fact or reality. When identity theft is already on the rise through typical means and it is possible to create a virtually undetectable video duplicate of a person, the possibilities for danger are extremely high.
Whether it is for use for a major motion picture, becoming a video game character or an animated gif on social media, deepfakes are not going away anytime soon. Like most technology, the use case truly determines whether it is good or bad, and that places this particular piece of technology firmly in the gray. However, the technology itself is impressive and honestly worth researching in greater depth if you have the time, though deepfakes are not the only emerging technology with the potential to change the rules.
- You Won’t Believe What Obama Says In This Video!, BuzzFeedVideo (YouTube)
- Jordan Peele uses AI, President Obama in fake news PSA, Good Morning America (YouTube)
- How Far is Too Far? | The Age of A.I., YouTube Originals (YouTube)
- We Made The Best Deepfake on The Internet, Corridor Crew (YouTube)
- DeepFaceLab, GitHub
- Snoop Dogg Had us Bring 2pac Back to Life!, Corridor Crew (YouTube)
- ROGUE ONE Movie Clip – Grand Moff Tarkin’s Death Star |4K ULTRA HD| Star Wars Movie 2016, JoBlo Movie Clips (YouTube)
- Robert Downey Jr and Tom Holland in Back to the future – This is heavy! [ deepfake ], EZRyderX47 (YouTube)
- Deepfakes Are Amazing. They’re Also Terrifying for Our Future., Popular Mechanics
- What Is a Deepfake?, PC Mag
- “Forrest Gump” Visual Effects, Dateline NBC (1994), Todd Vaziri (YouTube)
- ‘Give the old players a million dollars’: Charles Barkley explains why he isn’t on NBA 2K21 and other 2k versions, The Sports Rush
- Deepfakes: A Terror in the era of AI?, Norton Lifelock