Face swapping technology has been seen in recent years in Disney films with varying levels of success. However, a project coming out of the Disney Research Studios is poised to completely change that. While the technology has advanced through the years, it is about to go to a whole new level thanks to Disney’s neural face-swapping technology.
This technology addresses several of the challenges in creating these deepfake videos as they are known. One of them is creating a massive database of a person. This needs to be a look at every angle of every expression of a person. The larger the database, the better the quality can be for creating accurate face swaps. Unfortunately, quite often the source images aren’t able to get this much data. So, to make a truly accurate and convincing face swap video is getting high-quality origin data. If it is high enough quality, it can be used for film production.
Disney has created a new algorithm that can do just that. From the source images/video, it can first stabilize and smooth out the original target. This is to make it easier for swapping faces later in the process. Then the process of blending the new face with the original face begins. This is quite complicated but basically the algorithm works to create a smooth and blended compilation between the original face and the new one so it looks natural.
Here is a demo video that Disney Research Studios posted on how this process works:
We propose an algorithm for fully automatic neural face swapping in images and videos. To the best of our knowledge, this is the first method capable of rendering photo-realistic and temporally coherent results at megapixel resolution. To this end, we introduce a progressively trained multi-way comb network and a light- and contrast-preserving blending method. We also show that while progressive training enables generation of high-resolution images, extending the architecture and training data beyond two people allows us to achieve higher fidelity in generated expressions. When compositing the generated expression onto the target face, we show how to adapt the blending strategy to preserve contrast and low-frequency lighting. Finally, we incorporate a refinement strategy into the face landmark stabilization algorithm to achieve temporal stability, which is crucial for working with high-resolution videos. We conduct an extensive ablation study to show the influence of our design choices on the quality of the swap and compare our work with popular state-of-the-art methods.
The potential for this technology really is limitless as it continues to get more and more believable. However, the potential may not be the way one originally imagined. Yes, it could be used for face swaps between an actor and their stunt double. It could also help avoid reshoots with moments that didn’t go quite as expected. As the technology continues to improve this will become more and more the norm. It will also become a way to save money on production. One other thing it will do? It will make it much harder to recognize when these deepfakes are used. If things go as Disney Research Studios hope, soon the fake could be indistinguishable from the real thing!
What do you think about this deepfake/face swapping technology? Share your thoughts in the comments below or with DAPS MAGIC on Facebook or Twitter!
