Computer Vision: Python Face Swap and Quick Deepfake In Colab

There's an old saying that goes, "Seeing is believing." But in the "Deepfake" universe, what we see is not always reality. Let's use a video example to define deep fake.

I made these films with just a single stamp-sized photo of individual people. You heard it right. You can make a deepfake of someone with just one face image and a few minutes on a standard PC. Even the dead can be made to speak or sing for you.

Computer Vision: Python Face Swap and Quick Deepfake In Colab Course Overview

There are exactly two halves to this course.

We'll build a simple Python-based face swap application in the first half. We will first provide an overview of the deep fake technique, including its uses, benefits, and drawbacks. The next step is to set up our machine with all the necessary requirements.  Anaconda, the platform and IDE for our Python programming will be installed. For individuals who want to study the fundamentals of the Python programming language, there are a few optional courses that come later.

The remaining dependencies needed to create our unique Python face swap program will be installed later. After that, we will write more than 300 lines of Python code one line at a time to finish the program. As an alternative, you can download the entire code from the Google Drive URL given in the course's final session. We will initially perform the face swap using two still photos. The target image is the other, whereas the first is the source image. We'll test it out later with live footage from our computer's web camera. Afterward, we'll change it so that it may be used with a previously saved video that is stored on our computer.

How the curriculum works

You can see from the examples that this was only a very simple face swap program and that it was far from flawless. We were merely interested in understanding how things operated behind the scenes.

A study titled "First Order Motion Model for Picture Animation" will serve as the foundation for the implementation of deep fake later on. It was submitted by Aliaksandr Siarohin, Stéphane Lathuilière, Sergei Tulyakov, Elisa Ricci, and Nicu Sebe as their work to Cornell University.

We have a backup plan that involves using the free GPU provided by Google Colab because training a deepfake requires pricey GPUs. In order to prepare our Google Drive, we will create folders, upload a sample driving video, and then add the source and target images that will be used to create the animated target image.

Also, we will download the Google Colab sample notebook and link it to Google Drive. The first-order motion model repository will then be copied from Google Drive.

Later, we'll also go ahead and clone the repository for face alignment. We'll set it up in our Google Colab after installation. Then, using a built-in Python program, we will begin by cropping the driving footage after moving the files into the appropriate folders.

The frozen inference graph of the already trained model will then be downloaded to our Google Drive. Now that the source pictures based on the driving footage are ready to be animated, the process may begin. We'll download the animated movie once it's finished. For a handful of the other source photos, we will likewise follow a similar procedure.

There won't be any audio in the animated video. Thus, utilizing any free or open-source video editing software that is available online, we must mix the music to it. Finally, we have all the deepfake animated videos with audio. We will do that in our following session.

In the last session, we'll also go over ways to conserve the limited amount of Google's free GPU time as well as solutions in case it runs out.

Just one more thing before I wrap up. Please utilize the information and methods described in this course in a very responsible manner. It is solely meant to be used for educational and research purposes. Any improper or careless use of this technique is not the responsibility of me as the instructor or the site where I am hosting this course.

That wraps up the subjects covered in this little course at the moment. The files used in this course's code, graphics, and weights have been shared in a folder. The download link will be provided in the final lesson or in the course's resource area. There are no restrictions on the use of the code in your projects.

You will receive a course completion certificate as well after finishing the course, which will enhance your portfolio.

Bibliographies and Reference Credits

• NIPS Proceedings - First Order Motion Model for Image Animation - Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, Nicu Sebe

• Cornell University - Computer Vision and Pattern Recognition - First Order Motion Model for Image Animation

• Github - AliaksandrSiarohin - first-order-model

• Github Pages - First Order Motion Model for Image Animation

• Learn OpenCV - Delaunay Triangulation and Voronoi Diagram using OpenCV

• Learn OpenCV - Face Swap using OpenCV - Satya Mallick

• py source - Face swapping - Sergio Canu