How ‘Welcome to Chechnya’ used A.I. and machine learning techniques to mask the doco’s subjects

“For their safety, people fleeing for their lives have been digitally disguised.”
– The opening title card to the film.

The use of machine learning, A.I. and deepfakes are gaining a whole heap of traction in VFX right now. Which is one reason the documentary film Welcome to Chechnya has also garnered attention. It used these visual effects techniques to help protect the identities of those interviewed for David France’s film about activists working to help gay men and woman fleeing from persecution from the Russian republic of Chechnya.

After initially looking to filtering and rotoscoping and animation to cover the faces of his interview subjects, France turned to visual effects supervisor Ryan Laney and his team to ‘mask’ those on camera with the identities of ‘face doubles’ (the voices were also fabricated) using various techniques, including the machine learning ones. The idea was to preserve the emotional impact of the stories of these individuals without revealing who they were.

Here’s how they pulled it off.

 

b&a: I’m curious what you ‘called’ your face swapping effect in the end, but first, as a filmmaker, David, how did you come to choose the kind of approach you would take to mask the identities of the people in the film?

David France: I had, of course never had a need to disguise anybody before in a documentary. And I promised everybody who I was filming that I would disguise them to the highest standard, which was that even their own mothers wouldn’t recognize them. And I was sure I could do that. And it wasn’t until it came back that I realized that I was taking on something that was going to be quite a challenge. So the first thing that we started working with and keeping in mind Ian that all of this was being done in secret because we were protecting the footage potentially from state actors. We worked only on encrypted drives. We spoke only on encrypted lines. We created an edit suite that was air gapped so that none of the computers that worked on the footage had ever been on the internet.

So then we brought people into the studio to do this kind of roto-mation approach with pulling people out of the film, filtering them, putting them back in. And we discovered to our surprise that that looked good, but it didn’t disguise. And then we brought more people in, put more filters on the faces to expand the size of noses to alter the facial shapes, but it had the effect of actually making people more recognizable. It accentuated their noses and their foreheads and their ears, the things about them that their mother would know intimately. So we just failed over and over. We tried using kind of Snapchat-y kind of approaches, where we would put like glasses on people.

And we developed an entire language of masks where we tried like the Ninja Turtle masks. And we tried the stagecoach robber mask. And we tried a bunch of stuff that also didn’t disguise, or if it did disguise, it erased the humanity of the person and certainly didn’t fit or match the urgency and seriousness of the film. And that’s what brought us to Ryan. And Ryan had been thinking with his team about ways to do some of this stuff already. So we found a mind already engaged in this, and we were lucky we did.


 

Ryan Laney: We call them ‘veils’, just to answer that first question. So, David luckily had spent a good amount of time trying some other ideas. And even though they didn’t work, they were really valuable in giving us a message about what was working and what could work and where we needed to spend attention. We ended up drawing from academia and a few papers that have been written in previous years.

There’s a face research lab in Glasgow that has a really interesting faces around the world. And we are, in our human form, we’re very similar in a lot of ways, and we really kind of focused in on this idea that the changes we wanted to make were not big sweeping animation changes, but really subtle things. We talked about for a while, doing a bit like a prosthetic, like we were painting on new eyebrows or putting on different lipstick. That’s really where our mind was when we discovered the deep machine learning. And that allowed us to give it a lot of data from the film and from stuff that we shot to sort of pair up and be able to paint on this new veil.

The machine learning part of it really helps correlate the face shapes between the two different people and so the emotion comes through really well. I think what we learned through this process is how much we telegraph how we feel, not necessarily through face shapes, but through face motion. Even from our first test where it was working at a lower resolution, it was impressive how well the emotion came through and how honest and truthful we felt like the solution was.

b&a: Could you explain from perhaps both of your points of view what the workflow ended up being?

David France: We had been doing our R&D in our own studio while we were in post. And we found Ryan and his team as we were getting close to lock. And at that point we had no plan for distributing the film because we hadn’t found our solution yet. Once we saw what Ryan was doing and saw how successful it would be, we handed him a locked picture. We unlocked it after the effects came in, because it suggested a couple of things to us, but mostly it was a locked picture. And in that interim, Ryan had various approaches that he thought we could do it like this or this or that.

And we took them to another academic setting, to a neuropsych lab at Dartmouth, where the director of the lab is something of an expert in the Uncanny Valley. And we wanted to make sure before we pulled the trigger that we weren’t pushing people out of the film in any sort of kind of unexpected ways. And that’s how we settled on this final approach. She took it into a study, I think 200 people in the study, and confirmed that we were on the right track. And that’s when we said to Ryan, here’s our finished picture, take it from here.

b&a: And then what were your steps, Ryan, in actually completing this work?

Ryan Laney: Maxwell Anderson was the post supervisor on the production film side. He packaged up everything and all the original footage was actually delivered to us by hand on a drive for security reasons. So that was, I guess, unusual for today’s ability to transfer stuff over the wire. We produced a bunch of tests. Dr. Wheatley at Dartmouth College, who David was mentioning—we were concerned that the softness of the face would be a trigger for people. And very interestingly, our hypothesis was wrong in the sense that without a veil people felt more concerned for the person in the film, because they felt like they were at risk. And with the veil, the unsettling rating was slightly lower for the veiled person.

So we felt really, really good that we were going down the right track with this particular solution. And then, in just the footage we shot, I went through the entire film, Piers Dennis, the effects coordinator on our end, went through the entire film and selected all the people that we needed to cover. There were 23, and for each person, we looked at, how were they represented lighting-wise? So we had everything from bright daylight to dimly lit, under-lit by a car, sort of nighttime shots. And so with this matrix of people by lighting scenario, we set up a shoot in Brooklyn, New York and shot for a week to capture the data and all the scenarios that we needed. And then after ingesting that it was really just a matter of elbow grease, I think you’d call it.

b&a: Deepfakes and machine learning are obviously exploding right now. I’d love to talk a little bit about how much of it really was classic deepfake work. David, what has that been like seeing so much deepfake discussion out there and then having a film that taps into that? I find that quite fascinating that you really are right there on the cusp of the discussion.

David France: You know, I come to documentary filmmaking through journalism. I’m an old investigative reporter and journalism is very concerned about deepfakes, and rightly so. I think historians are concerned about it, this idea that you could create, out of nothing, historical falsehoods. And I’ve been to workshops where these things have been discussed and there’s been a lot of hand-wringing. I never saw this as being at all related to deepfakes. I think deepfake is the crime, not the technique. And when we first started talking about it with Ryan, we realized that, going further into journalistic ethics, we needed to do this very openly. We couldn’t deceive the audience. So we certainly weren’t deceiving the subjects. They had all agreed to do this conditioned on approval of the method that we were going to use.

And we weren’t deceiving the people who lent us their faces as human shields. They were mostly LGBTQ activists, most had already been involved in bringing attention to what’s happening in Chechnya. We cast them from their Instagram pages and kept sending images over to Ryan saying, will this one work for that one? Will this one work for that one, to see who would match, whose faces seemed to be ideal for it. And then we said to the audience, look, we’re bringing you in on this.

We emphasized that fuzzy edge around the face, that kind of halo around the face, to telegraph to the audience that what they’re looking at has been altered. We have a card in the front of the film saying they have been digitally altered. We worked on, I think Ryan said at one point it was like the first 20 shots in the film to overemphasize the halos on the face to teach the audience how to watch this. And so in that way, I don’t think it has anything to do with it. In fact, just the opposite. It’s a technique that is allowing a truth to be told that otherwise would have been repressed.

b&a: And Ryan, just on that deep fake implementation or the extent to which you used it, can you talk a little bit about that?

Ryan Laney: I think David’s said before that A.I. is the tool and deepfake is the crime. And, sort of differently, deep fakes are inherently nonconsensual. And so if you’re going to like define what a deep fake is, it’s when somebody in a video doesn’t know that somebody who was shot separately didn’t know that their face was applied, right, in an attempt to trick the audience. So kind of on all three of those conversations, we are being open and honest.

We use Google TensorFlow. We certainly looked to the papers that inspired deep fakes. There was a 2017 paper about semantic style transfer, which looked at how you could correlate between two datasets, understanding basically what was in the picture before you did the style transfer. Machine learning is being used a lot in a lot of different ways these days. And so we liked the idea of using a tool for good and for human rights and to help very specifically in visual effects. Our job is to support whatever story is there. And it has really been incredible to be able to support David and the story he’s telling that does affect human rights. So we feel like this is a, it’s a really powerful message that David’s crafted, and we’re just really happy to have helped him tell that story.

b&a: At Imageworks or ILM or elsewhere, had you done any face replacement or anything like this over the years?

Ryan Laney: Well, I think Ant-Man was the most recent face replacement, but you know, it kind of has different ideas that are going on there where you’re taking your key actor and you’re putting them on a stunt man. And we’re kind of doing the reverse. We’re putting the stunt man, the stunt person, onto the real person. Except that we’re also spending a lot of energy making sure that every micro-expression is there, which you don’t always get in a face replacement. In the normal face replacement, you’re more interested in the key actor.

 

b&a: And, David, I was curious, in a big visual effects film, the director gets to review shots and make notes. I’m wondering what were your notes that you would end up giving for these sorts of shots?

David France: It’s interesting, Ryan talks about the pace that we were doing this at. So he was rendering overnight and sending in the mornings—often we were just reviewing together. I don’t think that I gave him specific notes along the way. I mean, we were very concerned about when heads turned entirely away from camera and when the effect leaves the face and what that looks like. There was a lot of dialogue about that, because it would pop on and off at points. And then how do you smooth that out and make that a little less jarring? But our major conversation and dialogue was about adding the blurs, emphasizing the blurs, which was one of the things that I felt was going to be important at the beginning of the film to teach people about what we’re doing, to actually make the VFX a character in the film.

And then for that character to then dissolve as it does in the press conference scene with Grisha, that was probably the most work that he and I did together on notes. Although the design, the idea of how to do it, was his purely. And I opened up a file he sent me and I was just blown away by his decision-making in that moment of reveal. So then the question for us was where do we put that reveal? And that was something that we had a lot of conversations about as a team collaboratively.

 

b&a: I love that reveal, Ryan. Do you want to mention anything about technically how hard that was to do?

Ryan Laney: It wasn’t terribly hard technically compared to the other things that we were doing. David said he wanted something magical. And so, with visual effects, when you’re doing something like a transition, there’s always a balance between too much, not enough. So we peppered in, there’s 20 or 25 layers in there of different colours and reveals going on to try to open up into from Grisha into Maxim, through the transition.

Also, David mentioned the first 20 shots a couple of times, I think it’s worth mentioning from an editorial perspective that Max Anderson actually removed some dialogue from that first 20 shot section, that first several minutes of film, so that the audience wasn’t reading subtitles and that they were looking at the faces so they can, this idea of training the audience, acclimating them to what was going on.

Source : https://beforesandafters.com/2021/03/07/how-welcome-to-chechnya-used-a-i-and-machine-learning-techniques-to-mask-the-docos-subjects/?utm_source=rss&utm_medium=rss&utm_campaign=how-welcome-to-chechnya-used-a-i-and-machine-learning-techniques-to-mask-the-docos-subjects

  • gamefly

    Have you ever thought about including a little bit more than just your articles?
    I mean, what you say is fundamental and all. But think about if
    you added some great pictures or videos to give your posts more, “pop”!
    Your content is excellent but with images and video clips, this blog could undeniably be one of
    the best in its field. Great blog!

  • coconut oil

    Excellent beat ! I wish to apprentice while you amend your site, how could i subscribe for a blog web site?
    The account aided me a acceptable deal. I had been a little bit acquainted of this your broadcast offered bright clear concept games ps4 allenferguson games ps4

  • http://tinyurl.com

    Hey there would you mind letting me know which webhost you’re utilizing?
    I’ve loaded your blog in 3 different internet browsers and I must say this blog
    loads a lot quicker then most. Can you recommend a good internet hosting provider at a honest price?
    Thanks, I appreciate it! games ps4 allenferguson games ps4

  • Robertpounc

    “A fascinating discussion is definitely worth comment. I think that you ought to write more about this topic, it might not be a taboo subject but typically folks don’t speak about such issues. To the next! All the best!!”
    דירות דיסקרטיות בחיפה

  • froleprotrem

    F*ckin’ awesome issues here. I’m very satisfied to peer your article. Thanks a lot and i’m looking forward to touch you. Will you please drop me a e-mail?

  • Robertpounc

    “Hello there! I simply want to offer you a huge thumbs up for the great info you have here on this post. I will be coming back to your blog for more soon.”
    דירות דיסקרטיות בצפון

  • http://j.mp

    Hey! Do you know if they make any plugins to
    help with SEO? I’m trying to get my blog to rank for some targeted keywords but I’m not seeing very good results.
    If you know of any please share. Cheers!

  • http://bit.ly/3cLS6FQ

    Excellent, what a webpage it is! This blog provides helpful data
    to us, keep it up.

  • Robertpounc

    “Hey there! I just would like to give you a huge thumbs up for your great info you have here on this post. I am coming back to your site for more soon.”
    דירות דיסקרטיות בחיפה

  • ps4 games

    Unquestionably believe that which you stated.
    Your favorite justification appeared to be on the internet
    the simplest thing to be aware of. I say to you, I certainly get irked while people consider
    worries that they just don’t know about. You managed to hit the nail upon the top
    and also defined out the whole thing without having
    side effect , people could take a signal.
    Will probably be back to get more. Thanks

  • Robertpounc

    “I was excited to discover this website. I need to to thank you for ones time just for this fantastic read!! I definitely loved every part of it and I have you book marked to see new information in your blog.”
    דירות דיסקרטיות בבת ים

  • http://tinyurl.com/yjtpbd9a

    It’s not my first time to go to see this web page, i am visiting this website dailly and take pleasant facts from here daily.

  • web hosting

    I enjoy reading through a post that will make people think.
    Also, many thanks for allowing for me to comment!

  • tinyurl.com

    Do you have a spam issue on this site; I also am a blogger,
    and I was curious about your situation; we have created some nice procedures and we are looking
    to trade techniques with other folks, be sure to shoot me an email if interested.

  • web hosting

    I have to thank you for the efforts you’ve put in penning this website.
    I’m hoping to view the same high-grade content from you later on as well.
    In truth, your creative writing abilities has motivated me to get my own blog now 😉

  • tinyurl.com

    Thanks a bunch for sharing this with all people you really realize what you are speaking approximately!
    Bookmarked. Please additionally consult with my web site =).
    We will have a hyperlink change contract among
    us

  • web hosting

    Remarkable things here. I’m very glad to look your post.

    Thank you so much and I am taking a look forward to contact you.
    Will you kindly drop me a mail?

  • minecraft games

    Marvelous, what a website it is! This web site gives valuable
    data to us, keep it up.

  • tinyurl.com

    Please let me know if you’re looking for a writer for your site.
    You have some really good posts and I think I would be a good asset.

    If you ever want to take some of the load off, I’d absolutely love to write some content
    for your blog in exchange for a link back to mine. Please blast me an e-mail if interested.
    Cheers!

  • Isaiah Yerbich

    I do not even know how I ended up here, but I thought this post was great. I don’t know who you are but certainly you’re going to a famous blogger if you are not already 😉 Cheers!

  • Kayce Gorence

    you are really a good webmaster. The web site loading speed is amazing. It seems that you’re doing any unique trick. Furthermore, The contents are masterpiece. you have done a wonderful job on this topic!

  • 0mniartist

    I am really impressed with your writing skills as neatly as
    with the layout on your weblog. Is this a paid subject matter or did you modify it yourself?
    Anyway stay up the excellent high quality writing, it is uncommon to peer a nice blog like this one today..

    0mniartist asmr

  • asmr https://0mniartist.tumblr.com

    I think this is among the most important information for me.

    And i’m glad reading your article. But should remark on few general things, The site style is great, the articles is really great :
    D. Good job, cheers 0mniartist asmr

  • asmr https://0mniartist.tumblr.com

    I all the time used to read piece of writing in news papers but now as I am
    a user of web therefore from now I am using net for articles or reviews,
    thanks to web. asmr 0mniartist

  • signup.com

    I like the valuable info you provide in your articles.

    I will bookmark your weblog and check again here regularly.
    I am quite certain I will learn plenty of new stuff right here!
    Good luck for the next! asmr 0mniartist

  • http://bit.ly

    Please let me know if you’re looking for a writer for your site.
    You have some really great articles and I feel I would be a good asset.
    If you ever want to take some of the load off,
    I’d really like to write some material for your blog in exchange for a link back to
    mine. Please shoot me an e-mail if interested.
    Kudos! asmr 0mniartist

  • zortilo nrel

    I enjoy your work, appreciate it for all the interesting content.

  • this gamefly

    I don’t know whether it’s just me or if everyone else experiencing issues with your website.
    It appears as if some of the written text in your posts are running off the screen. Can someone
    else please provide feedback and let me know if this is happening to them as well?
    This might be a issue with my web browser because I’ve had this happen previously.
    Cheers

  • asmr an

    Hmm it seems like your site ate my first comment (it was super long)
    so I guess I’ll just sum it up what I had written and
    say, I’m thoroughly enjoying your blog. I as well am an aspiring blog writer but I’m still new to the whole thing.
    Do you have any recommendations for first-time
    blog writers? I’d certainly appreciate it.

  • the asmr

    Hey! I just wanted to ask if you ever have any issues with hackers?
    My last blog (wordpress) was hacked and I ended up losing a few months of hard
    work due to no data backup. Do you have any methods to prevent hackers?

  • asmr off

    Thank you for the auspicious writeup. It in fact was a amusement account it.
    Look advanced to far added agreeable from you! However, how can we communicate?

  • there gamefly

    Have you ever considered about adding a little bit more than just your articles?
    I mean, what you say is fundamental and all. But imagine if you added some great graphics or video clips to
    give your posts more, “pop”! Your content is excellent but
    with images and clips, this site could undeniably be one of the most
    beneficial in its field. Amazing blog!

  • your asmr

    Attractive part of content. I simply stumbled upon your site and in accession capital to say that I get in fact loved account your weblog posts.

    Any way I will be subscribing for your feeds and even I achievement you access constantly quickly.

  • j.mp

    Your mode of explaining everything in this post is actually pleasant,
    all be capable of easily be aware of it, Thanks a lot.

  • asmr was

    I do not know if it’s just me or if perhaps everybody else encountering problems with your website.
    It looks like some of the written text in your posts are running off the screen. Can somebody else please provide feedback and let me know if this is happening to
    them as well? This could be a issue with my browser because
    I’ve had this happen previously. Thank you

  • Verla

    Normally I do not read post on blogs, however I wish to say that this write-up very compelled me to take a look at and do it!

    Your writing style has been surprised me. Thank you, very nice article.

  • asmr is

    Attractive element of content. I just stumbled upon your web site and in accession capital to claim that I
    get actually loved account your blog posts. Anyway I’ll be
    subscribing in your feeds and even I fulfillment you get right of entry to persistently fast.

  • minecraft download

    The other day, while I was at work, my sister stole
    my iphone and tested to see if it can survive a thirty foot drop,
    just so she can be a youtube sensation. My iPad is now destroyed and she has 83 views.
    I know this is completely off topic but I
    had to share it with someone!

  • storno brzinol

    naturally like your website however you have to take a look at the spelling on several of your posts. Several of them are rife with spelling issues and I to find it very bothersome to inform the reality nevertheless I will definitely come again again.

  • http://j.mp

    scoliosis
    At this time I am going away to do my breakfast, after having my breakfast coming over again to read additional news.
    scoliosis

  • http://bitly.com/2RlwdXi

    scoliosis
    I think this is among the most important information for me.
    And i’m glad reading your article. But want
    to remark on few general things, The site style
    is ideal, the articles is really excellent : D.
    Good job, cheers scoliosis

  • http://bitly.com/

    scoliosis
    I am now not certain the place you are getting your
    info, but good topic. I must spend some time learning much more or figuring out more.
    Thanks for magnificent info I used to be in search of this info for my mission.
    scoliosis

  • https://785days.tumblr.com/

    free dating sites
    It’s enormous that you are getting thoughts from this paragraph as well
    as from our argument made at this time. free dating sites https://785days.tumblr.com/

  • https://6869milesoflove.tumblr.com/

    free dating sites
    Link exchange is nothing else but it is just placing the other person’s
    website link on your page at suitable place and other person will also do same for you.
    dating sites

  • asmr there

    Thanks to my father who stated to me concerning this
    blog, this weblog is really amazing.

  • dating sites your

    It’s perfect time to make some plans for the future and it’s time to be happy.
    I have read this post and if I could I want to suggest you some interesting things or tips.

    Maybe you could write next articles referring to this article.
    I want to read more things about it!

  • on free dating sites

    Great blog! Do you have any suggestions for aspiring
    writers? I’m planning to start my own website soon but I’m a little lost on everything.
    Would you suggest starting with a free platform
    like WordPress or go for a paid option? There are so many choices out there that
    I’m totally confused .. Any tips? Kudos!

  • our free dating sites

    I must thank you for the efforts you have put in penning
    this site. I’m hoping to see the same high-grade content by you in the future as well.
    In fact, your creative writing abilities has encouraged me to get my
    very own blog now 😉

  • with scoliosis

    Thanks for sharing your thoughts about or. Regards

  • that scoliosis surgery

    Hi there, this weekend is nice in support of me, for the reason that this
    time i am reading this great educational article here at my house.

  • on free dating sites

    I read this piece of writing fully regarding the resemblance of
    most up-to-date and previous technologies, it’s remarkable article.

  • free dating sites there

    Having read this I believed it was rather enlightening.
    I appreciate you spending some time and energy to put this information together.
    I once again find myself spending a lot of time both reading and leaving comments.
    But so what, it was still worthwhile!

  • http://tinyurl.com

    I know this website presents quality depending articles and other data, is there any other web page which offers
    these things in quality?

  • tinyurl.com

    Have you ever thought about including a little bit more than just
    your articles? I mean, what you say is fundamental and everything.
    However think of if you added some great photos or videos to give your posts more, “pop”!
    Your content is excellent but with pics and videos, this website could certainly be one of the most beneficial in its niche.
    Wonderful blog!

  • Nathaniel

    Hi to every body, it’s my first go to see of this weblog; this website includes awesome and actually fine information in favor of
    visitors.

  • mpc-install.com

    I could not refrain from commenting. Exceptionally well written!

    My webpage: mpc-install.com

  • http://www.atomy123.com/forum.php?mod=viewthread&tid=326148

    Excellent website. Lots of useful info here.

    I’m sending it to a few pals ans also sharing in delicious.
    And naturally, thank you to your effort!

    my web page :: http://www.atomy123.com/forum.php?mod=viewthread&tid=326148

  • pkv games

    Awesome pοst.

    my web-site; pkv games

  • Nuubu

    Of course, what a great website and instructive posts, I definitely will bookmark your site.All
    the Best!

    my web blog; Nuubu

  • mpc-install.com

    Its wonderful as your other content :D, regards for posting.

    My page … mpc-install.com