There is no doubt that one of the great phenomena that have shaken the world —and, especially, the networks— in recent months has been the boom artificial intelligence. In a matter of weeks, tools like Dall-E or Chat GPT became the viral sensation of the moment flooding our feeds with countless text-generated images, debates about the meaning of the concepts of art and artist, and particularly apocalyptic omens.
The reality is that, if we focus on the field of audiovisual creation —and, more specifically, on the post-production phase—, AIs have been with us for a long time making our lives much easier integrated in software as popular as Photoshop, After Effects, Premiere Pro or Final Cut Pro. However, these aids, with a very discreet profile, are being eclipsed due to a technological evolution that has reached dizzying speeds.
One of the latest signs of the limits to which these processes are reaching leaves the NVIDIA Broadcast update in its infancy, which makes our eyes look directly at the camera even though they are focused on another point. It is created by the company FlawlessIA and it opens up a future full of possibilities that will probably make more than one of them’s hair stand on end, but which could mark a before and after for film studios and dubbing actors and actresses.
fucks for Frick’s
In 2022, Scott Mann released his fourth feature film under the title ‘Fall’, a survival thriller that went through the last edition of the Sitges Festival in which two women get trapped 600 meters high after climbing a tower communications. Fortunately or unfortunately, the film starring Grace Fulton and Virginia Garden earned an R rating for his foul language, which included a whopping 35 “fucks” in its original version.
The problem to extract profitability from productions with ratings for adult ages caused the distributor Lionsgate to demand a softened cut to reach a PG13, much more accessible and easy to market. To make it, the MPAA required that there be only one use of the word of yore in all the footage; a real headache that could translate into a disaster in the editing room or a considerable outlay to shoot reshoots of certain scenes. And then the AI came along.
Scott Mann, who has combined his career as a filmmaker with medium-focused artificial intelligence research in his startup FlawlessIA found with ‘Fall’ the perfect excuse to test the system it was developing: a software that allows you to manipulate the faces of the performers and alter their facial expressions and mouth gestures to fit the dialogue.
In this way, and as can be seen in the video on these lines, Mann and his team modified the “fucks” and changed them to “fricks” —something like the equivalent of exchanging our “hosts” for “oysters”—; saving the ballot and allowing the film, budgeted at 3 million dollars, to raise more than 7 in the United States and exceed 17 million worldwide. they only needed summon the actresses to the studio, record a few modified dialogues and let the AI do its magic.
Behind the work of FlawlessIA is a very powerful tool that, in addition to increasing the room for maneuver for studios and filmmakers once principal photography is finished, is called to change the world of dubbing forever. Anyone who has ever stood behind the lectern to lend their voice to a character interpreted in another language will know that one of the greatest difficulties lies in a synchronization that could have the days numbered.
So that the dubbing is as natural as possible and does not mislead the viewer, the syllables and blows of the voice must adjust as much as possible to the movement of the lips of the interpreter on the screen. To do this, the dubbing actor or actress must measure the speed of her interpretation, sometimes being forced to repeat takes until finding the perfect “synchronous”. It is already known that in the audiovisual industry, time is money, and Mann’s work could bring great advantages in this regard.
Let’s calm down
Although many may, not without some reason, succumb to the temptation of giving in to catastrophizing and prophesying an almost dystopian near future and, taking into account that, as we mentioned earlier, the irruption of AI in the medium is not something recent, it is rigorous focus on the benefits that this technology can bring like this to sectors involved in film or television productions.
On the one hand, we find the studios and the men and women in suits and briefcases, who would have an extensive safety net when investing in projects. Factors such as potential savings on reshoots either the option of offering versions with different ratings by age focused on the language without excessive extra costs are two good examples of this.
On the other hand, the workers in the world of dubbing would also benefit, beginning with some translators who could locate texts much more easily without having to adjust syllables and search for synonyms and phrases that match the original lipsticks. In addition, the actors and actresses could focus their efforts on the interpretation —despite having to continue, with greater freedom, pending tempos and cadences—, which could translate into better results on a dramatic level.
Of course, there is still a lot to do and polish – the uncanny valley continues to make an appearance – but that is the least of it when talking about topics like this. The real problem lies in the use that can be given to this type of utility at the sociopolitical and that, if we get conspiratorial, could leave in diapers the horrors of post-truth.