Machine Learning
I chose to research machine learning/deep learning and how it is used online and in animation/film. Machine learning is the process of applying AI to certain systems to learn and improve by themselves without having to be programmed. The aim is to make computers learn automatically without human intervention. On the other hand, Deep Learning is a function of AI, mimicking the human brain in how it processes data and creates patterns to be used in decision making.
Deep learning and AI techniques can cut time and cost in creating CGI and VFX to increase accessibility. Even though technology has evolved, 3D animation/CGI/VFX is still very time-consuming and quite expensive, and much of the software used is not too easy to navigate. A lot of software is constantly developed by companies in-house and you can only access it if you work for one of those companies. Modelling, texturing and rigging is the real deal and it takes a very long time to learn (and apparently is a massive pain to do). 1 minute of CGI for Game of Thrones = 1600 hours of human labour = $80k
There is such a thing called intelligent animation. It is essentially speeding up and automating the process of animation and rigging, to cut time and costs. Creating new characters is the most challenging job for an animator. General Adversarial Networks, GANs (or Neutral network technology) can speed this up by learning from an existing set of characters. Deep learning and AI can learn about the shape of the face or body from an enormous array of characters, it can also learn about movement – from dancing to fighting, to a simple conversation.
One of the most challenging obstacles in animation is simulating the flow of light in a three-dimensional scene and convert the information into a two-dimensional image. It is a labour-intensive, time-consuming undertaking, and deep learning can help slash time and costs. Disney and Pixar have been collaborating to develop new AI-based tech to eliminate noise at higher speed renders. This new tech removes noise while preserving the detail in their scenes. https://www.news.ucsb.edu/2017/018150/intelligent-animation
A current example of new tech capturing light and texture and altering faces without animation would be The Irishman by Martin Scorsese, recently released on Netflix. ILM (Industial Light & Magic) developed a new light-based performance capture software called FLUX and a special camera rig to deform and de-age faces with rendering and compositing, without going through traditional keyframe animation. Scorsese didn’t simply want to de-age the actors as a younger version of themselves, but he wanted ILM to de-age the actors as a younger version of their characters. A very subtle but important difference, also without using any tracking markers or head rigs, so their performance wouldn’t be interfered with. Linked to these techniques are AI-based facial recognition technology.https://www.indiewire.com/2019/12/the-irishman-ilm-vfx-de-aging-1202194908/
These facial de-ageing and deforming techniques can also be applied to something more sinister, such as Deepfakes. Deepfakes are images or videos of people whose likeness are replaced by someone else’s likeness using AI-based neural networks. They combine/superimpose media onto source media using GANs and autoencoders. One can literally make someone look like someone else and make it look real. Deepfakes have been used for financial fraud, revenge porn, fake news and hoaxes. There are many apps for phones, such as face swap on Snapchat , which use similar techniques, but they are not as precise. The Youtube channel Control Shift Face has great examples of perfect deepfakes. https://www.digitaltrends.com/cool-tech/ctrl-shift-face-deepfake-changing-hollywood-history/
Something to look into more would be deep learning in games. I read about kinematica, a package developed by Unity Labs. The aim is to create a game without having to worry about building complex animation graphs and remove manual labour, letting animators focus solely on the art for a game. https://blogs.unity3d.com/2018/06/20/announcing-kinematica-animation-meets-machine-learning/
Another aspect of machine learning that I stumbled upon is machine learning fairness, which is machine learning applied to online search engines. It is essentially censorship, and censorship begins in the media. The silicon valley elite introduced machine learning fairness which is designed to eliminate hate speech. Online search engines are programmed to automatically recognise as ‘hate’ all expressions of opinion which violate some form of political correctness. An Orwellian vision which is now in practice, it is the manipulation of language so as to make heresy inexpressible. Sadly, the result is not a culture of ‘niceness’ but it’s a constant assault on those who are supposedly ‘preventing’ it. Images, like language, are manipulated as well. https://unherd.com/2019/09/how-identity-politics-drove-the-world-mad/
Luca's research blog 