Five Academy Awards for Cool Science and Engineering

This year's Oscar Sci-Tech awards go to new technologies that seamlessly blend virtual worlds with the real one.
Media credits

Image courtesy of Chaos Group

Emilie Lorditch, Staff Writer

(Inside Science) -- It's Oscar season, but before the bold and the beautiful, the A-list hopefuls and the Hollywood elite descend onto the red carpet for the Academy Awards main stage event, a lesser-known ceremony will take place this Saturday to recognize imaginative scientists and innovative artists. Each year The Academy of Motion Picture Arts and Sciences presents its annual Scientific and Technical Awards to those who weave their magic behind the scenes and help to captivate audiences. 

The awards will be presented on Saturday, Feb. 11, 2017 at the Beverly Wilshire in Beverly Hills. Actress Leslie Mann and actor John Cho will host this year's ceremony, which will award 34 recipients and 5 organizations with 18 separate awards for scientific and technical achievement.

Here is a sampling of this years' award winners, along with descriptions of how the respective technologies work from the engineers and artists who bring them to life.

1. Industrial Light and Magic (ILM) Facial Performance-Capture Solving System

The ILM facial performance-capture solving system at work on Teenage Mutant Ninja Turtles

Image courtesy of Lucas Films LTD

"ILM's facial performance capture solving system is a way for us to accurately capture an actor's performance on set, and transfer all the motions and nuances onto a creature, like the Hulk or an Orc," said Michael Koperwas, Creature Modeling Supervisor at ILM.

"The actor wears a helmet with two tiny cameras attached. We paint tiny dots, similar to freckles, on the actor's face, and track those so we can recreate the performance on a digital version of the actor. We usually sculpt the digital actor into a creature, which helps us transfer everything onto the creature. We have tools to allow for editing, which can mean anything from raising the eyebrows more, stitching multiple takes together, or inserting completely animated portions." 

"Seeing that character all put together makes it suddenly feel alive, and reminds us of the magic we're making," Koperwas said.

You've Seen This In: "Rogue One," "Teenage Mutant Ninja Turtles 1 and 2," "The Avengers" and "The Force Awakens." 

2. V-Ray from Chaos Group


Using V-Ray in The Walk – Atomic Fiction © Sony Pictures

Images courtesy of Chaos Group

"V-Ray is a rendering engine, that is, a piece of software that takes a digitized description of a 3-D world -- the objects in it, their materials, the light sources and the camera angle -- and calculates a photorealistic rendering of that world," said Vladimir Koylazov, one of the original developers of V-Ray at Chaos Group.

V-Ray is based on a technique called "raytracing," which accurately simulates the way light propagates in the virtual world to produce a believable image, often indistinguishable from taking an actual photograph.

"The whole point is that our work should be invisible to the public," Koylazov explained. "In the context of film, good CG [computer graphics] is one that people don't really notice, but behind all those fantastic worlds, there is software that crunches a lot of numbers in order to produce the final results."

You've Seen This In: "The Avengers," "Captain America: Civil War" and "Doctor Strange," and also smaller films like "The Walk."

3. Concept Overdrive Motion Control System

Concept Overdrive Motion Control System

Image courtesy of Concept Overdrive

"The Overdrive system is a toolkit for on-set mechanical and virtual motion, and for solving problems between the virtual world and the real world," said Steve Rosenbluth, President and CEO of Concept Overdrive. "If the camera operator can't see the interaction between the actors and the virtual world, he has to guess the direction of the camera or where to shoot. If we add a digital character later, the shot may look awkward. Overdrive helps solve that problem -- we help the operator frame the shot."

"In 'The Jungle Book,' Mowgli is riding on an animal like Baloo but it is actually an Overdrive robot that moves Mowgli's spine correctly while motion-streaming into the virtual world," Rosenbluth said. "Without the Overdrive system, combining the virtual and real world wouldn't look as life-like as it does on screen."

"Eighty percent of the shots in the film 'Avatar' passed through the Overdrive system," Rosenbluth said. "Science is ever-present in Hollywood through the unique collaboration of artists and technicians working together to solve a problem on the fly."

You've Seen This In: "Rogue One," "The Jungle Book" and "Avatar."

4. Animation Rig-Based Facial Performance-Capture Systems at ImageMovers Digital and Digital Domain

An example of the Animation Rig-Based Performance-Capture Systems and V-Ray working together in Maleficent.

Image courtesy of Chaos Group

"We can take an actor's facial performance and apply it to a digital character," said Nicholas Apostoloff, a former senior research engineer at ImageMovers Digital and a principle engineer with Digital Domain. "We capture every nuance of the actor's facial performance at the time of the production so that it can be faithfully recreated in the digital character." 

"The actor wears a head mounted camera (HMC) which records their facial performance while they are acting," said Geoff Wedig, a former senior software engineer at Digital Domain. "This data is analyzed using the facial performance capture system to convert it into commonly used facial poses -- some examples include 'jaw open,' 'lips pucker' and 'left eyebrow raiser.' Every expression from smiles and smirks to surprise and shock are captured. These poses are actions which the digital puppet rig understands and correspond to the animation controls an animator would typically use to create the facial performance on a digital character." 

"In 'Disney’s A Christmas Carol,' there is one particular shot where Jim Carrey falls face first on the ground," Apostoloff said. "In this shot, the head mounted camera he was wearing moved so much that only about a third of his face was seen by the cameras. The facial performance capture system was so powerful that a believable performance was reconstructed even though only a small portion of his face was seen!"

You've Seen This In: "Disney’s A Christmas Carol," "Tron: Legacy," "Maleficent," "Deadpool" and the upcoming film "Beauty and the Beast."

5. Viper FilmStream Digital Camera System

The Viper FilmStream Digital Camera System

Image courtesy of Grass Valley

"The Viper FilmStream camera uses three independent red, green and blue full-resolution image sensors receiving the image through a very high-quality lens and prism that evenly split the light to each of the individual sensors," said Mark Chiolis, regional account manager at Grass Valley.

"The Viper FilmStream camera was the first camera to use raw images directly from the sensor without any processing, just like capturing from a film negative, without any destructive permanent processing that would limit quality and range of correction in post-production."

Today, many still cameras, video cameras and even some phone cameras make it possible to shoot raw images that can be processed later if needed, but Viper was the first production model camera to do this. Viper also helped saved time on set. 

"What used to be called 'dailies' [the first views of the daily raw footage that was filmed] was coined 'instantlys' or 'nightlys' depending upon when this took place," Chiolis said, referring to this advance. "Having the capability to capture a visible electronic file image also allowed the director of photography to make notes in real-time and to send those faithfully reproduced images to the post facility for the colorist to move ahead with their portion of this if needed, once again allowing for faster completion of the project."

"Almost everything about the Viper was a unique challenge as it was radically different than both film, which was the current standard in Hollywood for television and movie production, and the current first generation digital capture camera from Sony," Chiolis said.

You've Seen This In: "Collateral," "Miami Vice" and "The Curious Case of Benjamin Button."

Author Bio & Story Archive

Emilie Lorditch is the former Assistant News Director at AIP.