Evolution of VFX and the making of Blade Runner 2049
Our CTO Graham Jack on the Content Makers Stage at #WebSummit2017. Watch his presentation about the evolution of VFX and the making of Blade Runner 2049.
Transcript
00:00So, my name is Graeme Jack, I'm the Chief Technology Officer at Double Negative Visual
00:23Effects.
00:25I'm going to be talking a little bit about the work that we did recently on Blade Runner
00:292049.
00:30Did anyone see the movie?
00:31A show of hands who saw the movie?
00:34So I'm going to try not to spoil it for those of you that haven't seen it, because you should
00:38all definitely go to see it.
00:41So before I get into Blade Runner, I'm just going to talk a little bit about the company
00:45and where we came from, and I'm going to do a little bit of a kind of very brief roundup
00:50of the history of visual effects.
00:54So Double Negative started about 20 years ago, just under 20 years ago.
01:00The founders were very passionate about film, and the core of our business has always been
01:04feature film visual effects.
01:07We've always tried to take a very collaborative approach with our clients.
01:13We try to create more of a creative partnership than just providing a service.
01:18And that kind of alongside an innovative but pragmatic approach to technology has
01:24been very successful for us, and the company has gone from strength to strength.
01:28Along the way, we've grown massively.
01:30When I started the company, there were about 30 people.
01:33The managing director would welcome everyone new to the company over beer and pizza in
01:38the kind of common room when they started.
01:41Now we're about 3,000 people worldwide.
01:43We've got facilities in London, Vancouver, Mumbai.
01:47We've got a facility in LA, just opened a new studio in southern India in Chennai, and
01:54we're opening up in Montreal soon.
01:56Along the way, we've been lucky enough to win a few awards, a few Academy Awards.
02:01So the company is doing very well.
02:05So I thought I'd talk a little bit about where visual effects has come from and the technology.
02:12I think visual effects sits at this interesting crossroads between creativity and technology,
02:19and that's what appealed to me about it.
02:20My background really was engineering.
02:22I did an engineering degree, but I always looked for something that would allow me to
02:27merge some artistic control into that as well, and visual effects gave me that opportunity.
02:36So right back in the early days, the technology of visual effects really was the technology
02:41of filmmaking.
02:42There was no difference.
02:43All of the visual effects processes were really processes of photography, double exposures,
02:49things like that.
02:50But over time, especially with the advent of computers, things have diverged.
02:55So a nice early example of technology in visual effects or processes in visual effects is
03:02something called the Shaftan process.
03:04So this was used in very early filmmaking in movies like Metropolis, which was a huge
03:09influence on both Blade Runner movies, and it involves placing...
03:15You build a miniature, and you want to bring some life to that miniature by placing actors
03:19within it.
03:20So you put a piece of glass at an angle in front of your miniature, and you have a bunch
03:24of actors off to one side, and you light them in such a way that their reflection shows
03:28up in the mirror, in the piece of glass, and you're filming the set through that piece
03:34of glass, which brings life to the miniatures.
03:39And that effect, that process, is borrowed from something that you may well have heard
03:44of called the Pepper's Ghost Illusion, which comes from theater and stage magic and was
03:49being used for years before filmmaking or visual effects was a thing.
03:55And this trend of visual effects sort of borrowing technology from other fields is something
04:00that we continue to see to this day.
04:05And that idea of using glass and filming through glass is something that in later years, that
04:11process was evolved so you would paint onto the glass, the origin of the term matte painting,
04:15so that you could take a scene that you were filming and expand it and extend it to develop
04:21this broader world beyond it.
04:24And that's a technique that was used extensively on the first Blade Runner movie, which kind
04:29of brings me up to the present and our Blade Runner movie.
04:33So I'm going to start off just by talking a little bit about the design process on the
04:37movie and how do we create the world, because you're really starting from nothing.
04:44This is a scene from the original movie, and we wanted to do justice to the original, but
04:56at the same time, when we looked at dystopian cityscapes in movies since Blade Runner, they
05:02all kind of leant heavily on the original, and we really wanted to do something different.
05:07We wanted to feel like we were seeing an expanded world from the original movie, but we didn't
05:12want it to feel just like a copy, just like a bigger version of Blade Runner.
05:17So we needed to develop some different ideas, and we sort of started to think about how
05:21to do that.
05:22And the key thing really was to create something believable.
05:27In visual effects these days, you can pretty much make anything, but if you want to make
05:31something that an audience will believe in and engage with, it needs to have some sort
05:35of grounding in perceived reality.
05:39And so we set out to sort of find what the story and the logic was that would make the
05:43world look the way it did.
05:46And the thing we came up with really was, if you kind of think of the original movie,
05:51the sequel is set 30 years later, and the idea is that the climate has continued to
05:56deteriorate.
05:57We're really looking at a world that's broken.
05:59Humanity has broken this world.
06:01And so that's kind of the story that we were trying to develop, and we were trying to think,
06:05how does that story affect things like the architecture and the design of the city?
06:10So we went and we looked at a bunch of reference images of the real world, of kind of polluted
06:14cityscapes.
06:16And we settled on a lot of images coming from China, and as well as these kind of hazy,
06:26polluted atmospheres, the other thing that we were inspired by was some of the monumentalist
06:32architecture that we're starting to see popping up in modern China, and things like the giant
06:38LED jumbotrons that you find on some of the buildings as well.
06:44So another thing that we liked the look of on some of these images was this interesting
06:50juxtaposition of scale.
06:52You kind of see these small buildings that are residential, and they're dwarfed by high-rise
06:59apartment buildings, and then those in turn are dwarfed by these monumental structures.
07:02And we wanted to take that into the movie, but really exaggerate it, so that you can...
07:10We wanted to create these huge structures that really show how man has kind of tried
07:15to take over the world and dominate it, but it needs to be relatable.
07:20And so by creating this juxtaposition of scale, where you see these little buildings, these
07:24houses, that everyone can relate to on a human scale, and gradually ramping up the size of
07:30the buildings, we were kind of able to draw that relationship.
07:35So in visual effects, what we often do is we start with a painting.
07:38So we took a lot of those ideas from the reference material that we'd gathered, and
07:43this was one of the early paintings that was quite liked.
07:46You can see it's got those kind of low-lying buildings.
07:49It's got some of the haze.
07:50It's got the high-rises, and then there's just a hint of some massive megastructure
07:56off to the side.
07:57But it didn't quite capture this feeling of a broken world in the way that the director
08:02wanted.
08:03So we kind of developed it further.
08:06This is kind of where we ended up with.
08:07This was something that the director really liked.
08:11It kind of has this cold feel, like LA has become this cold, fog-drenched, snowy landscape,
08:19and it really feels like a world that's dying.
08:23So from there, when the shoot finished, we had to try and figure out how we were going
08:27to translate that into what we see in the movie.
08:33And I'm going to talk a little bit about that process and how that influenced some of our
08:36technology choices.
08:39So here's a piece of concept art.
08:42This is a sort of early concept for one of the shots in the movie.
08:48And you can kind of see there's quite a lot of detail in there.
08:52There's all of this street-level lighting.
08:55And so if I move on to the next one, this kind of shows how that shot is assembled.
09:01So you can see there's quite a lot of complexity in there.
09:04A lot of buildings, and then on top of those buildings, there's extra detail coming in
09:11that gives you things like fire escapes, air-conditioning vents, things like that.
09:19And even though when you get to the final shot, a lot of it's lost in the fog and the
09:23darkness, we wanted to make sure that we kind of retained a very realistic and gritty image.
09:30And that meant that even though it's almost imperceptible, that detail still had to be there.
09:38This is another example, another shot.
09:41So as it sort of starts to come out of this effect and into the city, you'll see that
09:46this is quite a dynamic camera move on this one.
09:49And that dynamic camera move means that we have to design the city in such a way that
09:53even when we get pretty up close to those buildings, they'll still hold up.
09:59And again, as you sort of see these breakdowns, you kind of see all of this detail coming in.
10:04And then as the lighting starts to come in, lighting and the texture starts to come in,
10:08you lose a lot of that detail.
10:11And it almost seems like, why do you bother putting it in there?
10:13But then as the illuminated atmosphere starts to come back and all the spill light from
10:18the signage, you start to perceive some of that detail again.
10:23And it's something that if it wasn't there, you maybe wouldn't notice it wasn't there,
10:26but it would make the images feel less real.
10:31So all of this detail kind of gives us a problem.
10:35It's basically a data problem.
10:37So we have these huge datasets.
10:38We have hundreds of buildings on screen.
10:42Each of those buildings made up of hundreds of components, even before you start covering
10:45it and all of the additional detail that we need to make it look real.
10:51And in some scenes, most of the shots had upward of 900 billion individual polygons
10:58in the scene at any one time.
11:02And we have to turn that 3D data into the final images.
11:05It's a process that we call rendering.
11:09And it's a very computationally intensive task.
11:12We have a data center with thousands of blade servers, all pulling data from our storage
11:20in order to be able to compute these final frames.
11:23And someone, I can't remember who it was, somebody said that visual effects rendering
11:27essentially amounts to a 24 by 7 denial of service attack on your network infrastructure.
11:32So dealing with how you get all that data off our storage and onto the screen, that
11:37was probably our biggest challenge.
11:39So we turned to a slightly different approach.
11:44We had these file formats that are designed for dealing with hierarchical data, but we
11:48actually used something that was really designed as a storage format for computational fluid
11:53dynamic simulations.
11:55So it gets used in visual effects.
11:58It was created originally by DreamWorks.
12:01It's used by a bunch of visual effects companies.
12:02It's also used by companies like SpaceX.
12:05They use it for storing simulations of rocket exhausts and things like that.
12:09And in our case, each of those individual elements within this file, rather than being
12:14an element within a fluid dynamic simulation, was actually a reference to a component of
12:19one of these buildings.
12:20So we would be able to stream this data, because it's highly optimized for streaming spatial
12:24data, stream that off our storage to render these final frames, and that was what saved
12:30us from this onslaught of data.
12:34So that's a very brief journey through some of the creative process and one particular
12:39technological challenge on Blade Runner.
12:42As I was preparing this and I was thinking about early visual effects and how far we'd
12:47come since things like the shutdown effect and early map painting, I was wondering what
12:51the future would look like and where we'd go next for visual effects.
12:57One example is manufacturing.
12:59We're starting to borrow techniques from manufacturing.
13:03As I said originally, visual effects was something that happened as part of the filmmaking process.
13:10And with the advent of computers, that changed and visual effects was something that was
13:13reactive.
13:14So the film would get shot, then visual effects comes in and we insert elements into those
13:21visual effects shots, into those plates.
13:24What's happening now as visual effects becomes more prevalent, it's kind of turning around
13:28a little bit.
13:29So increasingly we're shooting elements on set that need to be inserted into a shot that's
13:34primarily computer graphics.
13:37So we need a way to apply some of the same techniques that we would use in matching computer
13:43graphics to the real world to match the real world to computer graphics.
13:48So that's where these robots come in because they can manipulate things in the real world
13:52with extreme precision.
13:54So as an example, you can just about see the manufacturing robot in the background.
13:59This is one of our previous scenes that shows how we're programming the robot in such a
14:05way that it will manipulate the boat to react to a visual effects water simulation that's
14:11already been completed.
14:13So that's just one example of how we're still borrowing technology from other industries
14:17to move filmmaking and visual effects forwards.
14:20The other examples don't have quite so many pretty pictures, but a lot of it's just big
14:23data.
14:24When I started, we were 30 people in a company, a big complex show might have 200 visual effects
14:31shots in it.
14:32So now we're looking at projects that have 1,200 shots, we're spread across five different
14:40sites across three continents.
14:43Each shot might have 14, 15 different stages within it.
14:47So how you communicate and deal with these huge data sets, that's becoming the biggest
14:52problem in visual effects.
14:53And we're looking at what the big web companies, the Facebooks of the world are doing and being
14:59inspired by them in terms of how we kind of approach solving those problems.
15:04So thank you very much for listening to me.
15:07You know, I think it's interesting to think about when we get to 2049, what's the world
15:11going to look like?
15:12What's technology going to look like?
15:13I hope it's not going to be as bleak as the movie makes out.
15:17And I'm going to leave you just with a little reel that shows some of the other work that
15:21we completed on the movie.
15:59And I'm going to leave you with just a little reel that shows some of the other work that
16:29we completed on the movie.
16:59Thank you very much for listening.