From Lidar scans to UnReal VR/AR production

Rapidly advancing technology has grown to become an important part of the creative industries. Exciting new developments are changing the way we create and connect, from Artificial Intelligence to Virtual Reality, the possibilities for innovation and experimentation are vast. 

Founder of Virtual Brooklyn, Chris Stewart, gave a fascinating presentation on what it takes to build a virtual set from real life. This presentation covered the different kinds of scans and software needed to make a virtual set. These scans can be useful in many different fields, such as film, live television, VR games and much more! He took us through the various different equipment and software needed for such a feat, including Lidar scanning (light detection and ranging), building scenes in Maya, real-time 3D creation with Unreal Engine and Brainstorm Production Software.

The key to creating a realistic virtual set - which can easily be switched between its real life counterpart - is a combination of exact lighting replication, mapping the real-life set from many different points and using textures to seamlessly mimic reality. This software shows the possibilities of technology within the creative industries, and how transformative it can be. 

Now, virtual sets can even adapt in real-time, meaning that the boundaries of what live TV can be are rapidly expanding. In particular, for live television, a 3D virtual studio set allows real objects and people to be combined with computer-generated environments in real time. Chris explained that within a virtual set, the camera can be moved around it, as the software calculates the space. Essentially, the software allows virtual set production designers to turn data into visual sets.

To map a set, the 3D laser scanning software scans the set and creates data points. These data points are then used alongside high dynamic range images, 360 degree images and texture images in order to ‘build’ the set virtually, as it is experienced in real life. Even complete textured 3D models built to exact scale can be created. Precision is key here. The lighting needs to be perfectly replicated - it’s hard to tell the difference between the real-life set and the virtual one. Chris ensures that all angles and heights are scanned, and creates fly-throughs that show all areas are properly documented. He may not be able to return to the location again, so it is pertinent that he gets all of the data needed. 

Technology uses the real world and turns it into data points. These data points are used to create geometric shapes which can then be translated into recognisable images. Pictured below is how the Maya software turns its data into a reality capture output. From here, Chris uses a combination of different images to make sense of and map out what is happening within the set.

None of this was possible just a few short years ago; technology is rapidly changing the creative industries, allowing for expansive possibilities. Studio sets can be changed within an instant - allowing creative freedom and saving time. It is certain that tech has taken us a long way, we can only watch where it will take us next!