How to combine real footage with CGI in iClone

In addition to making animations, iClone can be used to facilitate the compositing process by enabling you to add backgrounds or billboards, textured with real footage, to your 3D models. The tool reduces the number of meshes needed to create complex sets, while at the same time ensuring the scene is highly realistic.

See how the process works in the video below, or read on for a full explanation and some expert tips for taking advantage of this feature in your own projects. All these methods are surprisingly straightforward and flexible, allowing you to make adjustments on the fly.

The video footage or still images used as textures can be edited like any other material used in your project. This means it’s possible to define masking areas, making part of the billboards transparent. If you're using tessellated planes, you can also include displacement maps to create depth deformation in your surfaces.

Find out more about iClone

As all the elements are part of the same project, the lights within the scene will automatically interact with the billboards – so the billboard within the scene will cast shadows over its surrounding 3D models, and vice versa, depending on where the lights are placed.

Finally, it's possible to set the billboards so they're light-emissive. This enables you to illuminate the CGI objects within the set using a light that perfectly matches the images they contain. For example, you could simulate the flickering light emitted by screens or TVs very easily, but it's also possible to illuminate the 3D models in a very convincing way using real footage of explosions, fires or atmospheric effects. Particle effects and atmospheric effects can also be included in the scene, and similarly interact with the billboards.

Being able to mix video billboards with 3D models is very useful. We can control the effect of the lights and the cameras, observing the interaction between all the elements of the scene in real time. Working with iClone, what you see is always what you get, so we don’t need to wait for slow renders and further compositing operations to be sure that the shots have been properly arranged.

Working like this, the whole scene can be rendered in a single step, and there is no need to create render layers for compositing. It's an excellent way to speed up animations, saving a lot of time and effort in post-production.

Here are some top tips to help you make the most of iClone's compositing tools.

01. Use videos as a texture

iClone enables you to use videos as a texture by simply dragging and dropping the video file over a billboard prop. Using these panels with real footage as background and CGI props in the foreground means that we can move the camera to create a cool parallax effect. Set these panels to be self-illuminated, as well as not receiving and not emitting shadows, so they're unaffected by the light conditions of the scene. This way, they can be easily configured to match the original illumination of the footage.

02. Combine videos with transparent areas and CGI props

Images containing alpha channels can be loaded as a texture in iClone billboards. The transparent areas of the picture will also be transparent within the billboard, enabling you to create dioramas that mix CGI props, characters and particles with several layers of panels that combine still pictures with transparent areas and video footage.

Again, you can create a cool parallax effect when moving the camera through the scene, and some of the panels can be set to be affected by the light of the particles to integrate them more effectively.

03. Use panels with video to illuminate scenes

By setting the billboards as self-illuminated and increasing their Global Illumination parameters, they become a source of light within the scene. This way we get a perfect integration between the background videos and the light in the scene, especially when this light changes during the sequence, like in this example of a sci-fi city on a stormy night.

Combining 3D models with light emissive billboards, rain particles and panels texturized with semi-transparent fog animations, we can obtain integrated results in one single step, with no need for any extra compositing.

04. Mimic camera motion from real footage

iClone enables us to set a video as a fixed background. As it’s not affected by the iClone’s camera movements, we can track and simulate the original camera that filmed the video using dummy characters placed in strategic areas of the footage. Once the iClone’s camera mimics the background footage, we can add CGI elements that will integrate naturally into the scene, thanks to the situation and proportion provided by the dummy characters.