Module introduction
On the first day of this module, we were introduced to the course. Together with the class, we went through the brief and what we need to submit. The assignment in this module was to create a 3D object, either a building or a machine, and to seamlessly integrate it into moving backplate. This means that the render had to be as photorealistic as possible – the lighting, texturing and overall style should fit the plate to the artist’s best ability. The software used in this module would be as follows – Nuke for clip conversion and image sequence export, 3DEqualizer for camera tracking, match-moving and camera solve export, Autodesk Maya for 3D modelling and UV editing, Substance Painter for texturing and shading work, Arnold Render under Maya for rendering, and finally, Nuke once again for final slap comp that will be submitted as the end product for this module. Furthermore, the product we create in this module will be used in the next module, VX5002 – Compositing for Visual Effects.
Introduction to 3DEqualizer
3DEqualizer is one of the best 3D tracking solutions on the market today. While the UI and environment can seem a bit daunting at first, the program is very intuitive and the workflow is well designed. All quirks and inconveniences aside, it has the best algorithms and coding for solving camera and making 3D tracks, as well as calculating lens distortion, focal length and sensor size if the department does not have these details on hand.
Why is matchmoving needed?
Matchmoving is needed if there are any shots that require putting 3D/2D objects into a 2D plate. An artist has to manually track makers or patterns on the plate in order for the program to properly solve the camera. The easiest camera moves to track are the ones where parallax is obvious and depth of the environment being shot is clearly visible. A proper 3D camera track provides an artist with data about camera movement – what is the position of the camera at a given time, what is its orientation and physical details of lens and camera sensor. With this data, the artists can then place their objects in the 3D scene using 3D package of their choosing.
Tracking in 3DEqualizer
3DEqualizer requires at least 6 tracking points to be present in the shot at a given frame in order to get a camera solve. Creating tracking points is simple – control + left mouse click creates a tracking point, which then needs to be set to either pattern or marker tracking point. However, as you would imagine, if you have more tracking points, you can get a more accurate camera solve. A higher number of tracking points also help to better calculate the lens distortion profile, focal length and filmback height.
Solving 3D camera in 3DEqualizer
When solving camera, 3DEqualizer calculates the position of the camera using the position of tracking points on the plate and distributes them in 3D space based on the difference in their motion (speed, scale change, parallax, etc.) relative to the camera. Calculating a camera is done through ‘Calc’ – ‘Calculate all from scratch’ or a handy Alt + C shortcut.
3D camera tracking is an iterative process. It usually takes many small steps to complete. After solving the camera, 3DEqualizer displays a deviation curve for each point used to get the solve. What this means is that when a camera is 3D solved, each tracking point should stay exactly on the same coordinates in 3D space. Any deviation will be displayed as a curve in an XY graph, where the X field is the current frame number and the Y field is the deviation value in pixels. All of the curves for each point are then averaged into a single curve, which gives the user an idea of how good their track is. With this graph, the user can easily remove bad tracking points and get a better average curve.
Plate selection and 3D proposal preparation
On this day, we selected three plates we had an idea for. We were supposed to make a 3D proposal for three shots we intended to work with.