Nuke: Stabilizing a plate with CameraTracker

Discovered today that you can easily use  a 3D camera track in Nuke to stabilize a plate – and what’s more, you can easily choose which plane of the image you want to lock down (something in the foreground, mid-ground or background).

CameraTracker Stabilize screengrab

  1. Get a good 3D camera track, masking out anything that moves first.
  2. Solve the camera and go to a frame you’d like to use for reference, probably somewhere in the middle of the sequence.
  3. Copy and paste the camera, renaming them to something like ‘CameraMain’ and ‘CameraProject’.
  4. Open the properties of CameraMain and remove any animation (right click the parameter and choose ‘No Animation’)
  5. Create a Project3D node and connect it to CameraProject. Also plug the plate into the Project3D.
  6. While viewing the plate, bring up the CameraTracker properties pane so that you can see the solved camera tracks over the plate. Choose one that sits on the plane you want to stabilize, eg on the back wall. Then right-click the point and choose ‘create – card ‘ to create a card in 3D space at that coordinate.
  7. Plug the Project3D into the card.
  8. Switch to 3D view and you should see the image projected from CameraProject onto the card. Remove all rotation from the card in the properties pane so that it’s flat on to CameraMain, then use the Uniform Scale on the card to scale it up nice and large so that none of the picture gets cut off.
  9. Now, create a ScanlineRender, plug in the card and CameraMain (your stationary camera), then view the result. If you are still on the same reference frame, the output should be identical to the plate.
  10. Play it back and you should see a stabilized image. You will then need to crop the image to remove black edges appearing. To choose a different plane to lock to, just repeat steps 6-8 using a different point from the track (eg, something from the foreground). You can do this several times and keep the cards in your script, swapping them out to find which one gives the best result. Alternatively you can just play with the Z translation of the first card until you get what you want, but choosing an actual point on a plane will probably be quicker.

So, to summarize, the viewing camera and the card do not move at all, but the image being projected moves around on the card, which matches the camera move in the shot, cancelling out the movement.

EDIT: After using this technique quite a lot for various tasks, I realise that for best results you should create a card at the angle of the plane you want to stabilise (by selecting multiple points on that plane and creating a card), rather than pick one point and remove rotation. That’s what you’d do for a frame hold patch projection, so you’d do the same here. Only difference is you swap the cameras.

This means after you stabilise, you can copy the card and project it onto the new card with the cameras swapped to match move it back to the original. Useful for creating mattes, but not advised for patches because of the slight softness produced as it breaks concatenation between the two stages. I’ve been using it recently to create rain mattes; remove the camera movement using the 3D track, Min 3 adjacent frames together to remove the bright rain drops, use a Difference node between the stablised plate and the Min, then match move using the same camera setup. Gives a nice alpha of the rain as that’s the only thing moving in the plate.

 

Advertisements

About davemne
VFX artist, developer and musician from the UK.

25 Responses to Nuke: Stabilizing a plate with CameraTracker

  1. Thanks a lot man, this is pretty crafty! Especially creating the card from a tracking point, I had no idea you could do that. Great work!

  2. Ryan Hutchings says:

    nice one dude

  3. Greasylemur says:

    Thank you, for sharing your findings kind sir.

  4. Anonymous says:

    Thanks for sharing, very uselful indeed! 🙂

  5. janey says:

    Hi
    I was just trying to do this but im not sure how you view the tracking data from the camera over the plate?

    Any help would be very much appreciated

    Thanks

    • davemne says:

      After you track and solve the camera, double click the CameraTracker node so its properties are open, and also view it, you should see the tracking points overlaid on top of the plate (see step six).

  6. Anonymous says:

    thanks, I found this really helpful!

  7. Anonymous says:

    Very helpful and clear instructions thank you!

    Is it possible to use the camera tracker to remove jitter from the plate?

    • davemne says:

      If you mean very tiny wobbling, I suspect it would be easier and simpler to use a few Tracker nodes in succession; track one, using the average of a couple of points, set it to stabilize, then use the next one to track the stabilised plate, stabilise that and so on. They all concatenate together.

      If you mean keep the general camera movement but remove the high frequency jitter, probably best to use Furnace-Steadiness if you have NukeX.

  8. eduardo says:

    is possible in after effects with same plugin?

  9. Anonymous says:

    Thats amazing, thank you!

  10. Anonymous says:

    sorry i’m new to nuke. how do you make the Project3D node?

  11. Anonymous says:

    never mind found it!

  12. Chris says:

    Great tutorial – I used this method to stabilized a shot that just wouldn’t hold with the 2d tracker stabilize way. One question though, how do I put the movement back after I’ve finished my paint work?

    • davemne says:

      You can simply copy the whole 3D setup, plug the result of your painted stabilised shot into the new setup’s project3D, then swap the static and moving cameras around in the new setup. The result should look the same as the plate, but with the changes tracked on.

      However, be aware that doing things this way will soften the plate a bit, because it is being filtered twice due to the break in concatenation when you do the painting. Depending on the project, this may not be acceptable. If there’s another way to approach the painting, I’d do that instead.

      • davemne says:

        Just realised the resulting image will probably be a cropped subsection of the original at certain points along the timeline. So just put the result back on top of the original plate. Also, if you have applied any reformatting to the stabilised version, you will have to take that off for this to work.

  13. Pingback: Learning VFX Compositing Skill 3: Tracking/Stabili(s/z)ing « Surviving Post-production

  14. Pingback: Learning VFX Compositing Skill 8: Camera Tracking « Surviving Post-production

  15. wesley says:

    Awesome, thanks!
    Do you have a working script of this that can be downloaded?

    • davemne says:

      I don’t, I’m afraid. As you’d need to see it work on an actual camera tracked plate for it to make sense, I’d have to supply the plate as well and I don’t have anything copyright free offhand.

      If you grab your own footage and follow the script in the picture, you should see it working pretty quickly though. Just remember, it’s the same as projecting a freeze frame patch on a card, just swap the cameras around.

  16. Anonymous says:

    Thank you so much! Great technique !

  17. Anonymous says:

    So helpful! Thanks so much for taking the time to share this!
    Joel

  18. Pingback: stabilize 3D | eleanor nuke training

  19. Redden Visuals says:

    Reblogged this on Redden Visuals.

  20. Redden Visuals says:

    Awesome tip!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: