RotoMasking in SynthEyes 2210

Walks through the theory and operation of roto‐masking features in SynthEyes 2210: creating splines, including spline setup via trackers; assigning blips via splines; visualization and outputing mattes; and working with imported mattes.

Script for Search Engines:

Hey, this is Russ Andersson. In this tutorial, I'm going to give a pretty complete runthrough of what RotoMasking is for in SynthEyes, and along the way describe some new features in SynthEyes 2210 that make life easier. First, Rotomasking in SynthEyes is NOT intended for generating exact edge mattes for pulling things in or out of shots. Always keep in mind that SynthEyes isn't a compositor or animation app, it's designed for tracking, where things often have to be just a bit different. In the file you're looking at, I've set up some splines, and manually done the first step of autotracking, Blips All Frames. As SynthEyes scans through your shot one frame at a time, SynthEyes generates many new tentative image features, called blips. Now, for the purpose of tracking, rotomasking must answer exactly one question: to what camera or object should each blip be assigned? Each blip can only be assigned to one thing, to the camera or to one of however many moving objects there are. Or, it might be assigned to nothing at all, for garbage areas that are to be ignored. For each of these blips, SynthEyes is looking through the list of splines from top to bottom, and the first spline containing each blip wins --- that spline's object assignment is used for that blip. If the blip falls in a garbage spline first, the blip is garbage. If it falls in a camera spline first, it's assigned to the camera. Same for splines assigned to any moving object. And if a blip doesn't fall in any spline, it's also garbage, unless there's an incoming alpha matte, and we'll talk about that later on. To help you understand your spline setups, there's a new RotoMask button on the camera view's View toolbar. It overlays a checkerboard showing exactly which blips would be assigned to the current Active Tracker Host. Here we're seeing what would be assigned to the camera, and what would not. You can drag the RotoMask button to adjust the checkerboard's blending. We can also see what gets assigned to the moving object, by changing the active tracker host. Right-click the RotoMask button to turn it off. So let's jump to some common misconceptions. First, SynthEyes doesn't generate a matte image of any kind during auto-tracking. There's no need. The overlay you just saw is only for you. SynthEyes only needs to determine each blip's assignment during tracking, and there are only a couple thousand blips per frame, rather than the millions of pixels that are in a full image. That difference between thousands and millions is one of the ways that SynthEyes keeps its auto-tracking fast! Second, again, there's no matte image, we're deciding not between black and white pixels in a matte, we're not deciding between only two different choices, but potentially between three, four, or more. Just because your shots typically don't have moving objects doesn't mean that SynthEyes should dumb down; it's a pro tool meant to accommodate the tougher cases too, with a truly general solution to the assignment problem for any number of objects. Third, rotomasking may not be necessary at all! It can often be quicker to just run the auto-tracker, then delete a bunch of spurious trackers, rather than doing some complicated spline setup. Delete mode on the tracker panel makes that deleting pretty fast. And likewise, your rotomask don't have to be perfect either; you can trade off between the accuracy of your roto work, and how much tracker cleanup you might have to do. OK, let's look at making some splines. We'll reopen this shot from scratch. If you click the square or circle, you can create a simple spline, and then adjust it as needed. Notice that each button push now creates only one spline. Hold the shift key when you click the button to create several splines in a row. You can create arbitrary-shaped splines too. The first point is the center pivot point. Smooth spline control points are circles; corner control points are squares. Double-clicking a control point changes its type. You can do that while you're creating the spline also. Right-click to stop. Once you've created some splines, you can click them to select them, drag them around, hold control and rotate them, or alt- or command-scale them. You can even hold control and ALT or command to scale and rotate them at the same time. With a spline selected, you can drag the individual control points around. You can shift- or control-click them to select or deselect points, or even lasso them. Then you can drag them all around together. As usual, you can select rectangular or polygonal lasso regions from the Edit menu. Enough playing. All we really need here is a rectangle. You're going to adjust the spline throughout the shot; the control point locations are animated. Usually you position everything at the beginning of the shot and the end, then the middle, then the middle of each half, the middle of each quarter, etc. As you do that, scrub through the shot and adjust the worst positions. You always want to reposition the entire spline first, either by dragging and rotating the entire spline, or by dragging the middle pivot point, which is where the overall spline position information gets stored. There's a hidden scale and rotation point too that holds that animation; you can see and adjust it in the graph editor. By the way, you can click the center point to deselect all control points; shift- or control-click the center point to deselect the spline itself. As you're setting up your splines, you probably want to be adjusting their object assignments on the roto panel, making them garbage or camera or whatever moving object you might have. Keep in mind that the big full-image spline is just a handy initial convenience; if it doesn't help you, just delete it, or if you're not sure, Ignore it. The scrawny tree occludes the truck towards the end of the shot. I animate not only the position of a small garbage mesh, but its Enable track as well. I keep it out of the way when it is disabled. In SynthEyes 2210 you now have the option to Invert a spline. People wanted it, even though it isn't really necessary, so there it is. When you invert a spline, that means that blips that are OUTSIDE the spline are considered to be inside, and blips INSIDE are considered to be outside. That's just switching white and black in a matte, though again, there isn't really a matte. When you Invert a spline, the plus sign changes to a minus sign in the spline list. If you Ignore it, it turns to a little "o". Keep in mind that Inverting layers shouldn't really be necessary, I'm claiming that in every case you can take a stack of splines that uses Invert and change it into a slightly different, likely simpler, stack that doesn't use Invert. But that's for you to discover. Also, at present, the Invert setting is ignored by planar and geometric hierarchy tracking and by ViewShift, if you're using rotosplines with those subsystems. The RotoMask button has an additional feature, turning on both RotoMask and Alpha. If you want to look for little problems, it can be convenient to see a real binary matte. This binary matte can be saved to disk using Save Sequence with the alpha channel turned on. You can turn off the RGB output, and with alpha on, you'll get actual black and white images. Notice that the matte doesn't have soft edges, because that isn't necessary for tracking --- again, SynthEyes isn't a compositing app. You can read the manual to find out how to do that if you really really must. It can be helpful to look at the splines in the Hierarchy View; it gives you a bit more information and control at once. You can change the order of the splines using the arrow buttons on the Roto panel, or, you can just drag them around in the Hierarchy View. People have suggested putting the splines under their corresponding object in the Hierarchy View, but guess what? If you do that, then you lose the ability to change the ordering there, which seems more useful. Now, some slightly more advanced methods. If you set up a tracker, or even two, where you want to put a rotomask, you can import the tracker data into the control point's position, to perhaps set up the roto more quickly. Suppose we want to block out one of these other cars too. We'll create a tracker on the car, for illustration just this part of the shot. We create a spline. And now we import the tracker into the spline's center point. We'll need to adjust the control points now to compensate for that. If we have a second tracker, we can hold down shift or control when we click the center point, and the second tracker will be used to animate the spline's rotation and scale. Again, we probably have to adjust the control points some. Importing tracker positions can be most helpful on shaky shots, where you'd otherwise have to do a lot of frame-by-frame adjusting of the roto work. Let's talk a little about these Alpha values at the bottom of the Roto panel. What are they for? They are used when you want to use an alpha matte that you have generated in some other application to control which object the blips get assigned to. Typically you'll bring in that matte as a separate alpha channel from the shot setup panel. If you're not doing that, there's no need to worry about these alpha controls. If you do add a separate alpha channel, you'll be asked whether you want to configure for camera auto-tracking with a binary matte. We'll say yes here. The Object Alpha Value will be set to 255, and the Shot Alpha Levels set to two. The Shot Alpha Levels determines the tolerance for matching intermediate alpha values that might show up in your matte. With only two levels, any blip that falls on a pixel with an alpha over 128 will be assigned to the camera, the others won't. There's one last fine point to the alpha data. The alpha data is only examined if a blip doesn't fall into any rotospline. That's why the catchall rectangle is automatically set to be Ignored. With this scheme, you can add rotosplines that override and correct the alpha data, if that turns out to be necessary. It's just another pro feature. If you have both a camera and moving object, then you need 3 different levels in your alpha matte, including a garbage level. With Shot Alpha Levels of three, your matte should contain the values 0, 128, and 255; the cut-offs will be at 64 and 192. You can increase Alpha Levels to require closer matches. If you increase Alpha Levels all the way up to 256, then the values must match exactly. You might do that if you're using something like a renderer's object ID number output, where the values go 0, 1, 2, and so on. You're probably happy to know that most people are unlikely to have to use this last capability. Hopefully you now have a lot better understanding of the Roto masking capabilities in SynthEyes. Take care and thanks for watching.