Aligning Shots with AprilTags

Shows how to align a shot to a reference shot, using 3 or more AprilTags in the scene for easy alignment. This example uses an outdoor scene from a Wiral cable rig, though AprilTags may be even more useful indoors and in green‐screen shots.

Script for Search Engines:

Hi, this is Russ Andersson with a tutorial on how you can use AprilTags to easily and automatically align two shots of the same scene. AprilTags are an open source fiducial scheme from the University of Michigan; there are multiple families of tags and each of the many tags in the family can be automatically found and individually identified in an image, producing a consistently-named tracker. We're going to be working on some outdoor shots, although AprilTags are equally as useful indoors, including on green screen shots. Here's what the tags for this tutorial look like, a 24 by 20 inch blank yard sign, covered with an inexpensive blueprint-quality graphic from a local office shop. The tag is 16 inches on a side. Let's take a look at our shots. A clean version of it is the reference shot. And a live version is to be matched to the reference shot. Now both of these were shot with a GoPro on a Wiral cable system. We'll start out by going to Camera 1 and running the auto-tracker. Now we can run the AprilTags Detector. Now these are standard 36H11 tag type. We do want to check this Prefix name with camera option, and we'll see why later. You can see it doesn't find all the trackers, they're a little small for this distance. I also probably should have used a light gray, instead of white, to reduce blooming from the white into the black. If you look at these tracks up here, you'll see that despite all the trees, AprilTags are always re-found with the same identifying tag number, and they don't result in multiple different trackers. No coalescing is required. So that's definitely one of the advantages of the AprilTags. Now we're ready to solve, and we can set up a coordinate system using a couple of these trackers, maybe. So that could be whatever coordinate system we want. Usually, when you want to match multiple live shots to a single reference, you want to solve the reference once, and not have it be affected by the live shots. Now, there are a variety of ways to do that in SynthEyes, but the easiest is just to disable Camera 1 at this point, and that way it won't changed further. With that, we're ready to move on to Camera 2. Similarly, we're going to run the auto-tracker for it, and now we can run the AprilTags Detector. We're going to do one thing here, which is to select the Make link option. And I'll point out that there's no point using corner trackers on this shot from the AprilTags, because they are so small in the image. Essentially only the center has any usefulness. Now you see that we've located some trackers here, some of the tags, and created links. Lets go take a look at those. Now you see why we checked the prefix name with camera option, otherwise you couldn't tell the trackers from one camera apart from those of the other. There have to be at least 3 links like this to allow the two scenes to be aligned. So let's go solve. You can see that the camera paths don't match up exactly and in the Coordinates View the values don't match up exactly either. Now first, we haven't done the necessary tracker cleanup that's clearly needed with all the trees flying by, which create little tails and wipes and whatnot on trackers. We can ask SynthEyes, though, to make those values match up exactly. We'll switch to Refine mode now, for, well at least for that camera. And Refine now. Now you see that the paths are closer together. The coordinates do match up exactly. In this shot we can make the solve even closer, by making the field of view match up. Here I'm just going to copy it from one to the other. Set the other camera to be a known field of view at that same value ---just a little copy and paste. Now we'll go and Refine again and you can see that they're really quite close together, but of course that does require that you know that the camera settings were identical during the two shoots, and that includes things like the focus distance, and iris perhaps, as well as the focal length. Now in this case, it's just a GoPro, so it doesn't really have any adjustments like that. Typically, you'll store a reference scene file with only Camera 1, and copy that scene file and Add Shot for each live shot. There are some additional options available. When doing AprilTags tracking of the second shot, you can select the Copy Coordinates option, which assigns the tracker's lock coordinates to be the coordinates of the corresponding tag in the reference shot, as a one-time-only step at the completion of AprilTags tracking. That will give you some more flexibility, since you can then decide whether you want to lock each axis individually, rather than the all-3-or-none nature of a cross-object link. For example, you may want to avoid an over-constrained coordinate system setup on the second shot. You can take this one last step further, and use the "Plain Trackers" exporter to save away the reference tracker locations. Then you can use the "Tracker Locations" importer to bring them in for each new shot. You can see that there are plenty of workflow options available. AprilTags present a nice option for simplifying handling of multiple shots that must all be aligned to a common coordinate system. Be sure to test that the tags are big enough and well positioned to be visible and able to be analyzed correctly in your live shots. You'll need to mask them out of the shot later, but at least you'll know where they are. Thank you for watching.