Object Tracking with AprilTags in SynthEyes

Shows how you can use the AprilTags tracking capability of SynthEyes for object tracking, illustrated by tracking two tags taped to a hand‐held phone. Source and result footage is available on our website in the Downloads | Example Files area.

Script for Search Engines:

Hi, this is Russ Andersson. In this tutorial we're going to show how to use AprilTags for object tracking. AprilTags are an open-source fiducial tracking scheme from the University of Michigan. You can see two of them here. There are several different families of AprilTags; within each family are 30 to 65000 tags that SynthEyes can individually identify. Let's get ready to track. The tag family must be set here. You may not know or remember which tag family it is; SynthEyes can check for you. While you're at a frame with tags visible, click "What is it?" It can take a little while for the more complex families. You can cancel early if it is one of the simpler families. The tag family dropdown is set either way. For this tutorial, we want moving objects, we'll set that mode. AprilTags analyzes only the playback range, which we've reduced to skip empty frames at the beginning and end. Before starting, for the correct perspective, you should set a known or estimated camera field of view. To set the coordinate system scale for the paths, set the Tag Size on the control panel. See the manual for what to measure. Just like that, here are the results. You'll notice that they can only be slightly obscured. More complex tag families have more error tolerance, but require the pattern to occupy more of the image to have a similar number of pixels per block --- here, around 45. Let's have some fun and look at the results in more detail. I call this shot the two-man bobsled. You can see that it loses track when the finger covers the pattern. If you look at the time bar, you can see there are no keys--- the tracker wasn't detected. You can see that in the graph editor also. You have the option of hand editing the curves here... Or with some tag families you might try a larger Maximum Error Bits value during setup. Here's another way to look at the accuracy. You can see it's OK but not perfect. If you look carefully, you can see that the larger errors are due to motion blur. For something closer to the phone's surface, you'd just use one pattern, and it would likely work out fine. If you're wondering, there is no direct mechanism to average the paths of the two tags together, that is not a sound plan, especially for an object that is flat. There is something better to do, by producing corner trackers, but that's a subject for a different tutorial. To summarize, using AprilTags for object tracking is fantastically simple. Just make sure that: the patterns are pretty big in the image, they are only slightly obscured at any point, and that there isn't too much motion blur. Be sure to check out the manual for more information. Thanks for watching.