< Previous | Contents | Manuals Home | Boris FX | Next >
Neural-net tracking was initially conceived of as “can we train a neural net to do the supervised training that a human being does?” So far this is a work in progress, but one that appears as a new supervised tracker type. As with auto-tracking, it offers the possibility to track various features specifically.
Important : The pixel aspect ratio should be 1.0 for neural-net processing. Use the image preprocessor’s Output tab to set up resampling to make that the case.
While auto-tracking uses a single network for the entire shot, you can use a different neural net for each individual tracker.
You set the neural network for a tracker (or group of trackers) when you initially set the tracker to neural net, by selecting the NN icon from the tracker type dropdown. You can later examine or change the selected net by right-clicking the selector type.
Tip : Don’t track several trackers with different neural nets simultaneously, or performance will be slow as it must switch back and forth. Make sure that the active trackers all use the same net; lock trackers that use other nets.
The neural network size can be chosen independent of the size of the tracker. The tracker size corresponds to a number of pixels in the original image; the neural net’s (kernel) size can be larger, resulting in oversampling of the original image, or smaller, resulting in undersampling of the original image. Note that undersampling results in less accurate tracking results, while oversampling can result in more accurate (sub-pixel) results. While the large neural network may be too slow and look for things that are too big when used for automatic tracking, for supervised tracking it might be just the right thing!
You may need to adjust the size of the tracker to get the neural net to focus on what you want it to: scale is important! For example, consider a light pole: is the network to look at the center of the end of the pole, or the left or right corners of its end. A wide area will see the end of the pole, a smaller tracker can focus on the corners.
At present, the supervised tracker picks the most promising-looking feature in its field of view—regardless whether the feature has the same general type or look as other keys locations for the tracker. We expect to address this in a later release (which may appear as some additional supervised-only neural net types).
Note : Neural net trackers are similar to symmetry and spot trackers in that they pick a single best location, displayed as an X in the tracker mini-view. You can move the tracker on a given frame, but that won’t affect what happens in subsequent frames, though it might grossly affect where the best feature is looked for. They don’t auto-key or do bi-directional blending, that’s just not a thing for neural trackers.
©2024 Boris FX, Inc. — UNOFFICIAL — Converted from original PDF.