Tracking SynthEyes

(A blog by developer Russ Andersson)

Why Is My Lens Workflow Padding So Big?

September 15, 2023.

If you're using an advanced lens model and run the Lens Workflow script, you may find that the generated padding is very large, no matter what the settings. What's happening? It turns out that's a result of having source images with “unrelated” horizontal and vertical resolution! It can addressed with the “Maximum rounding error” value at the bottom of the (Advanced) Lens Workflow panel. (Be sure you're using the latest scripts from the Customer-Only area!)

The issue is that the new lens workflow works to maintain the exact aspect ratio of the original image for the highest accuracy results, in order to avoid creating subtle changes that create tiny sliding errors around the edges (or having to specify the exact aspect ratio to 8 digits of precision).

For standard 16:9 footage, you can add any multiple of 16 to width and any multiple of 9 to height and the aspect is identical, as long as the multiples are the same, so the “extra” padding to maintain accuracy is negligible.

But for less regular sizes, the required addition can be much larger. From grade school arithmetic class, you can reduce the Width/Height ratio to a “proper fraction” by removing any factors in common between the two, ie dividing both numbers by the greatest common factor. The proper fraction tells you what must be added to each to keep the aspect the same. For example, one customer asked about 2160H x 882V images; the greatest common factor is 18, so padding can only be added in increments of 120H x 49V pixels. It can be worse for higher-resolution images!

In order to prevent excessive padding, the Advance Lens Workflow has the “Maximum rounding error” value. The idea is that if the padding is unconscionably large, you can start increasing the error a little bit at a time until it's able to find a size with tighter padding that you can live with, at the expense of a slightly higher error at the edge. You get to make the tradeoff between the small amount of error being introduced and how much padding results.

It's an unfortunate weird and painful consequence of using odd image sizes — but one that's still easily addressable.

SynthEyes 2304 Gets a Lens Makeover

April 14, 2023.

Whew! There's a lot of work on lenses in SynthEyes 2304! These upgrades bring not only better fitting, but better integration of lens distortion with other applications. They also prepared the way to add more complex distortion models and solving modes to SynthEyes, as needed. I think the new lens models are going to be huge for everyone.

And what about anamorphic distance?! I got alarmed when I saw that in some optics literature, and the more I looked around, the more I realized that it could explain several anamorphic issues. It is really quite a different effect than we address with traditional 2-D lens distortion equations. It initially seemed that even if the solver calculated it, it wouldn't help anything, until I realized the vertex caches could be used to compensate for it.

However, don't get hung up on anamorphic distance; I see it as a small innovative experiment and a last resort for difficult shots. I don't want you to think that SynthEyes 2304 is somehow forcing you to make strange workflow changes just to use it; it's not. If the anamorphic distance helps you, use it; if not, keep it off.

In any new release, there are some things that did not meet the (time) cut and have to wait for future versions:

Hopefully I'll be able to address these in future versions, in addition to the other new features I have in mind. As always, you can lobby for your favorites.

Shooting useful lens grids is hard, as illustrated by those I've seen, and they are compromised by focus breathing — deprioritizing Lens Master Calibration.

Per-frame lens focus distance metadata can be helpful in several ways, from identifying and assisting shots with lens breathing, to helping match live shots to calibration grids (preferably with focus racks). Please encourage this metadata to be retained! And check out the lens grid shooting guide from time to time: it gets updated as new features and strategies become apparent.

ProRes Decode Broken on macOS Ventura

Updated April 14, 2023.

Reading ProRes files in SynthEyes on macOS Ventura produces an all-black image on M1 Minis, even though it is fine on M1 Pro and M1 Max Macbook Pros***. All the shot resolution and metadata is fine, but the image is nothing but black pixels. If you try some other apps, such as Resolve, they decode ProRes files OK. Must be SynthEyes, right? Not so fast.

SynthEyes doesn't decode most movies itself, it asks macOS to decode the movie for it; that's what operating systems are for. SynthEyes uses the same code to request decodes of all movies, and other movies still read fine on Ventura. How can this be?

The key difference is that SynthEyes 2204 and later request a 16-bit/channel decode of ProRes movies, so that you can have the full ProRes file accuracy if you want it (selected by the image preprocessor). Apps using only 8-bit decodes work, including SynthEyes if I force that internally, so there's something the matter specifically in Apple's 16-bit/channel decode. Everything goes normally, there's no error indicated, there's just no picture! M1/M2 machines have special ProRes hardware; it seems like there is some sort of driver issue there for the Mini. I've reported this to Apple and they agree it looks like an OS bug; there's no additional information at this time.

Workaround in SynthEyes 2304+: Turn on the “Force 8bit ProRes” preference in the ProRes section on affected Mac Minis. All ProRes files will be read as 8-bit and produce images, even if you have requested a 16- or 32-bit image, so this is only a temporary workaround until Apple (hopefully) restores the broken decode functionality.

Starting SynthEyes from Flame Python on Apple Macs

August 5, 2022.

If you're running Flame on a shiny new Apple Silicon Mac, and use SyPy3 to launch SynthEyes from inside Flame's python, you'll find that SynthEyes crashes at launch. Yet SynthEyes launches directly fine, as it has excellent Apple Silicon support.

What's happening? Because there's no native Apple Silicon version of Flame yet, the mac is running Flame using Rosetta. When Rosetta starts a new task, it starts the Intel Mac version, even if an Apple Silicon version is present. I'm sure Apple had a good reason for it, but in this case it's a fatal mistake. SynthEyes on Intel requires CPUs with at least the AVX instruction set, but Rosetta does not support AVX instructions, and crashes when it hits them.

As a workaround, remove the Intel x64 code from the SynthEyes app. Then Rosetta can't botch things up and will be forced to launch the native SynthEyes code it should have launched in the first place. Here's how, using Terminal:

cd /Applications/SynthEyes/SynthEyes.app/Contents/macOS
lipo -remove x86_64 SynthEyes -output SynthEyes_stripped
mv SynthEyes SynthEyes_original
mv SynthEyes_stripped SynthEyes

You may have to run as root (sudo) or otherwise convince macOS to let you do this.

Folks using Flame Python or other Intel-only apps on Apple Macs to start native applications may be able to this same workaround on them. It might be a good idea even if the Intel version runs OK under Rosetta—forcing a computationally-intensive Universal app to run its native version would give higher performance.

Upgraded Floating Licensing for SynthEyes 2204

April 22, 2022.

With SynthEyes 2204 comes a new floating license manager, SyFlo2, with two new license types, cloud/VM support, a web interface, automated license file updates, and support for a secondary SynthEyes batcher instance on client machines. (There's no charge to upgrade from SyFlo to SyFlo2.)

The new Managed Seat license is similar to a regular seat license, but unlike a regular seat license, can be used on cloud workstations. Assigning it to a single user allows a reduced price compared to a floating license. The new Managed Temporary license is a managed seat variation for surge situations, where you need multiple tracking artists to come onboard a cloud setup for a a few months.

Switching to SynthEyes 2204 and SyFlo2 requires more care than typical install and go updates, because not only does SynthEyes 2204 require SyFlo2, but SyFlo2 can only support SynthEyes 2204 and later, not earlier SynthEyes versions, due to all the improvements.

As a result, you must switch all your existing SyFlo and SynthEyes installs to SyFlo2 and SynthEyes 2204 at the same time. SynthEyes licenses with active support (FL-2204- and up) can move into the new SyFlo2 pool, and you can install SynthEyes 2204 on all your clients. Older out-of-support SynthEyes licenses must be upgraded to move into the SyFlo2 license pool and be usable. We've considered different approaches, all of which have pros and cons, and this all-at-once conversion is the simplest and soundest approach.

You're forgiven for thinking that all of these new license types and updates sound a bit complicated. These new capabilities are here in response to how customers have asked to use SynthEyes—to expand into cloud-based workstations for cost and security reasons, not only from a technical standpoint but also from a business standpoint: changes to how companies hire and provision tracking artists. This is our response to accommodate those needs. Let us know how we did.

Updated Scripts for Apple Silicon (M1) Macs

February 26, 2021.

If you are running SynthEyes 2102 on an Apple Silicon (M1) Mac, you should install the updated set of scripts now posted in the Customer Only area. Otherwise, there could be problems, especially relating to slashes vs backslashes and auto-run, the latter of which can result in a crash. If you're running on an Intel Mac, Windows, or Linux, it's not necessary to install the scripts, although there's no harm either.

The backstory on this is at least a little entertaining, going back Apple's last processor transition, from PowerPCs to Intel Macs. To be able to distinguish between the two, there are two different platform codes, MAC and MACTEL, that can be reported to scripts. That selection occurs automatically as a result of a chain of events back to some buried processor setup code... which worked perfectly. It recognized that it wasn't going to be running on an Intel Mac, and said “MAC.”

Of course the scripts haven't had to deal with PowerPCs for well over a decade, so all but the oldest were only looking for “MACTEL,” and that was a problem. As a result, we have a set of updated scripts.

If you have your own scripts, either in Sizzle or Python, that test for Windows vs macOS vs Linux, you too may have to update your scripts. The recommended approach is substr(Scene.platform, 1, 3) == "MAC", which can encompass whatever comes next too.

SynthEyes 2102: Prettier, Faster, Not More Controls

February 11, 2021.

D'oh! There was a combination of unfortunate events that resulted in legacy file formats being unreadable on macOS and Linux in SynthEyes 2102 Build 1049. These formats, Cineon, DPX, PSD, and RGB/SGI, come from the time of SGIs and PowerPCs, when binary bytes were written in “big endian” order instead of the “little endian” order now predominant in Intel and AMD CPUs. When reading those formats, bytes weren't being swapped back to the right order anymore, due to said unfortunate events.

But, that was easily rectified once this problem was pointed out (thanks Chris!), and now we have Build 1050.

As a bonus, 1050 contains an initial release of new Fusion scripts so that you can select some trackers in SynthEyes, run the script, and then paste those trackers into Fusion for use in your 2D effects. Important note: because these put data onto the clipboard, not a file, they are on the Script menu, not the Export menu. There are also two new modes for shifting frame numbers around between SynthEyes and Fusion, including for the regular comp export script. The User Manual has been updated to discuss these.

As always, if you see something, say something.

SynthEyes 2102: Prettier, Faster, Not More Controls

February 5, 2021.

It's been a busy few months! A lot of work throughout SynthEyes went into this latest release: improved cosmetics and icons everywhere, high DPI/Retina support for all three different operating systems, new AVX and NEON optimization throughout for Intel and Apple Silicon processors, some gruesome changes due to ARM's less coherent memory model, even squeezing the new control-middle-mouse scrub mode into all the major viewports.

One thing you won't see: giant new control panels with lots of buttons. I feel a bit bad about that, but I think we can all agree SynthEyes already has a lot of buttons to start with, so the focus in this release was more on making things work better than adding more stuff. And of course faster, since that's how I roll.

Despite that, I think you'll find there are still some new things that you'll find handy (different people, different ones, generally). As always, I encourage you to spend the time to read through the full recent change list, it may save you time on a project or from having to file a Suggestion and me having to say it's already there.

A caveat or two about the Apple Silicon Universal build: first, you need to have purchased or renewed your SynthEyes license after June 2020, when Apple Silicon was announced. If you purchased before then, you'll need to renew your license to use SynthEyes on an Apple Silicon machine. (The renewal will be pro-rated if it is early.) You can check your serial number instead of your purchase date: if the first four-digit number in your serial number is -2106- or greater, you can run on M1. If it is -2105- or less, you need to renew.

Second, a Universal build that can run on two different kinds of machines doesn't automatically mean that it's OK to do so. The standard conditions on simultaneous installation still apply; you'll need to check on them before installing on both an Intel and Apple Silicon machine.

Tip: If you're thinking of getting one of the new M1 machines, I strongly suggest going for the 16GB versions, rather than the 8GB, if you can. Not only is there a lot more 4K material around that can take a lot of RAM to decode, there's the Unified Memory model in M1 that means that “GPU” memory comes from your main RAM space. With the M1's fast SSD, you can use SynthEyes's Disk Cache to mitigate the limited RAM, at some possible cost to its lifetime.

Enjoy the new version. Be sure to let me know if you see any problems (don't leave it to someone else), and file those new feature Suggestions.

Hope the new functional “splash” screen helps get you through these trying times. Give it a little while, and it'll grow on you.

More about SynthEyes on Apple Silicon

November 10, 2020. (SUPERCEDED! SynthEyes 2102 support Apple Silicon!) We've posted a press release about this, but wanted to post some more FAQs for customers. Again, there's no firm date on when a version with Apple Silicon (ARM) support will be available. Depending on the availability of release versions of third-party libraries (ARRI, Blackmagic, RED, Filmbox), it's possible that initial ARM SynthEyes builds may not support the features they implement. Aside from that, the ARM version is functionally the same as the Intel version, although quite substantial amounts of optimized code had to be rewritten to accommodate the ARM instruction set. And of course the SynthEyes release will include new features common across all machine types.

The ARM version will be part of the macOS “Universal” build in the next SynthEyes release. You'll only be able to run the ARM portion of that release if that license was purchased in June 2020 or later. (Notice that Apple Silicon machines were first teased in that month.) As a result, there will be some customers who can run the Intel portion of the macOS release, but not the ARM portion. They will need to do an early (pro-rated) license renewal to be able to run the ARM portion. This corresponds to serial numbers M6/C6/FL-2106-... or later.

Note: SynthEyes 2004 and 2007 will not run under Rosetta emulation on Apple Silicon machines, due to their use of AVX. The native version will be necessary.

Just because you will have an Intel build and an ARM build in a Universal binary doesn't magically give you additional licensing rights. It's more either-or than both. The conditions on Simultaneous Installation still apply.

The new M1 chip Apple Silicon machines look pretty cool. Don't forget to go for the 16GB of RAM if at all possible, especially if you're handling anything above HD resolution.

Here's one last thought.... now that we've got the ARM version for macOS, that same optimized code should be usable on Windows. That would seem to simplify a possible native Windows-on-ARM build of SynthEyes, if computer manufacturers were to introduce suitably powerful Windows on ARM computers to make that worthwhile. This is true for other software vendors, not just SynthEyes. So Apple may have been the one to break the logjam for Microsoft! Crazy.

Read More on SynthEyes 2007

July 14, 2020. SynthEyes 2007 is out; roughly it is a bug-fix release for SynthEyes 2004, with a few subtle narrowly-targeted new features. There are three user-reported crash bug fixes; as always please point out the details of any crashes so we can eliminate them!

The main new features are file readers for ARRIRAW and Blackmagic RAW file formats. As part of that, handling of "half" floating-point numbers has been improved, including the OpenEXR file reader. Half format should now perform similarly to full float format on most modern machines; consequently the Shot Setup panel now allows half format to be selected directly as a processing format.

The other area of work concerned installation. The Windows and Linux installers now install a CPU-only version of Tensorflow, greatly reducing the installer download size. If you'd like to use the GPU version of Tensorflow, you can overlay the GPU version of Tensorflow over the regular installation. The Linux and macOS installers now warn if your processor doesn't support AVX; the Windows version will warn when you start SynthEyes. Also, 2007 includes the latest version of the Microsoft library that was causing a "SynthEyes is already installed" error on some machines; hopefully they've corrected that.

For the full set of changes, see the Change List. Or read the press release.

"Already Installed" on Windows

June 16, 2020. (Updated November 10) A few people are reporting that when installing SynthEyes or SynthEyes Demo, the installer reports that SynthEyes is already installed, when it is clearly not. You may see an error message like this:

Setup Failed
One or more issues caused the setup to fail. Please fix the issues and then retry setup. For more information see the log file.
0x80070666 - Another version of this product is already installed. Installation of this version cannot continue. To configure or remove the existing version of this product, use Add/Remove Programs on the Control Panel.

What's actually happening is that a later version of a required Microsoft library (sub-)installer named VC_redist.x64.exe is already installed. Windows installer declares it an error, even though from a customer standpoint, we want it, we've got it, yay, let's get on with the show! Microsoft says not their problem, every developer should just work out how to deal with it. Mmmmm.

What to do? From the Windows Apps control, uninstall the pseudo-application "Microsoft Visual C++ 2015-2019 Redistributable (x64)." There are two packages that appear to have that same name—if you hover the mouse over the name you will see that one ends with "(x86)" and the other ends with "(x64)" — it's the latter one that is usually the problem (VC_redist.x64.exe in the install log file).

Once you've uninstalled it, reboot, then install SynthEyes. If you still have that problem, uninstall the x86 version and try again. Note that some other apps may not be able to start between the time you uninstall the library and install SynthEyes. If you want, you can download the latest versions of those libraries if you like.

New CPU Requirements for SynthEyes 2004

May 13, 2020. We discovered after release that the CPU requirements of the generic Tensorflow library were less generic than expected, causing us to have to increase the CPU requirements for SynthEyes 2004, for the first time in many years.

(Note: SynthEyes 2102 is now itself extensively optimized to use AVX.)

SynthEyes now requires that your processor implements the AVX instruction set. If SynthEyes crashes at startup, it does not and you won't be able to run SynthEyes 2004 and following versions on that machine. We can provide download links for SynthEyes 1905, the last non-AVX version, if you like.

It may be a surprise to some that an expensive workstation from past years (ie a Mac Pro) is unable to run current software. CPU manufacturers have been adding features every year, not just increasing speed and core count. The AVX instructions were introduced in 2011, almost a decade ago. There are already several sets of instructions after AVX that we are not yet requiring. Requiring AVX support is therefore not unreasonable. We can't have the oldest machines holding back the newer ones forever. By bumping up the CPU requirements to include AVX, we'll be able to provide additional speed increases in future SynthEyes versions. We've been wanting to increase the requirements for some time; Tensorflow forced the issue.

Thoughts on the release of SynthEyes 2004

April 20, 2020. It's been a while, but SynthEyes 2004 has finally made its debut. There was a lot of development effort, not only things you see, but each of the three supported operating systems forced significant different behind-the-scene tasks to be performed. And with COVID-19, we had to create specialized licensing schemes for our educational users and work-at-home users, and even 3D-print a few hundred face shields as part of a community effort for our local hospitals, health care workers, and police and fire personnel. It's been an adventure! Hopefully, you're reading this from relative safety.

As far as new features, the main event was the development of neural-net-based tracking algorithms. You can read about it at Neural-Net-Based Tracking in the manual. The neural net development was a high-risk, high-reward effort to try to capture a high level of tracking expertise and bring it to everyone. Initially intended solely for supervised tracking, models for auto-tracking turned out to be an intermediate stage. At present, applying them on current CPU and GPU hardware is slooowww, probably not suited for everyday high volume production work. (You can continue to use the speedy previous scheme, of course.) You might find neural auto-tracking useful dealing with X marks on green screens, in handling checkerboard lens grids, and maybe for shots of buildings where you will do mesh reconstruction. Performance shouldn't be as much an issue for supervised tracking, if you choose to use it. What's here now is an entry point to being able to do interesting things in the future.

A few words of warning: SynthEyes 2004 uses open-source software initiated by Google, named Tensorflow, to run the neural nets on the CPU or suitable GPU hardware (on Windows and Linux, sorry macOS). As a result, the SynthEyes installers are now much larger, ranging from over 100 MB to 850 MB, with installed sizes from 400MB to 1.3 GB.

As a semi-related feature of the neural tracking work, SynthEyes 2004 also introduces a subtle new lighting-invariant option for supervised tracking. This should do a better job on shots where the overall lighting level is changing from frame to frame, which can otherwise throw off the supervised tracker, and it should eliminate the need for high-pass filtering on very flickery shots. Of course, it does burn off some of your CPU cycles to obtain this better result, though that should not be noticeable on solid current workstations. We have so much confidence in it that it's on by default, and it doesn't even appear in the manual (hmmm)! You can turn it on and off from the Track menu and preferences. (Tip: don't make your trackers so small that there's no texture within them.... a flat color can match any and every other flat area!)

In addition to the headlining features, there are many enhancements and fixes suggested by users. We didn't want any further neural net development to hold them up further. You can see the Recent Change List for details.

What's next for SynthEyes? After the long big-feature push here, look for a small-feature push next, focusing more on small usability features and a shorter development cycle. As always, the Sug button's feature suggestion page is the source of many ideas. Please understand that SynthEyes users work on many different tasks and have many different workflows. The ideas most likely to be implemented are specific, simple, well-defined, positively affect many users, and make incremental changes to the current program and procedures.

I'm also hoping to make some combination of Arri, Blackmagic, and ProRes formats usable within SynthEyes. As always, a lot of the “behind the scenes” things necessary to make a viable cross-platform product can throw in monkey wrenches at any time as well, as they did in the current release.

In the meantime, we'll all plow ahead, and please stay safe.

Regards,
Russ Andersson

Press Releases

Past ssontech.com press releases.

Read

Contact Us

How to contact us.

Read

Privacy Policy

Information on our data policies.

Read

SynthEyes easily is the best camera match mover and object tracker out there.

Matthew Merkovich

More Quotes