SynthEyes™ 2024.1 User Manual

Welcome to SynthEyes™, our camera-tracking system and match-moving application. With SynthEyes, you can process your film or video shot to determine the motion and field of view of the camera taking the shot, or track an object moving within it. You can combine feature locations to produce an object mesh, then extract a texture map for that object. After reviewing the match-moved shot, inserting some sample 3-D objects and viewing the RAM playback, you can output the camera and/or object motions to any of a large number of popular 3-D animation programs. Once in your 3-D animation program, you can add 3-D effects to the live-action shot, ranging from invisible alterations, such as virtual set extensions, to virtual product placements, to over-the-top creature insertions. With the right setup, you can capture the motion of an actor’s body or face.

SynthEyes can also help you stabilize your shots, taking full advantage of its two and three-dimensional tracking capabilities to help you generate rock-solid moving- camera shots. The comprehensive stabilization feature set gives you full directorial control over stabilization.

If you work with film images, especially for TV, the stabilization system can also help you avoid some common mistakes in film workflows that compromise 3-D tracking, rendering, and effects work.

And if you are working on a 3D stereo movie (two cameras), SynthEyes can not only help you add 3-D effects, but it can help you match the two shots together to reduce eyestrain.

Bonjour! Hej! こんにちは！Привет! हैलो ! Hallo! 您好！Ciao! Welcome to all our non-native-English SynthEyes users. If you can read this far, your knowledge of English far exceeds our knowledge of any other language, so congratulations! Please see the Global, Not Just English section of the manual for information on translations and Unicode character set support.

Once you've checked out the basics in SynthEyes, explore our revolutionary Synthia™ English-language instructible assistant , via the IA button at top right and the Help/Synthia PDF manual. Command SynthEyes to perform tasks that would otherwise require programming, or just use its voice recognition and response to save time. Perfect for those strange one-off fixits.

IMPORTANT : to minimize the chances of data loss , please read the sections on auto-save and on file name versioning , then configure the preferences to correspond to your desired working style!

To help provide the best user experience, SynthEyes has a Customer Care center with automatic updates, messages from the factory, feature suggestions, forum, and more. Be sure to take advantage of these capabilities, available from SynthEyes’s help menu.

Be sure to check out the many video tutorials on the web site. We know many of you are visual learners, and the subjects covered in this manual are inherently 3-D, so a quick look at the tutorials, and playing along in the program as you read, can make the text much more accessible.

DON’T PANIC . SynthEyes is designed to integrate into the most sophisticated visual effects workflows on the planet. So it is only natural that you will probably see some things in this manual you do not understand. Even if you don’t understand 3-D color LUTs or the ‘half’ floating point image format, you’ll be fine! ( hello, google! Hello, Wikipedia!) We have worked hard to make sure that when you do not need that functionality, everything still works, nice and simple. As you learn more and do more, you’ll likely discover that when you want to do something, some control that hadn’t make sense to you before will suddenly be just what you need. So jump on in, the water’s fine!

How to Read this Manual Additional Manuals Contents Quick Start: Automatic Tracking Quick Start: Supervised Tracking Quick Start: 3-D Planar Tracking Quick Start: Geometric Hierarchy Tracking Quick Start: Stabilization Quick Start: The Works Shooting Requirements for 3-D Effects Types of Shots: Gallery of Horrors Basic Operation Coordinate Systems Rotation Angles Menus Key Mapping Main Tool Bar Main Selector Active Camera/Object Customer Care Buttons Undo/Redo Right-click Undo/Redo History Multi-way Redo Branching Control Panels Viewport Layouts Room-Selection Tab Control Floating Panels Buttons, Selectors, and Editable Selectors Edit Fields and Keyboard Focus Spinners Viewports Viewport Rendering Engines Play Bar Time Bar Error Curve Mini-View Floating Views Tooltips Status Line Color Scheme Click-on/Click-off Mode Scripts Script Tidy Up Script Bars (for Menus too!) Notes Preserving Placement and Settings Settings Preferences High-DPI Monitors Windows High-DPI macOS Retina Mode Linux High-DPI Auto-Save Filename Versioning and Limiting Re-Finding Files Mesh De-Duplication De-Duplication Modes Single Files vs Versioned Files Forcing a Re-Read Sending SNI files to Others. Global, Not Just English Translating Tooltips, Menus, and More International Character Sets Language-Specific OpenGL Fonts Initial Conversion to Unicode Keeping Names Simple Exporting to Other Applications Community Translation Project Opening the Shot Image Sequences Reading AVI, Quicktime, etc "Movies" Basic Shot Settings Shot Load Performance After Loading Frame Numbering (Advanced!) Changing the Imagery Image Preprocessor Basics Image Adjustments Rez Tab. Color Tab. Using Kurves Color Correction 3-D Look-Up Tables (LUTs) Floating-Point Images Advanced Details: Reading "Movie" Files Background Movie-Reading on Mac OS X Movie-Reading on Windows Threading Control for Reading Movies Disk Cache Shot Naming Placing the Disk Cache Folder Disk Cache Preferences Native vs 8-Bit Caching Filling the Disk Cache The Disk and RAM Cache Pipeline RAM Cache Size with Disk Caching Using Disk Cache Files Directly Reading ARRIRAW Files ARRIRAW CPU and GPU Options Reading Blackmagic RAW Files Blackmagic RAW CPU and GPU Options Reading RED R3D Files RED Rocket Decoding RED GPU Video Card Decoding RED CPU Software Decoding Reading DNG/Cinema DNG Sequences Color Processing File Sequences Performance Shot Metadata Retrieving Focal Length from Metadata Writing Metadata Separate Alpha Channels Explicitly Adding Separate Alpha Separate Alpha Numbering for Movies Implicit Separate Alpha Separate Alpha Channel File Types Rolling Shutter Rolling Shutter Compensator Minimizing Grain Handling Strobes and Explosion Shots Memory Reduction Animated Shot Setup Temporarily Disabling Preprocessing Disabling Prefetch Correcting Lens Distortion Image Centering Image Preprocessor PrepSet Manager Rendering Sequences for Later Compositing Automatic Tracking Overall process Spot vs. Corner Detection Motion Profiles Controlling the Trackable Region Green-Screen Shots Promoting Blips to Trackers Keeping Track of the Trackers Advanced Feature Dialog Effects Green Screen Shots Too-Busy High Resolution Shots Skip-Frame Track Strengths and Limitations Supervised Tracking Types of Trackers Channel Selection Prediction Modes and Hand-Held Shots Adjusting While Tracking Staying on Track and Smooth Keying Animating Tracker Size and Search Size Tracker Size/Aspect Details Track by X Frames Script Supervised Tracking with SimulTrack Reference Crosshairs Occluded or Off-Image Trackers Cliffhanger Trackers Changing Tracking Direction Finishing a Track Hand-Animating a Tracker Offset Tracking Creating a Temporary Offset Using an Offset to Compensate for Drift Tracking an Additional Feature with Offsets Combining Trackers Filtering and Filling Gaps in a Track Zero-Weighted Frames Pan To Follow Skip-Frame Track Neural-Net-Based Tracking Why It’s Not Just Cats and Dogs Currently Distributed Neural Nets X-Finder Checkerboard Automatic Tracking Controls Supervised Tracking Controls GPUs and Performance Installing the Tensorflow-for-GPUs Overlay Preferences Auto Tracking Preferences Neural Supervised Preferences GPU Preferences Suggested Usage AprilTags Tracking Choosing an AprilTag Family Shooting with AprilTags Printing and Mounting AprilTags Tracking AprilTags Producing 2-D Trackers Producing Planar Trackers Producing Moving Objects Working with Multiple Scenes Combining Automated and Supervised Tracking Guide Trackers Supervised Trackers, After Automated Tracking Controlling Fine-Tuning Converting Automatic Trackers to Supervised Trackers Fine-Tuning the Trackers Usage Suggestions Pre-Solve Tracker Checking Finding Erratic Trackers Checking the Tracker Trails Checking Tracker Lifetimes Introduction to Tracker Graphs Interpreting Figure of Merit Velocity Spikes DeGlitch Mode Finding Spikes Before Solving Nodal Tripod-Mode Shots Solving Tripod-Mode Shots Lens Field of View from Tripod Shots Far Trackers Alignment of Tripod-Mode Shots Low-Information Tripod Shots. Don't Clone Trackers! Proper Single-Tracker Workflow Mixed Nodal and Normal Shots Introducing “Holds” Multiple Hold Sections Preparing Trackers for Holds Hold Tracker Preparation Tool Usage Hints Lenses and Distortion Focal Length vs Field Of View Zoom, Fixed, or Prime Lens? Introduction to Lens Distortion Determining Distortion With Check Lines Calculating Distortion While Solving Distortion, Focal Length, and the Field of View Cubic and Quartic Distortion Correction Anamorphic Lens Breathing Lens Distortion Profiles What About 2-D Grid Corrections? Accurate Presets from Lens Grids Match-moving with Lens Distortion Basic Lens Distortion Workflows Delivering Undistorted Imagery Delivering Distorted Imagery Undoing the Lens Workflow Script Zero-Pass Workflow Lens Distortion Models vs Lens Correction Models Lens Distortion Interchange via Image Distortion Maps The Reference Image Bit Depth, Color Space, and Resolution Unmapped Pixels and Blue vs Alpha Inverse Maps Writing Image Distortion Maps Writing Image Distortion Sequences Applying Image Distortion Maps Exporters and Future Work Lens Distortion Interchange via Projection Screens Lens De-Centering Working with Zooms and Distortion Summary of Lens Issues Running the 3-D Solver Solving Modes Automatic-Mode Directional Hint. Lens Settings Lens Models Parameter Animation Solving Modes Strategies for Animated Parameters About Anamorphic Distortion and Rolling Shutter World Size Go!During Solving After Solving RMS Errors Solving Long Shots Every Solve is Different Macroscopic Differences Computer arithmetic has limited precision Tasks are distributed across multiple processors There are many local optima There are many arbitrary decisions Adding randomness can improve performance When it helps When it hurts Using Zero-Weighted Trackers as Canaries Effects of Rolling Shutter Checking the Lens Solving Issues 3-D Review Quad View Error Curve Mini-View Camera View Overlay Checking Tracker Coordinates Checking Tracker Error Curves Check the Radar Display Check for a Smooth Camera Path Path Filtering Cleaning Up Trackers Quickly Bad Frames Far-ish Trackers High-Error Trackers Unlocking the User Interface Setting Up a Coordinate System SynthEyes and the Coordinate Measuring Machine Back to Reality Three-Point Method Place Tool: Automatic coordinate system setup Manual Alignment Retaining Manual Adjustments Use on Moving Objects Impact on Meshes Size Constraints Configuring Constraints Directly Tracker/Tracker Constraints in Various Views Details of Lock Modes Configuring Constraints for Tripod-Mode Shots Constrained Points View Upside-down Cameras: Selecting the Desired Solution Subtleties and Pitfalls Alignment Versus Constraints Pegged Constraints Constrain Mode Limitations and Workflow Tripod and Lock-off Shot Alignment Single Frame Alignment What Lines Do I Need? Operating the Panel Advanced Uses and Limitations Using 3-D Survey Data Constraining Camera Position and Motion Concept and Terminology Basic Operation Using a Camera Height Measurement A Straight Dolly Track Setup Using a Supplied Camera Path Camera-based Coordinate System Setup Soft Locks Overall Distance Locks Orientation Locks Locks in Object Tracking Aligning to an Existing Mesh Field of View/Focal Length Constraints Spinal Path Editing Spinal Align Mode Spinal Solve Mode Avoid Constraint Overkill Advanced Solving Using Phases (Pro, not Intro) Overview of Phases Quick Example: Set Horizon Phase Coordinate System Setup with Phases Phases for Moving-Camera and -Object Shots Introduction to Multi-step Solving Solved vs Unsolved Phases Post-Solve Tracking Zero-Weighted Trackers Solve As You Track Juicy Details Stereo ZWTs Adding Many More Trackers Basic Tracker Requirements To a Range of Frames To a Specific Area Zero-Weighted vs Regular Trackers Other New Tracker Properties Coalescing Nearby Trackers What Does Nearby Mean? What Trackers should be Coalesced? An Overall Strategy Coalesce closest in 3-D Perspective Window Locking to a Camera Image Overlay and Projection Screens Freeze Frame Stereo Display Navigation Navigating with the Mouse Toolbar Navigating Maya-Style Navigating with Control and ALT Navigating When Locked More Navigation Details Zooming and Panning in 2-D Other Mouse Toolbar Buttons More About Toolbars Creating Mesh Objects Importing Mesh Objects Vertex Caches Mesh Vertex Numbering Lidar Meshes Opacity Moving and Rotating Objects Placing Seed Points and Objects Grid Operations Shadows Edit Mesh Preview Movie Render Cameras Technical Controls Exporting to Your Animation Package General Procedures Multiple Exports Exporting with Filename Suffixes Exporting to Subfolders Exporting the Image Sequence Too Saving Export Multiple Configurations Setting the Units of an Export Image Sequences Rotation Angles, Gimbal Lock and Motion Blur Shots With Lens Distortion Generic 2-D Tracker Exporters Generic 3-D Exporters After Effects 3-D Javascript Procedure 3D Mesh Support in After Effects Everything is Measured in Pixels! Exporter Controls Cinema 4D/CINEWARE Integration Updating After Effects Projects Matching SynthEyes Distortion in After Effects After Effects 2-D Bulk Trackers Exporter After Effects 3-D "Maya ASCII" Procedure (Obsolete) After Effects 2-D Planar Exporter After Effects 2-D Single-Tracker Path Procedure Alembic 1.5+ Alembic Controls Bentley MicroStation Exporting from SynthEyes SynthEyes/MicroStation Export Parameters: Importing into MicroStation Transferring 3-D Coordinates Transferring Meshes To Record the Animation Blender Directions Blender Directions Blender Settings Blender Directions─Before 2.57 Cinema 4D Procedure Normal way: Cinema 4D Python Export Very Old way: Cinema 4D Via Lightwave COLLADA Exports DotXSI Procedure ElectricImage 3D Equalizer Overview and Setup. How to Solve in SynthEyes for 3DE4 How to Export from SynthEyes What’s Exported How to Import in 3DE Filmbox FBX Application-Specific Filmbox Tips Importing Filmbox Files Fusion Fusion Composition Export Controls Working with Lens Distortion Sending 2D Tracking Data to Fusion Known Fusion Issues HitFilm Project HitFilm Export Controls Important HitFilm Notes Houdini Instructions Lightwave Maya ASCII Updated Modo Limitations Directions Nuke Distortion Grid Planar Tracker Corner Pin Export Metashape/PhotoScan Poser Resolve's Fusion Shake trueSpace Universal Scene Description (USD) Vue 5 Infinite Vue 6 Infinite Realistic Compositing for 3-D Basic Overlays Compositing 3-D Renders Sliding Rotoscoping Matching Lighting Matching Motion Blur Matching Defocus Matching Noise Eroding Alpha Building Meshes Creating 2.75-D 'Cards' Creating the Card Moving a Card Texturing Tricks Creating Meshes from Tracker Positions The "Edit Mesh" Example: Ground Reconstruction Adding Vertices Controlling Auto-Triangulation Using the Assemble Mesh Mouse Mode Mesh Editing What Happens If I Refine? Working with Primitives to Build Set Models Three-Point Alignment Align Via Links Dialog Summary of Ways to Create Links Using Pinning Other Helpful Operations Applications, ie What Next? Changing Camera Path Depth Maps Front Projection Texture Extraction Mesh Preparation and Texture Coordinates Texture Display Texture Creation and Format Controls Animating the Enable Blocking Blocking Meshes Garbage Mattes Texture Extraction Mode Average Pixel Average w/alpha Best Pixel Weighting Control Tilt Control Run Control Texture Expansion When Is the Texture Saved? Alpha Generation Alpha Painting Far Meshes Mesh and Texture Export Using the Textures Stabilization Why SynthEyes Has a Stabilizer How NOT to Stabilize In-Camera Stabilization 3-D Stabilization Stabilization Concepts Overall Process Stabilizing on a Subject Stabilizing a Traveling Shot Minimizing Zoom Animating POI Too Much of a Good Thing? Auto-Scale Capabilities Tweaking the Point of Interest Resampling and Film to HDTV Pan/Scan Workflow Changing Shots, and Creating Motion in Stills Stabilization and Interlacing Avoid Slowdowns Due to Missing Keyframes After Stabilizing Outputting the Shot Creating a Stabilizing Rig Flexible Workflows Rotoscoping and Alpha Channel Mattes Overall, and Rotoscope Panel Spline Workflow Animated Splines Roto-Panel Spline Operations in the Camera View From Tracker to Control Point Seeing and Checking Roto Spline Stackups Writing Alpha Mattes from Roto Splines Using Alpha Mattes to Direct Tracking Object Tracking Automatic Tracking Supervised Tracking After Tracking Using a 3-D Model Difficult Situations Goosing the Solver Inverting Perspective Overall Distance Constraints Joint Camera and Object Tracking Workflows Coordinate Setups are Crucial! Survey Shots How to Shoot Survey Shot Setup Tracking Solving Oops! Not a survey. Multi-Shot Tracking Camera/Camera or Camera/Viewport Matching Camera/Perspective View Matching Match By Name Ready to Solve ViewShift ViewShift Fundamentals Output Modes Source Frame Timing Modes Basic Timing Modes Distance Mode Same Direction for Tripod Shots Controlling ViewShift with Splines Controlling ViewShift with Meshes Disjoint Overlap Modes Softening Edges with Alpha Illumination Compensation Working from a Clean Plate Animated Texture Map Extraction Multiple ViewShifts Overlaying Multiple ViewShifts Running Multiple ViewShifts at Once ViewShifting 360VR Stereoscopic Movies What’s Different About Stereo Match-Moving? 24/7 Perspective Easier Sizing Basics of 3-D Camera Rigs Synchronization One-Toe vs. Two-Toe Camera Rigs From Where to Where? Dynamic Rigs Camera/Camera Stereo Geometry Parameters Two-Toe, Revisited 3-D Rig Calibration Overview Electronic Calibration Opening Shots Stereoscopic Tracking Automatic Supervised Supervised Setup using Camera+Perspective Views Creating Stereo Pairs by Linking Stereo Pair Selection Doing the Stereo Tracking Stereo ZWTs Checking Stereo Correspondence Configuring a Stereo Solve Stereo Solving Mode Automatic/Indirectly Solving Mode Tripod/Indirectly Solving Mode Setting Up Constraints Weights Identical Lens FOV Weight Handling Shots with Changing IOD and/or Vergence Post-Stereo-Solve Checkup Stereoscopic Object Tracking Interactions and Limitations 360 Degree Virtual Reality Tracking Simple Stabilization 3D Tracking, Stabilization, and Effects Workflow Native 360VR Solving Special 360VR Solving Considerations 3D Solves from a Linearized Shot Using Add Many Trackers Using a Roto-d Auto-Track Manipulating the Viewing Direction Inserting Rendered 3D Objects Using External 360 VR Cameras Areas of SynthEyes Supporting 360 VR Footage Scene Exporters Supporting 360VR 360 VR Scripts Reference 3D Tracking Setup 3D Tracking Wrap Unstabilized 3D Tracking Wrapup Align Camera to Path AfterEffects 360VR Stabilization Blend Stabilizations Importer Copy ONLY Adjustment Tracks Copy Stabilization Create Next Piece Create Spherical Screen Export Stabilization Fine Tune Rotation Follow Mesh Fusion 360VR Stabilization Generate VR Lens Map Importer HitFilm 360VR Stabilization Import Stabilization Level Scene from Frame Re-aim stabilized shot Remove Stabilization SkyBox AE 360VR Stabilization Stabilize from Camera Path Unfollow Mesh Motion Capture and Face Tracking Introduction Comparison of Motion Capture and GeoH Tracking Camera Types Camera Placement Lighting Calibration Requirements and Fixturing Camera Synchronization Camera Calibration Process Matching Plan A: Temporary Alignment Matching Plan B: Side by Side Matching Plan C: Multiple Camera Views Matching Plan D: Cross Link by Name Completing the Calibration Body and Facial Tracking Marks Preparation for Two-Dimensional Tracking Two-Dimensional Tracking Linking the Shots Solving Exports & Rigging Modeling Single-Camera Motion Capture Light Tracking Light Illumination Lights from Shadows Lights from Highlights Curve Tracking and Analysis in 3-D Terminology Overview Shot Planning and Limitations Initial Curve Setup Roughing in the Curve Keys Manual Roughing Automatic Roughing Curve Tuning Flex Creation Editing a Flex Flex Endpoints Flex Solving Flex Exports Optimizing for Real-Time Playback Image Storage Refresh Rate Optimization Actual-Speed Playback Safe Areas Troubleshooting Log File Crashes Catastrophe: Out of Memory! Recovering Crash Dumps SNI File Recovery Merging Files and Tracks File/Merge Tracker Data Transfer Exporting Importing Transferring 3-D Paths Writing 3-D Tracker Positions Reading 3-D Tracker Positions Batch File Processing Details

‌SynthEyes™ 2024.1 User Manual