www.ssontech.com

[ssontech]

If you're like me, you've been reading more and more about how 3-D movies are the next big thing, not sometime in the future, but now. And there are high-profile 3-D projects such as U2 3D and Avatar. I think that's great. 3-D films are fun to watch, and the kind of compelling experience that will draw audiences to watch them.

SynthEyes has been used for 3-D projects, including Magnificent Desolation, which I'm particularly fond of, and there's a cool way to handle 3-D tracking in SynthEyes by reversing one of the shots.

A few people have asked about adding stereo features in SynthEyes, to make it easier to match-move stereo imagery, and be able to get 3-D information out of static or nodal shots. I'd love to add more stereo capabilities. [There's even been some disabled code in there for years.] I have a whole sheaf of notes of nifty things to do to help handle stereo imagery. I had even blocked out some time to work on them.

Here's the rub: what you can do with stereo imagery depends on what you know about the camera. The more you know, the more things you can do (and with less work), and the more accurate the results can be, in terms of consistency between the two views.

I asked a variety of people doing 3-D work what they could tell me about the relationship between the two cameras, what degrees of freedom there were, how closely they were matching the lenses, what they were doing about mirroring the convergence between cameras, etc.

I'm still waiting for a single reply.

This is the time for the industry to "get its act together" to systematize 3-D cameras, and adopt a common approach, so that tool vendors can respond with tools that make handling 3-D footage as simple as possible.

By far, it is conceptually soundest (especially shooting on film or high-resolution camera such as Red) to shoot with a camera rig that is converged at infinity (camera axes exactly parallel). 3-D Camera vendors could "focus" on calibrating their rigs to that standard as exactly as possible, mechanically and possibly with a standardized software assist. Why bother trying to have a complex mechanism and encoder scheme, and control it during the shoot? Cameras are simpler, lighter, more reliable and more repeatable with fixed infinite vergence.

With such a setup, convergence effects are generated in post. Hey, the director's going to want to change convergence then anyway...

The only complication with this scheme involves shots where the camera/subject distance becomes relatively small, down into the 2-foot range. In that case, infinite vergence isn't such a good idea. Instead, I propose using a separate camera rig with a fixed calibrated vergence. That might be a different mounting plate, or could be a different camera completely. It's going to need different lenses anyway.

If you're working with 3-D cameras, I hope I'll hear from you. They aren't just two cameras bolted together.

[jw]

I just finished working on a 3d film last year with Frantic Films. It was quite a fun challenge and also a very frustrating one, but the convergence worked great in the end for us. Its quite funny how many convergence changes came from the director in post.

[rtep]

Anyone has something else to say about stereo tracking in SynthEyes?

Now I'm at the part when you're about to create a fixed stereo rig movement from two independently solved cameras... Know this can be done by some very beautiful script, based probably on mocalib one... But my Sizzle and math skills are million miles below this...

So I solve both eye views as best I can, trying to maintain stereo relationship as much as possible with help of some gentle and sweet position constrains...

Then I'm about to do some crazy setup in Maya, allowing me creating a camera rig movement based on smart averaging two solved cameras, with optional animated weighting to one of solved cameras, instead of just averaging, in cases where I know that second camera don't have that accurate translation data so averaging will not be the best idea...

This new position data will be brought back to SE and further refined...

This is an approach I belive in... But anyone has some other tips?

From your experience, how crucial and important is to strictly maintain stereo relationship between cameras, with exactly the same movement of both, except the interocular offset and symmetric convergence in non-paralel cases? With no stereo rig constrains in SE2008 this is pretty hard (and not totally accurate) to achieve this by such cheating as desribed above...

Did you face cases where "only" accurate tracking of both eye sequences were good enough to produce stereo-useable result? I'd guess this will be when you're lucky enough to have loads of corresponding trackable features in both eye views very close to cameras and when you're placing CG only behind these close trackers, not in front of them, close to cameras, where every bad thing, like moving one camera slightly back and one forward, will be obvious and probably cause eye-strain...

Or did you find it sufficient sometimes when you tracked only one cam, created the second one based on known or estimated interocular+convergence data, and just refined rotations of second one to give it the right jitter?

Thanks for any help!

Petr

[eoffermann]

I know I'm following up on this post pretty late in the game, but it'll be here for posterity's sake anyway next time someone rolls through the board looking for info on this.

The brain is a very forgiving thing - our visual centers aren't very critical of what they see: blurred grainy photos of a roughly manlike shape are Bigfoot, an oblong dark shape in the sky is a Flying Saucer. They just want to make sense of it and aren't really developed to notice if the interocular distance for Bigfoot is different from the interocular distance for the trees he's standing in front of.

I've found that while a more sophisticated effects shot may require true stereo tracking, maybe even some custom tool design to really make the camera data your slave, there's PLENTY of stereo tracking that only requires one camera track and some basic information about the stereo rig. Convergence is immeasurably easy to hand-animate convincingly and accurate interocular distance is much less important than you might think. Think about interocular distance this way: If your camera position in a standard track was off by a centimeter or two, how likely is it to matter? The answer is, unless your area of interest is shorter than half an arm's length, it probably doesn't! Just keep the left eye on the left and the right eye on the right and if the notes from set (or the specs for rigs with non-adjustable IOD) indicate what the interocular distance was: use that, otherwise, make it around 2.5cm and be done with it. You'll be fine.

There are, of course, exceptions to that. But these are situations where the director or the vfx supervisor or the DP wanted a very specific effect - for instance, the viewer needs to feel very tiny, like they're riding on the back of a bee, buzzing up to a flower. But special cases come up all the time in visual effects and shouldn't scare you away from taking the easy route to get great results when it works.

I also wrote on my blog about the subject of interocular distance and convergence distance on 3d perception at the end of last year. There's a lot more to write, but it's a good jumping off point with some nice videos if you've got a handy pair of anaglyph glasses (there's even a link in the blog to order some if you don't).
http://eddieoffermann.com/blog/?p=61

And of course, in the months since this thread started, SynthEyes has added some good stereo tracking capabilities out of the box which also have a lot of attributes revealed to Sizzle, so all sorts of advanced tools can be built to support your particular needs.

[rtep]

Thanks for reply Eddie.

I've learnt AN INCREDIBLE amount of stuff related to stereo when leading the tracking work on one reaaaly big movie during last eight months...
It was truly crazy and we often hit the wall when trying to get a really accurate tracking, mainly because of the construction of stereo rig and lenses.
It's kinda weird because this stereo technology was meant to be "revolutionary"... hehe. Well at least I'm pretty sure it will never be worse... it will be the same or, much likely, easier

And SynthEyes helped A LOT by the way They always do.