Workflow for VFX shots with Nuke regarding camera calibration

[shakebox]

Hi there!

Just have a project where I shot alot of still photos on set to recreate a dummy version of this set. Set is a large square with small surrounding buildings. Recreating the model in Photoscan worked fine so far.

Now my idea for doing the actual shots (most of them locked cameras) was to get 1 frame from each shot, import it into my final Photoscan scene, add the camera data from the film camera (focal length, sensor size, etc.) and have them aligned as well, so I can export the camera positions for all the shots with my model and add all the additional stuff in relation to this dummy set and render with the cameras I got for all the shots.

But there is one problem I see with this process: camera calibration/lensdistortion
If I get correct camera positions for all my shots, they'll match to the undistorted image but not to my original footage of the shots.
Normaly when using camera tracking software I'll use a matching lensdistortion node in Nuke to apply the lensdistortion to the 3D rendering, so it fits on my original footage. For Syntheyes or other apps there are nodes for Nuke that uses the same algorithm as the camera tracking software. But how do I apply the Photoscan lensdistortion in Nuke? Is there a node/algorithm to use that gives the same result as when doing the undistort in Photoscan?

Using Photoscan for this kind of VFX work (especially with shots with locked camera) would be really helpful, but if I can't get the resulting model (or more precise: renderings of this model) to match my original footage, it's quite useless.

Any ideas? Thanks!

[Mr Whippy]

There is camera calibration information that Photoscan will create for your images automatically, or there is Agisoft Lens which can generate (probably) more accurate lens distortion information for your camera.

I'm assuming that with this data you can script some distortion parameters in Nuke. Or perhaps the data is even Nuke compatible as there are XML exports available (export cameras)... I assume these fit some general lens distortion conventions that Nuke will be compatible with?

[vlanger]

there are no general lens distortion conventions

[Mr Whippy]

Well if you were to use "camera X" and un-distort in Nuke for any old project Agisoft aside, some kind of process must be available to do that already any way (which we know it is)

Given Agisoft's purpose is to produce a 3D model only, not detect and generate lens distortion profiles for use externally for other purposes, then it's not a surprise to see Agisoft not supporting lens profile generation for all the softwares out there, especially if there are no conventions on storing that data.



Using Photoscan for this kind of work is perfectly reasonable without the camera distortion data isn't it?

Just like ANY 3D asset that you might comp into un-distorted footage, the un-distortion is up to you, not the person/tool that helps you make the 3D asset to begin with?


Or am I missing something? I might be missing something hehe, but I can't see what the problem is right now.
Just un-distort however you would doing any old 3D comped into real life footage?


Hmmm

Dave

[shakebox]

yes, of course I could do that. My idea was to use Photoscans align feature to also match still frames from my film shots. To get a good alignment, I thought it would be necessary to also let Photoscan do the undistortion of these frames.

So you suggest to do the undistortion in Nuke (which will have a different undistortion geometry because of different algorithms), using these undistorted images in Photoscan, deactivate the undistortion part in Photoscan, align them and use the found cameras for rendering the model and distort these renderings with the undistortion from Nuke. Right?

Is it possible to deactivate the undistortion in Photoscan and will the alignment possibly work if the image is not undistorted with the Photoscan algorithms?

Thanks, Abraham

[Marcel]

We use photos which have been corrected for lens distortion in Adobe camera Raw all the time,  and the results are excellent. We don't disable any distortion correction in Photoscan (not sure if that is even possible),  we assume that Photoscan just detects that the images are free of lens distortion. There are certainly no alignment problems.