Problems Making 3D Model From Video Frames

[rdobson82]

I am trying to make 3D models from extracted video frames. I am not shooting HD video but rather taking 4K (8.8 MP) and 5K (13.8 MP) video. I am reconstructing a relatively flat surface and after aligning the photos the point cloud shows a distinctive arch that is not accurate (see attached).

I have made 3D models of this surface before using several different Nikon DSLRs with recent versions of Agisoft without a problem. I have tried to edit the EXIF data by adding the focal length in (since it is not the 50mm default) and tried changing the Fx and Fy in the camera calibration. I have also tried constructing small chunks as well as the full data set of 80 frames and both versions Agisoft Photoscan Pro 0.9 and 0.9.1 but the same problem persists.

Any thoughts?

[Wishgranter]

Try with higher overlap, say 2-3 times more photos and report back..... you get better align but it will be not ideal, have just one strips flyovers or more near each other ??

[rdobson82]

Hi Wishgranter,

The project that I showed in my first post had a little more than 60% overlap. So a feature on the ground would be represented in at least two frames. I have attached another example with increased overlap where a feature on the ground is represented in least four frames. I still have the same problem.

[Wishgranter]

Hi Robson, this sort of problem was few times discussed, 60% is small for some stuff like this( "low" res images, try to go much higher overlap say 100-120+ % that mean not just one line but more of them overlapping same area.. Personally im get best results with overlap over 500 % for some stuff that have low "texture-features" on it......

So if can try run it with more images, and if possible use higher res images( 18+ Mpix) for this sort of stuff , then you need lower overlap altogether...... 

If possible upload the dataset somewhere and will try show few things how to process it...

[rdobson82]

That's interesting... I have made reconstructions of the same location with a 12.3 MP Nikon D5000. I have attached an example point cloud from D5000 photos of the same location and photo overlap as the 5K video frames that I attached previously. The photos from the D5000 are aligned properly and the 5K video frames are not...

The reason I am pursuing the video route now and not just continuing to use the D5000 which has always worked for me is that I need to have a higher frame rate. I have a Nikon D800 that will give me 4 frames/sec but for what I'm doing I need at least 20 frames/sec.

I have also posted example frames from the 4K, 5K and D5000 collect to our ftp for you to try.

ftp://ftp.mtri.org/pub/Agisoft/

Thanks for your help!

[Wishgranter]

i think this is something that could interest you  ....

http://www.xbitlabs.com/news/memory/display/20130716215020_Toshiba_Introduces_SD_Memory_Cards_with_SSD_Like_Performance.html

[rdobson82]

That is pretty neat. I use Lexar Professional 1000x compact flash cards in my D800. They have a write speed of 145 MB/sec. It would be interesting to see what the Exceria Pro cards could do for me in my other cameras.

Any additional thoughts on the video frame issue? I just find it interesting that the D5000 with 12.3 MP works and the video at 13.8 MP per frame does not.

Also I don't have access to the RED Epic for the little while so I wont be able to try a high frame rate or the multiple flight line suggestions at this time.

[Alexey Pasumansky]

Hello rdobson82,

And what where the focal lengths and sensor pixel sizes for 4K and 5K datasets?

Also it looks like that shadow remain more or less still on every frame - this may cause false matches between images.

[bisenberger]

I had a project that looked arched. The camera EXIF didn't have enough information for the calibration. I created a calibration file using Agisoft Lens and ran the project again using the Lens calibration file. It straightened the model.

Don't know if a calibration can be done on video frames.

[rdobson82]

Hi everyone,

So after a fare amount of testing with more imagery at various resolutions and overlap, I think I have found the root of the problem. In an earlier post I have mentioned using a 12.3 MP Nikon D5000 camera to make models and everything worked fine a couple of years ago. The extracted video frames I used have the same overlap and are 13.8 MP but I was not having any success.

So I went back to the original imagery that I took two years ago and ran it through PhotoScan Pro 0.9.1 and I was not able to make a 3D model... In fact I was seeing the same issues as with the video frames. I then loaded the older version (0.8.4 build 1289) that I used two years ago on another computer. I then ran the same set of photos from the D5000 through each version. The older version produced a 3D model while the newer version didn't. I have attached screenshots of the results. What's even more interesting is that I was able to generate an accurate model when running the extracted video frames though the older version of Agisoft.

Was something changed in the way models are constructed between the these two versions?

Thanks

[Dmitry Semyonov]

Hello rdobson82,

Can you please provide the project file you have obtained with 0.9.1 version (preferably together with source images)?

It looks like something was done wrong during processing, maybe the reconstruction volume is not specified properly and doesn't include surface points.

We can provide an FTP account if needed (please send an email to support@agisoft.ru).

[7eicher]

Dear rdobson82,

i am interested in your prefered video setup. Could you share some insights please? I tried it one time myself, but i stopped after extracting stills from myD800 video, as they were too blured cause of the movement. Do you find any solution to that with the D800 or you would recommend using a prof. video cam, with higher frame rate? What image extraction tool did you use, quick time?

Would be thankful to hear from you.

Take care,

Markus

[rdobson82]

Hi 7eicher,

For taking video I was using a RED Epic while I used the D800 to capture full frame pictures at 4 fps. The video resolution on the D800 is only 2 MP per frame which is too little for what I want to do. The RED Epic is capable of 5K video which is 13.8 MP per frame but its also significantly more expensive. I have used VLC to extract frames from video captured from GoPros but the RED Epic requires Adobe Premiere.

The problem of motion blur can be solved by adjusting the shutter speed. Faster shutter speeds will "stop" the motion across the sensor. To do this you'll have to test and adjust your camera for your setup (shutter speed, aperture and ISO), the object you are shooting at and the amount of light. Higher frame rates will only allow for more overlap of the frames.

[7eicher]

Thanks rdobson82 for your insights.

Using video in interieur seems therefore very hard, as light isn't enough to have faster shutter speed in order to get sharp stills without blurring effect.

Take care,

Markus