Forum

Author Topic: 1080p and 4K Video --> Point Cloud --> Mesh  (Read 16817 times)

dtmcnamara

  • Jr. Member
  • **
  • Posts: 73
    • View Profile
1080p and 4K Video --> Point Cloud --> Mesh
« on: March 05, 2015, 06:13:56 PM »
Ive been playing around with a bunch of non traditional ways of creating meshes and point clouds in Agisoft and wanted to share some recent results. Photogrammetry is all just pictures to points to create point clouds, but what is video and can we create pointclouds using individual frames from video?

What I have found is that it is possible and can be faster to capture and record a object with video to process later than taking photos. The nice thing about video is that depending on the camera you can capturing upwards of 60+ frames per second and can really use anywhere from 50-90% of the frames. I say 50-90% because depending on lighting and your shutter speed you might get some blur ever few frames, but either way you will still generally have 500+ useable frames per minute of video recorded. This is nice when time is limited and you need to quickly get in and out of an area.

I have two examples below that I created using two completely different methods of video to mesh design. This first example is from a cemetery. I took a close up, 3-5ft away, 1080p video using a Sony camera. Due to the resolution you will need to keep the camera pretty close in order to get decent detail. The entire video was 30 seconds long to create this model and ended up being 250 pictures once I pulled frames.

https://sketchfab.com/models/337263c53bdc4f908f64c7ffa7b74b61

The second example is a test that I did using Google Earth and Camtasia to screen capture what I was looking at on my monitor. Due to the fact that I have 4K monitors I was able to record my screen at 4K and then process the video the same way I did the model above. The total video was 60 seconds long and I pulled 625 frames to process the entire project.

https://sketchfab.com/models/c29a84f14a3f45cdaf1166cd15756a6e

These scans are obviously not the most accurate ones out there but if time is limited and you have access to decent video cameras this process might work for what you need. Ill be testing a few other projects over the next few weeks and post up more results as I get them.

Hope you all enjoy

James

  • Hero Member
  • *****
  • Posts: 748
    • View Profile
Re: 1080p and 4K Video --> Point Cloud --> Mesh
« Reply #1 on: March 05, 2015, 08:29:09 PM »
That's really cool! I love the 'quick and dirty' (no offence!) solutions to serious problems...

I did a similar thing using google earth images recently on a site in new york, here:

https://sketchfab.com/models/6affda836ef84e0e9aec8df01c40c09d

from about 500 images (i disabled half because i can never wait for more than about 10 minutes processing!) the images are only 1200x750px

i used chronolapse screen capture software (https://code.google.com/p/chronolapse/)

it was a kind of joke because a client asked for a 'google earth level of detail model' of this site, and i just wondered if it could be done without having to cross the atlantic :)

Marcel

  • Sr. Member
  • ****
  • Posts: 309
    • View Profile
Re: 1080p and 4K Video --> Point Cloud --> Mesh
« Reply #2 on: March 06, 2015, 11:58:11 AM »
Awesome idea using Google Earth as source images :)


I am wondering if it makes sense to use every frame of the video? The camera positions will be very close to each other, so the amount of information Photoscan can extract is low. (With regular photos, if cameras are close together it can lead to bad alignments and noise in the scan)

dtmcnamara

  • Jr. Member
  • **
  • Posts: 73
    • View Profile
Re: 1080p and 4K Video --> Point Cloud --> Mesh
« Reply #3 on: March 06, 2015, 02:18:57 PM »
The only reason i used so many photos was becuase of the resolution of the recording and the length of the video. I will be testing out zooming in more and using less pictures later this weekend to see how things work this way.

Mr Whippy

  • Newbie
  • *
  • Posts: 26
    • View Profile
Re: 1080p and 4K Video --> Point Cloud --> Mesh
« Reply #4 on: March 10, 2015, 07:17:31 PM »
I've been capturing images from GE (automated fly-overs with interval capture using FRAPS or whatever), or manual ones, and rebuilding.

The results can be quite good for some quick/dirty projects, vs having nothing at all.

I'm working on a project right now and my rough out source is a point cloud from GE captures.

Infact the only stumbling block is getting a nice alignment after construction. Oh for an align to camera option in Photoscan. I mentioned it over a year ago and it's sad to still not see it added.
So far I need to export camera positions then do a transform in Cloud Compare to the camera position. A bit of a faff!






I've also built a few clouds from random videos on the internet. I worked on one project where someone was nearby a rapidly changing environment that had changed a lot since a site visit, and they'd been flying over with UAV.

Even with rolling shutter issues and so on, I managed to get a really nice rebuild. The point cloud was quite noisy, but with filtering and so on I had a fantastic top-down layout, and the height data was good enough to pick ok average values from.




Agisoft Photoscan is really excellent imo.


OK for some tasks where you want perfection, then perfect inputs need to be captured and then that costs time/money... but for lots of projects it's amazing how you can trim time/costs in capture and get a good technique and still get pretty good results for your needs!



So thumbs up to Agisoft!

But please add an align cloud to camera option!


Dave

thesystemera

  • Newbie
  • *
  • Posts: 19
    • View Profile
    • realityvirtual
Re: 1080p and 4K Video --> Point Cloud --> Mesh
« Reply #5 on: March 14, 2015, 09:43:05 AM »
Man this is very interesting stuff guys. Might try this with my GoPro. The GoPro has a real fast shutter under good lighting conditions, compression of video is a little high though. Oh well, will give it a go. Got plenty of source video from our drones.
Simon Che de Boer @ http://www.realityvirtual.co/

igor73

  • Full Member
  • ***
  • Posts: 228
    • View Profile
Re: 1080p and 4K Video --> Point Cloud --> Mesh
« Reply #6 on: March 14, 2015, 07:21:30 PM »
thesystemera,  please let us know how it goes.  Have the new inspire 1 drone   http://www.dji.com/product/inspire-1 .  Have just done one test using stills from the camera and got great results.   Looking at 4K  video frame grabs the images look very good so have to try using video to.  Would make it easier if the quality is near the same.  No exif data though but i suppose i could just save a calibration profile from a project aligned with stills and apply that. 

When using vide fram grabs have any one noticed improvements by calibrating? 

Mr Whippy

  • Newbie
  • *
  • Posts: 26
    • View Profile
Re: 1080p and 4K Video --> Point Cloud --> Mesh
« Reply #7 on: March 21, 2015, 07:31:07 PM »
I recently received some feedback on the project I worked on which I noted earlier.

I can't go too far into details probably because it's all NDA stuff, but needless to say, a random rolling shutter video off Vimeo of the site gave me an accurate enough point cloud to generate my project with.

The real life data collected for their intended purpose matches what I provided for them to utilise in a simulated environment which is very reassuring from my perspective.

I'll have to give some rough numbers for it to make sense, but I'd guesstimate at a drone flying about 75m high with the camera looking forward about 35deg-45deg along the direction of travel.
Over about 200 images from the video, and about 2km, I had very little vertical or lateral drift... to the point that I went with the point cloud data for xy and z, over the formal xy plans which were obviously vector based, based off aerial imagery, but with the issue of height parallax distorting features that moved up/down in altitude over the location.



In the end I dumped every 30-40 frames iirc, so I'd go as far as saying something like a Ricoh GR on 1s interval without the rolling shutter issues, and a higher resolution and quality outputs, would result in superior quality results by far.



I just wonder if the DJI Phantom frame could work nicely with a Ricoh GR. Hmmm.


Cheers

Dave

dennison

  • Newbie
  • *
  • Posts: 1
    • View Profile
Re: 1080p and 4K Video --> Point Cloud --> Mesh
« Reply #8 on: June 17, 2015, 09:36:28 PM »
I did an experiment capturing a fashion model using HD video on a Canon EOS M. It was very quick but it did work. I imagine with more care, (sharper focus) and 4K a really quite high quality scan could be made. I did not use Agisoft software (Autocad 123D) but I thought you might be interested as well. There is very little so far online about creating 3D scans from video.

[sketchfab]7ed5760c1d064dcd80f7e4a62f3acfca[/sketchfab]
Camille - Captured from HD Video. by dennisonb on Sketchfab