How far can a GoPro go...

[ozbigben]

First off let me start by saying that I know a GoPro is not the ideal camera for photogrammetry and I wouldn't consider using it in future. I bought mine for cycling/hiking videos and I've taken a few image sets to try out in 123D Catch before getting Photoscan.  As part of getting familiar with PS I've been seeing how good a model I could get using a GoPro and the standard version of PS. It's taught me a lot and I thought I'd share my findings.

1. Lower your expectations 

2. Forget "wide". The "normal" fov may only result in a 7mp image, effectively being just a crop of the 12mp wide image, but it avoids getting a lot of distant stuff in your sparse point cloud which you end up cropping out, reducing the number of points for the next steps in the workflow.

3. You need a lot more images and need to photograph a lot closer than you might think. I shot these sets on timelapse at 0.5secs per frame

4. Mask the sky.  I set up a photoshop action to mask out most of the sky before starting. I try and shoot on lightly overcast days so the sky is usually white.

• Add a white border to the image
• Magic wand select the top left corner with a tolerance of about 20
• Invert the selection
• Create an alpha channel from the selection
• Remove the white border
• Save as TIFF

This provides a reasonably good starting point for creating masks.  Adding the white border lets the selection grow around trees/other objects that extend to the edge of the frame. Might look at using Imagemagick for that as it might be quicker (and simpler)

5. Create a sparse point cloud with at least 500,000 points
Remove "bad" points using gradual selection: Reprojection error 0.5, Reconstruction uncertainty 50 ( I know I've been exploring other values for these settings, but these seem to be consistent for my GoPro image sets)

From there, do everything at as high quality settings as you can cope with, export and clean up the mesh.

Attached a screengrab of the latest test. 360 images (shot as 12mp, wide) around the sculpture in the middle (I didn't chose that number, just a coincidence after removing under-exposed frames)  This is the actual location: http://www.360cities.net/image/sheep-shape-behind-the-arts-centre-melbourne-victoria/#288.71,1.34,63.1

Sheep sculpture: http://studio.verold.com/projects/5394f586372e88020000043d (easier to see the mesh quality without texture)

Tree trunk (~180 images) http://studio.verold.com/projects/5392aa11eef3b50200000077

[Marcel]

Thanks for sharing, it's interesting stuff!

I would still like to try GoPro video one day, but from a very close distance. For example, at 20cm distance when scanning something like a tree trunk. I wonder if you could get a comparable quality to a good DSLR at longer distance (probably not).

[bigben]

I think time lapse with normal(7mp) would give you the best compromise. I doubt it could compete with DSLR images for efficiency as you would be processing a large number of images to get the same quality and the JPEG compression on the GoPro can be a bit of a killer. It would be a little better if you had some control over the compression level rather than just the resolution.

[SAV]

As mentioned by ozbigben, the GoPro camera is not ideal for photogrammetry due to it's focal length (2.77mm) and and the rolling shutter (which is a problem if the object and/or the camera are moving). You will get skewed images, which is a no-go for photogrammetry.

I did some PS processing with images from a GoPro Hero black 3 which was attached to a quadcopter and flying at about 15-20m above the ground (covering an area of 200m x 100m). Even though I had highly accurate GCPs (using RTK GPS, accuracy of 0.006m!!), the resulting model had a total error of 0.68m. I tried all possible settings, but 0.68m was the best I could get. This means that the GoPro's focal length and rolling shutter lead to an increased error.

I am also using a Sony Nex 7 - 16mm lens, attached to a hexacopter. The resulting DSMs I get from this setup are outstanding. I can get accuracy values which are within the error of the GCP (again, aquired with RTK GPS).

Cheers
SAV

[Porly]

Hello,

I wanted to share my experience withe the GoPro and a cheap UAV. Against all the opinions in the internet that the GoPro is nothing for photogrammetry, I tried it anyway.

First I have to say that my GoPro is modified with a 60° lense. So that it is possibile to use the entire sensor (12MP) without big disortion (no fish eye). The focal length is twice bigger, from 2.7 to 5.5 mm. Using a vibration resistent gimbal on my DJI Phantom 2, I have never had problems with the jello effect caused by the rolling shutter (Flightspeed 5-7 m/s @ 100m height).

The resulting pointclouds are very good without any noise. We also have a professional UAV System from MAVINCI with a Sony NEX 5, so that it was possible to compare the results. I was realy wondered about the fact, that there was no difference between the quality between both meshes (DJ Phantom2 + GoPro and Mavinvi SIRIUS I + Nex 5). Out of the resulting point clouds I have created a DEM for each system and combined this in Global Mapper. The difference Model showed a perfect plane with a z variation of +- 5cm, what is reasonable using two independent datasets from differnt days.

No doubt, the quality of an orthophoto generated with the NEX 5 is still much better than with the GoPro Cam, but for me there is no difference in the dense point cloud quality.

I am not sure but I guess it was in April 2014 when DJI released the Ground Station Software for the Phantom 2. For 150€ you have a autonomous flight controler that makes waypoint flights possibile with the DJI Phantom 2 (Up to 15 waypoints with NAZA V2). The software includes the photogrametric tool where you can plan your flight, including overlapping by given altitude. It works very good. Until yet I had over 30 autonomous flights without any problems. Even the automatic landing works fine.

Surely each system has its advantage and disadvantage:

Capturing Area of one flight with the MAVINCI SIRIUS I ~ 1,5*1,5km (max flight time ~ 45 min @ 17 m/s (no wind)) = 2250000 m^2

Capturing Area of one flight with the DJI Phantom 2 ~ 450m*300m (max flight time 15min @ 5-7 m/s (no wind)) = 135000 m^2

It becomes quiet clear that for huge areas, the cheap DJI drone is not a replacement for the MAVINCI plane. The area of the Mavinvi system is 15 times bigger than of the DJI Quadcopter!


Mavinci GSD of 3CM by ~ 150m altitude
DJI Phantom 2 GSD of 3CM by ~ 100m altitude

Image quality of the Nex 5 is obviously better, without any doubts.

BUT:
With the DJI Phantom 2 you dont need a big area for starting and landing, and you dont have such a material consumption due the landing, since the MAVINCI UAV lands with the body on the ground.

Other advantage of the copter vs plane is, that we can make flights deeper and shoot oblique photos. But these are well known advatages of each multicopter, nothing special.

I dont want to advertise the cheap DJI drone. These are my experiences and for small areas it is a perfect solution for me and my task field (volume calculation, terrain mapping and orthophoto
generation).

I think that it is worth to know and share that such a system costs only 2300€ including modified GOPro and full FPV System w/ diversity receiver, H3-2D 2 axis Gimbal, datalink and so on.

Best Regards!

Paul

[Porly]

I recommend to use calibrated camera file as initial parameters for BBA before align photos. For this you can use a testfield with a reasonable amount of known GCP's.

[ozbigben]

Hi Paul. That's an interesting system, although technically that's a significant modification to the Gopro so it's not an entirely fair comparison. 

I did save a calibration for the normal view but I haven't done one for wide yet.  I use a specific target (a laneway with a brick wall and cobble stone paving) and camera orientations for my calibrations. It's still a little agricultural but follows a similar principle to the process I use for calibrating my lenses for stitching 360° panoramas.  I found that this can improve the quality obtained from image sets that are not "ideal".

[SAV]

Hi Paul,

Thanks for sharing your experience. Seems like it is worth to modify the GoPro. Btw, where did you purchase the 60 degree lens?

The rolling shutter is more of an issue in my case because a) I fly at a low altitude of only 10 to 25m (at low speeds) and b) I do not have a gimbal. Might be worth to invest in one 

I reprocessed some old images today, and thanks to Porly's post earlier this week I thought I would give masking another try. I already used it to remove people (the pilot, myself) and waves / reflections in the water) from the images. But today I generated a mask in Photoshop and applied it to all images in order to get rid off all the blurry areas around the edge of the images (see attached png-file, works for GoPro Hero 3 black images with 12 MP, WIDE setting). So you can still use shoot in WIDE with 12MPG (instead of approx 7MP when in normal as mentioned by Porly).
I must say, the result is much better. Ground control points are accurate. Dense point cloud and mesh look both great, without a lot of "cleaning" necessary.

Happy with the results from the GoPro Hero 3 black.

The Nex 7 on a hexacopter will of course give you better pictures, but at a much higher price. Plus you have a quite bulky setup. I actually prefer to have a small system like a Phantom for photogrammetry because of several reasons.

a) CASA seems to change the laws for UAVs here in Australia and will allow to fly small ones (under 2kg) without a special operators certificate  They say that it has not enough kinetic energy to really harm a person. I agree.
b) It is less expensive to buy it and to repair it in case it crashes 
c) Easier to carry around (important for the work I do, because I work in remote areas where I have to carry the equipment for several km)
d) You can take it on a plane as carry on luggage
e) Charging batteries in the field is much easier (you only need a car and DJI's special car battery charger). Try to charge a 6S LiPO with 10000mAh in the field. Possible, but expensive.... and heavy.

Really like the exchange of experience here in the forum. Great stuff guys, keep up the good work.

Cheers
SAV

[Marcel]

I would make the mask even tighter, and round the corners instead of a square mask (the drop off in lens quality is circular). I assume you have enough overlap with the 60 degree fov lens.

[SAV]

Good point, Marcel. I'll definitely round the corners of the mask.

I have taken more pictures than I really need, so overlap should not be an issue.

Cheers
SAV

[Porly]

Btw, where did you purchase the 60 degree lens?

Hello SAV,
here the link to the shop 
http://www.vd-shop.de/set-bestehend-aus-dem-54mm-10mp-megapixelobjektiv-302116-und-diy-objektivhalter-fuer-die-gopro-p-560.html

Best regards

Paul

[Marcel]

Btw, where did you purchase the 60 degree lens?

Hello SAV,
here the link to the shop 
http://www.vd-shop.de/set-bestehend-aus-dem-54mm-10mp-megapixelobjektiv-302116-und-diy-objektivhalter-fuer-die-gopro-p-560.html

Best regards

Paul

That's quite a pricey lens. Won't a light compact camera have better quality for the same price? (+ the cost of a GoPro)? Or aren't there any small cameras in the same weight class?

[Porly]

There is no other light camera that is compatible with the 2/3 axis Gimbal, as far as I know. With the Gimbal you have already a Plug n Play FPV Port that works with the Phantom 2 in combination with the GoPro.

Of course the price Lens+GoPro is comparable with a DSLR or DSML camera, but they are too heavy

By the way a gimbal for the nex 5 costs ~1500€

I have already seen setups with cheaper compact cameras (bigger sensor), but there were used without gimbal. FPV is not so easy to realize with each camera model. Also the simple option of continuos shoot of photos.

(german)
http://www.kopterforum.de/topic/5625-dji-phantom-und-sony-rx100/

I am not an expert in these things, but there people using a sony rx 100 with the phantom 2. Maybe there is a custom way to make it FPV ready, I dont know. But it will not be easy. There is no official Gimbal for these cameras.

[ozbigben]

But today I generated a mask in Photoshop and applied it to all images in order to get rid off all the blurry areas around the edge of the images (see attached png-file, works for GoPro Hero 3 black images with 12 MP, WIDE setting).

If your mask gets too close to the one attached here, just switch to 7MP, medium. I ran a test and confirmed that 7MP medium is just a 12MP wide cropped.

[SAV]

I did the same test. The pictures are 3000px x 2250px, simply a cropped version of the full 4000 x 3000 image. But you might/will see changes in ISO and shutter speed, because the camera will adjust these values to the cropped portion of the photo.

So you can now choose what you want to do:
1) Shoot in wide and apply a mask later in PhotoScan
2) Shoot in wide and crop to 3000x2250 using Photoshop or IrfanView (batch process)
3) Shoot in medium and directly use the images

Anyway, there should not be a big difference at the end.

Cheers
SAV