Sharp video frame extraction software?

[ThomasVD]

Hi everyone!

I'm currently working on a project in the Netherlands where we are recording a wreck using GoPro video footage; we extract roughly two frames per second from the footage and then align those frames using PhotoScan.

While it is working quite well, at the moment we have the issue that about half of our extracted frames are sharp, while the other half are blurry. Our workflow would be sped up significantly if we could simply get rid of the blurry images, or use frame extraction software which filters and only extracts sharp frames.

Does anyone know of a software package or workflow that could help us obtain only SHARP frames from our video footage?

Any help is much appreciated!

Tom

[bigben]

As a relatively crude check you could try listing the std-dev of the image pixel values for all of the images and then choosing the image with the highest value within a group of 3-5 around the intervals you want. e.g. if you have 15fps, choose the highest std-dev from frames 1,2,3 and again for 8,9,10 for every second.

On Windows for example:

List files in a directory

Code: [Select]

dir /b *.jpg >dir.txt
Using ImageMagick

Code: [Select]

convert @dir.txt -format "%f %[fx:standard_deviation]" info: > stats.txt
reb-UAV1-001.jpg 0.152147
reb-UAV1-002.jpg 0.156188
reb-UAV1-003.jpg 0.156777
reb-UAV1-004.jpg 0.152967 <- deliberately blurred image
reb-UAV1-005.jpg 0.164074
reb-UAV1-006.jpg 0.163621
reb-UAV1-007.jpg 0.162498
reb-UAV1-008.jpg 0.169952
reb-UAV1-009.jpg 0.16501

[ThomasVD]

Thanks for the tip Bigben! 

I'm in the field now but when I'm back home I'll try and do some research for sharp frame extractions - apparently there's some good astronomy software capable of extracting sharp frames from various blurry video frame exposures

[bigben]

That may work, although at a quick glance they seem to be mainly for stacking images ie. comparing images of the same view, so it might not do what you need... but it's a promising approach. 

I wasn't entirely happy looking at the numbers from the script that I first posted as the first image has a similar value to the deliberately blurred image.  Noise and compression artefacts complicate things a bit if you're just using stdev.  To make the test a little more robust, a sharp image should have better "edges", so I added an edge enhancement to the command.

Code: [Select]

convert @dir.txt -edge 3 -format "%f %[fx:standard_deviation]" info: > stats.txt
That looks a bit more useful if you end up falling back to trying this approach.

[James]

Not sure if any pre-existing software out there does this, but another metric you could use is similarity to previous/subsequent frame(s) as that would correspond to camera movement and therefore motion blur

However in my experience, although it may sound intuitive, explaining to a computer what constitutes 'similarity' is not alwaye as easy as it sounds

[dtmcnamara]

If there is a way to do this quickly I would love to hear as well. I am in the same situation where we pull every frame from video and manually go through each to find the best, sharp image. When the video is 30 seconds long and thats it this isnt bad, but most of the time we are dealing with 2-3 min long videos and then 5-8 different shots as well. When you have a folder of 50-100K images you need to go through for one project smiles turn to frowns really fast.

[Wishgranter]

for videogrammetry, try set  export "speed" to 1 - 5fps depend on the camera "speed" movement. and its easier to search for blurry imgs there.. Have good results from 2-4k videos with this workflow..

[ThomasVD]

To revive the discussion a little bit - after some more research it seems there's 3 general avenues that could be followed to solve this issue:

A) remove blur in video, then extract frames
B) extract frames, and use deblurring algorithms to remove blur
C) extract frames, and use algorithms to filter out only the sharp frames

Option A doesn't appear to be very widely used, though there are several software packages claiming to remove blur from videos.

Option B can either work through regular sharpening techniques (such as Photoshop's 'unsharp mask') to deblur out-of-focus images, but there are also some super-impressive algorithms for determining direction of and removing actual motion blur. Haven't really tested any of the software because I haven't seen one that offers batch-processing, but this avenue seems promising.

Option C would appear to be the simplest option to implement, though I haven't yet found easy-to-use software to do this yet. I suppose PhotoScan's 'estimate image quality' function would be similar to this, though this function doesn't seem to do a very good job of filtering out unsharp images. 


Some interesting examples of option B: http://www.cse.cuhk.edu.hk/~leojia/projects/robust_deblur/ , http://research.microsoft.com/en-us/um/redmond/groups/ivm/sharppanoramas/ , http://www.extremetech.com/computing/138363-the-cia-de-blurs-license-plates-from-outer-space-now-you-can-too-for-free .

[bigben]

I've done some more testing of option C with some image sequences I have but they all change too much between images. Will have to shoot something with my GoPro and see how that goes.

[chari]

Good afternoon,

may could I ask you which software do you use to extract still pictures from the video? I got video and would like to get pictures out of it and run test by myself. Thanks,

Chari

[bigben]

Adobe Media Converter, Quicktime Pro and/or VirtualDub depending on the format and my mood  ;-)

[chari]

Thank you for your fast answer. I am going to try that. Hope I will be able to get something out of it.

[ThomasVD]

I just use "Free Video to JPG Converter"

[ThomasVD]

Perhaps another option would be to automatically mask all blurry images in Photoshop CC using the new "focus based image selection" feature: https://helpx.adobe.com/photoshop/how-to/focus-mask-selections.html
- I don't have Photoshop CC so I'm not sure if this feature can also select blurry images, or just images that have low depth of field.

Perhaps someone who does have Photoshop CC could try this and report back on the results?

[bigben]

I'm a little sceptical of demo videos that choose subjects that are obviously going to work... and then fail to pick up parts of the image. The demo could have been equally successful with a magic wand selection on the background.  But in theory, you could save the mask as an alpha and then use the alpha to make some kind of statistical measurement. The biggest problem with that (or any other mathematical measurement) is that your image content will change so it would be difficult to use absolute values as a measure for all images. Comparing images within short sequences, assuming the image content doesn't change much, may be useful but there are still potential problems with that approach.