Messages

1

Feature Requests / Re: depthmap hole filling

« on: October 29, 2024, 03:22:08 AM »

Any more thoughts on this?

Seeing more and more developments in depth estimation, pretty much weekly

2

General / Re: Rigid Multi Camera Rig with Parralax Panorama Generation

« on: October 16, 2024, 08:08:29 PM »

There’s some more insight into the issue in the comments of this LinkedIn post: https://www.linkedin.com/posts/adam-havkin-6753b263_gaussiansplatting-3dscanning-realitycapture-ugcPost-7251940144834879488-AwRV?utm_source=share&utm_medium=member_ios

3

General / Rigid Multi Camera Rig with Parralax Panorama Generation

« on: October 16, 2024, 08:04:48 PM »

I’m aligning frames from Insta360 pro 2 footage

I’m taking the raw data from each of the 6X sensors and importing as a rigid multi camera rig

The alignment is great - no problems, but I want to see the stitched panorama from this multi camera rig… So I set this group to “Station” and generate the panorama but it’s all broken up

Do panoramas always need to be perfectly nodal? Is there no way to generate a panorama if there’s a tiny bit of parallax? Would be great if we could get our footage perfectly aligned then generate the stitched frames from it

4

General / Re: Importing FBX & ABC Cameras not Working

« on: October 10, 2024, 11:33:41 PM »

any pointers for this?

5

Feature Requests / Re: depthmap hole filling

« on: October 09, 2024, 02:59:05 PM »

Thanks for the feedback!

I want to emphasize that photogrammetry is a key tool in VFX, and in this field absolute mm-precise accuracy is usually less critical than achieving passable visual results. VFX artists would benefit from more flexibility, and a tool like Agisoft could maybe help by introducing masking for hole-filling - this could allow users to designate areas for filling and exclude zones where precision is required, circumventing the very valid pitfall you described (I didn’t think of that!)

Instead of relying on heavy AI models like DepthCrafter, A custom Stable Diffusion LoRA or hypernetwork could fill holes in depth maps - these run effectively on RTX 30 series GPUs, offering a practical and resource-friendly way to improve outputs without needing crazy hardware.

This approach could really help those of us working in creative industries who don’t always need engineering-level accuracy but do need solid, reliable results.

6

General / Importing FBX & ABC Cameras not Working

« on: October 09, 2024, 11:57:47 AM »

I have a sequence of 900 images that need to be aligned, but I don’t have the time to run a Highest quality alignment on these images

To speed things up, I used an accurate camera tracking software to generate the 3D camera positions which are more temporally stable. I then export these cameras to FBX. They are named identically to their corresponding images.

Here’s the problem - when I export cameras from Metashape as FBX and re-import them, they work fine. But when I export cameras from my tracking software as FBX and re-import them to Metashape, they don’t build a point cloud and the Chunk gets labeled with “[T]”

I’ve tried everything and cannot figure out why a list of cameras in FBX from Metashape is somehow special. Is it something to do with the version of FBX? Binary vs ASCII? Maybe something to do with Keypoints being encoded?

I’ve also ran the same process using Alembic, running into the same issue. Sadly I can’t export as XML or Bundler .OUT from my tracking software… I’ve seen another user struggling with this here:

https://www.agisoft.com/forum/index.php?topic=9845.msg45039#msg45039


TLDR:
What is so special about Metashape’s FBX and ABC file format that I’m unable to replicate. I cannot seem to debug these file types as it seems they cannot be explored via text editors like Notepad

7

Feature Requests / Re: depthmap hole filling

« on: October 09, 2024, 11:42:45 AM »

This just came out and  almost perfectly illustrates what I’m describing:

https://monst3r-project.github.io/

8

General / Re: Tweaks/Tricks for solving low parralax footage reconstruction

« on: October 09, 2024, 11:37:12 AM »

Ah that’s an interesting approah! Thank you

 it would indeed crop the footage, but ultimately what’s most important is a decent mesh reconstruction - I can matchmove to the mesh to the footage afterwards

If I cropped and stabilized, I’m guessing I’d need to adjust my alignment settings to be adaptive?

9

General / Re: Tweaks/Tricks for solving low parralax footage reconstruction

« on: October 08, 2024, 09:08:40 PM »

"because this footage is still handheld, so there will still be micro amounts of parallax even if it's a few centimeters offset here and there"

to reiterate, I'm trying to find out if there's a way to increase the sensitivity to micro changes in parallax. It's not a demand, it's a fair question based on my understanding of Tweaks and how they can be used to drastically improve alignment and meshing when used correctly

10

General / Re: Tweaks/Tricks for solving low parralax footage reconstruction

« on: October 08, 2024, 07:31:10 PM »

olihar - i work in VFX and matchmoving - i know what photogrammetry is... We use the same SFM approach to match cameras in CGI to live action footage. The reason I use Metashape is because I can reconstruct a film location via the GPU which is much faster... Any questions that may seem unusual are from more of an R&D standpoint

As for my question, yeah it's quite urgent and I've been asking all around with no luck. I don't see any consolidated list of tweaks and their effects anywhere so it's a bit of a guessing game and I dont have the time for that. Tweaks have helped me a great deal in the past reconstructing tricky objects, so I'm hoping there's something out there to increase parallax sensitivity, because this footage is still handheld, so there will still be micro amounts of parallax even if it's a few centimeters offset here and there

11

Feature Requests / Re: depthmap hole filling

« on: October 08, 2024, 06:30:01 PM »

Hey Alex


Thanks for getting back to me


Like I mentioned earlier, featureless or inconsistent surfaces (like reflective glass/metal), typically appear as dead zones in depth map generation, or just introduce high amounts of noise.

Tools like TenCent's DepthCrafter have the ability to produce depth maps that are temporally stable (no flicker) and also are rarely negatively affected by reflections, etc...


So as an example, let's say we are having to work with 4K or 8K video frames, maybe for VFX purposes - we might only have a limited amount of data captured from the location rather than a full survey, so we have to try and make a reconstruction from just a couple of clips. These clips may have been filmed in a room with plain white walls. We are able to reconstruct *some* of the detail in the walls due to marks and texture, but there is a lot of data still missing in the depth maps. This is where a hole filling tool comes in handy, essentially content-aware filling in the depth maps based on the recognized scenery. Now we have full "complete" depth maps for every image which should hopefully combine together to make a clean consistent mesh even for featureless objects.


Another paper that just dropped was Apple's Depth Pro https://machinelearning.apple.com/research/depth-pro - Astonishingly detailed results. Whilst I don't know if it's temporally stable, it's amazing fidelity and accuracy...


In 2024, we really should have more advanced features to help us reclaim lost detail

12

Feature Requests / Re: depthmap hole filling

« on: October 08, 2024, 03:56:11 PM »

ok guess not!

13

General / Re: Tweaks/Tricks for solving low parralax footage reconstruction

« on: October 08, 2024, 03:55:03 PM »

man can someone just weigh in on this please? IDK if if been flagged or something for suggesting pretty relevant updates to the software on this forum or on facebook but even the most basic questions i'm asking just gather dust

14

General / Tweaks/Tricks for solving low parralax footage reconstruction

« on: October 08, 2024, 12:47:33 AM »

I’m working with video footage to reconstruct part of a ruined section of an old abbey in England using Metashape. The first half of the clip has decent parallax, which allows Metashape to solve most of the reconstruction. However, the latter part transitions to nodal camera movement, with little to no parallax, making it difficult to reconstruct the topology accurately. I’ve managed to improve the results slightly by filtering the point cloud and culling images with poor matches or low projection accuracy. I’m looking for suggestions or Tweaks that could enhance the reconstruction, particularly for sections with low parallax.

15

Feature Requests / depthmap hole filling

« on: October 04, 2024, 10:01:28 PM »

https://github.com/atapour/depthComp

Something along these lines could be quite useful for for depthmap hole filling…

For example, slightly reflective or featureless shapes could potentially be accounted for during depthmap generation

There’s also DepthCrafter by TenCent which just came out, which whilst not physically accurate, had good temporal stability (sequential photo adherance)

Will we ever see some kind of enhancement like this implemented into Metashape?