Instant NERF?

[aaronfhd]

Wondering if Agisoft will have instant nerf in the future.

[johnnokomis]

This is kind of like asking the National Football League if they'll start hosting baseball games in the future. NeRF's are completely different from a point cloud/mesh.. That's an understatement, they share basically no similarities to each other. From processing to end results. NeRF's are really only good for exporting a video from, not for 3D modeling or calculating volumetrics.

[kb2zuz]

Photogrammetry and NeRFs are two different things but there is some overlap, particularly in the capture method. Also NeRF does require having camera locations, and they're using COLMAP SfM as a separate script to determine the camera locations. If we've done photogrammetry on a scene or object and have done the work to optimize the camera locations, it would be nice to just be able to export it in a format that could be fed into creating a NeRF. There are some use cases where NeRFs might be desirable. Maybe the material properties don't work well for photogrammetry but want to see if rendering a video from a NeRF might give a useable representation.

While I don't expect Agisoft to build NeRFs, a simple to implement feature that would be useful would be if we could export the cameras in the JSON format that NVidia's InstantNeRF is using so that we could take the images and try them in NeRF.

[tazzo]

Have a look at github, for example https://github.com/EnricoAhlers/agi2nerf

[mrv2020]

Several factors can affect the accuracy of the NeRF model, including:

Training Data: The quality and quantity of the training data can significantly affect the accuracy of the NeRF model. A larger and more diverse dataset can result in a more accurate model.

Neural Network Architecture: The complexity of the neural network used in NeRF can affect the accuracy of the model. A more complex network can capture more details but may also require more training data and take longer to train.

Camera Positions: The accuracy of the NeRF model depends on the accuracy of the camera positions used to capture the training data.

Lighting: The accuracy of the NeRF model can be affected by the lighting conditions during the training data capture. Changing lighting conditions can result in inaccurate predictions.


Comparison of Accuracy

Several studies have compared the accuracy of traditional photogrammetry and NeRF methods for 3D modeling. One study compared the accuracy of the two methods in creating 3D models of small objects and found that photogrammetry resulted in higher accuracy than NeRF for small objects with sharp edges and fine details.

Another study compared the accuracy of the two methods for 3D modeling of large objects and found that NeRF resulted in higher accuracy than photogrammetry for large objects with complex shapes and textures.

However, it is worth noting that the accuracy of both methods depends on various factors, including the quality of the data and the complexity of the object being modeled. Additionally, both methods have their own advantages and disadvantages, and the choice of method depends on the specific use case and requirements of the project.



In general, traditional photogrammetry is more accurate for small objects with sharp edges and fine details, while NeRF is more accurate for large objects with complex shapes and textures. However, both methods can produce accurate 3D models when used correctly and with the appropriate data and equipment.

[andyroo]

This (Photogrammetry+NeRF fusion) is something I've been thinking about for a while - with my geologist's inadequate understanding of the mathematical specifics of both of these machine-vision algorithm families.

Specifically how NeRFs could inform MVS, or more generally the  photogrammetric dense reconstruction algorithm(s). From what I understand NeRFs, from Plenoxels to 3D Gaussian Splatting, still use the/a SfM-derived sparse cloud and generate dense clouds using trained models with much more poorly constrained geometry than MVS-type algorithms... BUT - it seems like coregistration would be relatively trivial if the sparse points are shared, and if the MVS dense points with highest confidence and lowest error are used to further constrain the NeRF such that the NeRF can contribute valuable information where MVS falls apart - like with specular reflections or low contrast surfaces (roads/water/sand from my aerial photogrammetry perspective).

I feel like the NeRF and photogrammetry families are walking separate paths with a fence in-between and they need to make a love child.