There are a wide number of variables that could cause differences in quality.
1) The location of each captured image is probably not identical between PS and DSLR, therefore resulting in different initial image data for Photoscan to process. This is most likely the main difference.
2) The quantity of images captured. 139 vs 133, also goes back to one set not having the same pixel information as the other.
3) The sensor types and how they capture pixel color information can play a factor.
4) Camera settings, like ISO, aperture, and shutter speed. Can all play a factor in how data is captured.
5) Each camera can reduce and record image data differently to a memory card.
There are many more factors, but what it really boils down to is that you:
- Capture images with a wide depth of field to keep as much of your subject in sharp focus as possible.
- Stereo capture is helpful for the software, so with a single camera you wanna keep each image you shoot within a certain distance of each other. Essentially to where you can take any two photos that are next to each other and easily make a stereo image from it. This require consistent camera location around your subject and will vary depending on the size of your subject.
- Keeping the lowest ISO possible will help reduced grain in the image. The less the grain the better the pixel information for the software.
- Tracking is a HUGE part of it too. If your subject has a smooth, non-detailed surface, then the software has nothing to match in each image no matter how high the resolution. Having some kind of unique texture, be it a printed pattern or in your case a rough material will help in solving the cameras. Ideally having specific markers in the scene work best.
- Coverage, coverage, coverage. This kinda goes back to the stereo pair note above, but making sure as many hidden areas of your subject are captured will help with having a full complete solved surface.
- Avoid macro lenses if possible. They completely destroy any way of getting wide depth of field. It usually chokes it to razor thin space and forces too high of an aperture. High apertures usually above f8-f11 will actually cause diffraction and start softening the image. Sharpness is what you need for the details.
For your two sample images, I can see what looks to be a heavier noise pattern on the DSLR camera. This may also just be the forum compressing the image. What where the camera settings and resolutions used for both capture sessions?