But that's the point. I would prefer not to switch modes. Instead, I'd like to hold down Alt while left/right-click-dragging in order to quickly change the viewpoint, release Alt and continue with selection, for example.
I agree with this comment. The viewport navigation in Max/Maya is very nice and quick. For those of used to 3D applications, the navigation in Photoscan does feel a bit akward.
Maya viewport navigation works like this:
Alt+LMB = Rotate
Alt+RMB = Zoom in/out
Alt+MMB = Pan Left/Right/Up/Down
Another issue with viewport navigation is that the more you zoom in, the less responsive the camera panning becomes. So when I am zoomed in on a detail, I need to wildly move my mouse dozens of times to move the camera a small distance.