With regard to Key Points, the manual says
Key point limit
The number indicates upper limit of feature points on every image to be taken into account during
current processing stage. Using zero value allows Metashape to find as many key points as possible,
but it may result in a big number of less reliable points.
So, you are adding points, but they may be less accurate points. The higher you go, the more "less accurate" points you will get and therefore photo alignment may go down.
Maybe try to add a tie point limit which, if I understand the manual, will act as a quality filter on the pile of key points and only select the points of highest quality leading to a better alignment. I think at a certain point, alignment will be as good as it can get and adding more "data" can only make it worse and not better. That is where the defaults come from.