Hello NORIM,
If you are interested in model georeferencing, you should use coordinates of the camera locations or markers (GCPs) in the Reference pane. Manual referencing via Move/Rotate/Scale object instruments would not allow you to get properly referenced model in geographic coordinates.
Getting bad referencing usually means that the coordinate information is incorrect or the referencing points are placed improperly (some cameras are not correctly aligned or markers are not correctly placed).
Maybe you can share the problematic project with the alignment results to support@agisoft.com?