Your display seems to always want to fill to 16:9. That’s why 3x and 5x looks correct but not the other two. You need to find the setting on your display for 4:3 (for 2x mode) or unscaled 1:1 mode (for 4x mode).
Can I prevent 5x from cutting off the sides?
You can increase h.active (and decrease h.backporch to re-center the image).
Forcing 4:3 for example.
You cannot force aspect on the display by any setting on OSSC. OSSC sends what it sends, the display scales to what it wants to scale.
Or maybe I should be trying to fix the stretching and bad positioning on 4x since it’s closest to the monitor’s 1600×900 native res?
That’s what I would go for. Note that you cannot get 4x output to 1600 active horizontal.* If you tweak v.active to 225, the output (active area) will be 1280×900. If your display has no way to display that 1:1, then this mode is not an option.
*(in 4x mode h.samplerate is 1560, which is less than 1600, and if you increase samplerate to above 1600 you will screw up the aspect ratio)