You’re not out of luck quite yet, we’ve just got to put back some of the letterbox.
Start by using the advanced settings to restore the originally intended image — letterbox and everything. Make sure you’re doing this with the PSP sampling mode enabled and with line2x turned on (although if it doesn’t work, try with line2x turned off). Here’s how you do that:
H. Refresh 858
H. Active 720
H. Sync 62
H. Backporch 60
V. Refresh 525
V. Active 480
V. Sync 6
V. Backporch 9
In passthrough mode, this should give you the originally intended letterboxed 480p image. In line2x, it’s 1440×960, in passthrough it’s 720×480.
Now start to remove the letterbox by decreasing H/V Active. If your TV doesn’t drop sync, you’ll see the image growing larger, but also moving away from center of the screen. To re-center it, increase H/V Backporch until centered. Keep decreasing it until your TV loses sync. Whatever value you end up with (as defined by HxV) is the minimum resolution (aka most zoomed in) that your TV can support given the PSP’s true 858×525 input resolution (but see the next paragraph).
It’s also possible (likely, in fact) that your TV supports discrete resolutions around standardized values (e.g. 480p, 720p, etc.), but not the weird ones in between. So there’s one more resolution to try, you can cut the line2x 1440×960 image down to a 720p image. The 16:9 720p resolution is 1280×720, so to get that, get back to the default settings for the PSP I mentioned above, make sure you’re in line2x, and use the following:
H. Active 640
V. Active 360
Adjust H/V. Backporch to re-center