I believe I found a sweet spot for PS1 line5x. Generic 4:3, line5x mode set to 1920×1080. It outputs as 1920×1200, but allows the image to be centered and not cropping/overscanning – this is true for both my screen and the camlink (camlink in OBS only, GCHD would throw a “not supported” message).
Line5x 1600×1200 mode works, displays as 1600×1285*
Line5x 1920×1200 mode works, displays as 1920×1285*
*Both of these were not perfectly stable in my testing. May need some minor tweaking to the
backporches and/or synclens to get 100% stable
V. Synclen 4
V. Backporch 1
V. Active 264
H. Samplerate 2046
H. Synclen 150
H. Backporch 234
H. Active 1536
I tested on multiple games, all look good. This SFEX2@ just happens to be my fixation momentarily. :p
Some side by side comparisons of scanlines vs. no scanlines.