If the PS1 comes out of 240 (for example), then in x5 = 1200. And if your screen is 1080 it is necessarily impossible. 1200> 1080 logic.
For this image corruption it will be necessary to see with the pros of the forums;)
Line5x processing for 240p allows for a few different output modes, namely 1600×1200, 1920×1200, and 1920×1080 (top and bottom cropped), of which the last is the default; so, assuming that all the processing settings are defaults as the OP describes, then line5x should be outputting 1920×1080, which should not cause image corruption on a 1080p display.
It might be useful to know the exact model of the displays in use, as well as which firmware is installed on the OSSC (obviously try the latest if you haven’t already).