The OSSC chooses a “video mode” automatically based on the input it receives and vertical multiplication mode selected, but it doesn’t matter which one it is because you can still freely change the parameters in Advanced Timing, just go into the one that is preselected for the console (it will be there when you select Sampling Opt -> Advanced Timing). Modes that end with L or LB are 288p modes also. (you can check what the different video modes are in the github file: https://github.com/marqs85/ossc/blob/release/software/sys_controller/tvp7002/video_modes.h )
Now for x5 mode (set to 1920×1080 mode (default)) it will always reduce V.Active to 216 “under the hood” to fit the v.active area of 1080. It doesn’t do that for the 1200 modes, but if you increase V.Active in that mode, compatibility will probably drop because the output is less standard. Also note that when using PAL consoles, compatibility for higher line modes is reduced because the refresh rate is not the same as for PC modes (which are normally 60Hz). That’s one reason Steos tweak is great, because 4x PAL signal is much closer to 1080p 50Hz standard than 5x mode.
The increased vertical resolution of PAL is not normally utilized by older consoles anyway, certainly not for the Genesis, it will still just have 224-240 active lines inside the 288. So cutting off vertical area is normally not a problem. Not sure about the PAL PS1 though.
If you want to display more vertical area, then the tweak Steo gave you is good, because it will display 270 lines (4×270=1080).