scaling filters

[redguy_]

I had some time to look at the scaling filters today. I added the filter and 4x linebuffer changes to the latest release code. The hqx output doesn’t look right so I ported the code (with as minimal changes as possible) to a snes emulator and got it working there. Unfortunately, that didn’t fix much as there was only one minor bug in the implementation of the hq3x algorithm.

The real problem is the comparison logic working on values from the TVP which are not stable like direct digital input in an emulator. With scale, it’s easier to work around than hqx since it does a binary comparison on a smaller set of neighbor pixels to decide to copy a neighbor (lqx does an average) or keep the current pixel value. The comparison can include some margin. The flashing pixels are due to comparison function failing on a set of frames and then passing on the next set of frames. With a small margin the delta between the pixels is small and the change between a comparison toggling isn’t noticeable, but nothing compares successfully and scale doesn’t do anything useful. With a larger margin the delta between pixels is bigger and flashing is noticeable, but then scale starts doing useful smoothing. A hack to fix this would be to perform interpolation at the edge of the comparison function to have the change be gradual and also support a margin.

Hqx also performs a binary comparison, but it does it against all neighboring pixels which feeds into a lookup table to say how to blend several of the neighbor pixels and current pixel together. The lookup table then provides an opcode for the type of blending to perform. So random changes in the comparison make the image crawl a bit and generally result in bad output. I can’t think of an easy hack to fix this.

Ideally, there would be a way to remove or ignore the variations in pixel values frame to frame when it isn’t actually changing. Something simple like dropping a number of the low order bits of the RGB values doesn’t seem to work well. I was hoping the latest release code with the oversampling of the input would improve things, but nothing changed. My current plan is to collect some statistics from static images and find out which pixels tend to change value and what the set of values are. This will hopefully lead to a better set of comparison functions. A frame buffer would help with this, but testing a few rows at a time is still useful.

Any ideas or suggestions are welcome.

[paulb_nl]

I understand pixels change between frames due to noise or sync jitter but since the hqx filters work on the linebuffer that cannot be affecting it. The pixels are stable in the linebuffer.

The filters should be working fine even though pixels change every frame. For example a game with scrolling background.

It is a circular linebuffer right? Are you taking into account that when a new pixel comes in all the pixels in the linebuffer shift position? So if you are viewing it as 3 lines then the leftmost pixel on the second line is suddenly the rightmost on the first line.

Does the filter only process the active 256 pixels or all 341?

[redguy_]

The problem is pixel change between frames even with a static image. Within a frame (inside the linebuffer) they are stable unless I introduced a bug with the read/write addressing and enables I haven’t found yet.

For example, I added a filter that outputs a red pixel if the compare fails and a green pixel if the compare succeeds. With the TVP input there are pixels that toggle between green and red on a static image. With a generated input done within the RTL (input is written into the linebuffer the same way as the TVP input) the image is stable and no pixels change from green to red.

[redguy_]

https://drive.google.com/file/d/0Bx9gzQZCkH8qakVPXy1Gc1YyT1k/view?usp=sharing

FW 0.76-rg0b

– Merged with latest release
– Added hq2x/hq3x filters
– Fixed some bugs in scale3x filter
– Changed linebuffer to drop samples on write to support scaling filters

I like the way the hqx filter turned out. It’s heavily based on the fpganes implementation with updates to support hq3x. There’s also a temporary debug “diff” filter mode to help see what the hqx diff function output looks like. Pixels still flicker with the scaling filters, but I have some ideas on how to fix that now that I know what the problem is.

With the diff mode it’s easy to see why the pixels flicker. Even with static images there is some temporal change in pixels across frames. Since the diff function has a sharp cutoff any comparison that is near that boundary will cause it to bounce between different blending functions and any large changes in the output pixel will result in the flicker. I think I know how to fix it for the scale filter, but hqx will require some more thought.

[BuckoA51]

Excellent work Redguy thanks for helping make OSSC even more awesome 🙂

[NeroAngelo]

hi i was wondering if anyone has hq2x/hq3x filters for 480i content i tried FW 0.76-rg0b it was it had lots of pixels like 240p ps1 i like the effect of hq2x pity about the res.

[redguy_]

Currently, the filters are only enabled in line2x or line3x mode and if it’s one of the optimized 240P resolutions. That was to simplify the check for a H_TOTAL – H_SYNCLEN that’s under 1024 pixels so the linebuffer isn’t overrun. 480p and some of the other resolutions should work by just enabling the filter for them. Some of the higher resolutions may work with some changes to drop useless samples. I’ll give a try this weekend and see what happens.

Not sure how good 480i will look with filters given that it’s comparing every other row.

I haven’t solved the flickering pixel problem. It may be possible with a frame buffer, but there’s no room for one.

[LazyEpic]

The hq filter works like a charm with Metal Gear solid on PSX, almost completely negates the dithering without any noticeable negative effects. This is wonderful, thank you so much 🙂

EDIT:
Now still images doesn’t really do it justice compared to when in motion but don’t have a hdmi capture device and recording with the phone isn’t ideal but still, scan lines at auto, without them it looks clean but not “old school” 😛

Snake – no filter

Snake – hq filter

[redguy_]

Glad it’s working out for you. I don’t have a psx to test on, but you did motivate me to dig out my xbox and ps2 to see what they look like.

https://drive.google.com/file/d/0Bx9gzQZCkH8qekhrdldtbFRaNU0/view?usp=sharing

FW 0.76-rg0c

– Fixed non-optimized modes “black screen” bug.
– Added filter support for 480p line2x mode.

I had some time to look at the firmware again and realized the previous version broke a lot of the non-optimized modes.

I also looked at 480i and when line2x (bob deinterlacing) is enabled it can use the filters. Although, I’m not sure how desirable the effect is due to how a field gets scaled to produce 480p.

480p had me scratching my head for a while as going through the filter logic caused it drop pixels. It looks like the it6613 chip is used to perform the scaling on certain resolutions. I disabled that and forced it to behave like the other optimized modes. The pixel scaling filter effects start to vanish on inputs with much higher color selection and higher resolutions. 480p linex2 on ps3 with filters really only added flickering pixels and not much else.

[funny_mikey]

Will these filters be updated for 0.77? It is only with 0.77 that I have been able to get line triple mode to work reliably. Also would it be possible for them to work for l4x and l5x? I enjoyed the filters on line double mode and would love to use them on the higher modes if possible. Thank you for all the work you’ve put into the filters so far. They make a noticeable difference in image quality, especially in regards to smoothing the image to make it a bit less jagged.

[redguy_]

Yes on merging with 0.77. I already merged with an old version of 0.77 a while back but never posted it. I just tried to merge with the latest 0.77 changes and diff gave up and just told me the whole scanconverter.v file is different without identifying specific changes. It will be fixed this weekend and a new file will be posted.

l4x and l5x filtering is a lot of work that I don’t have time to look into right now. There’s not much space left on the FPGA without making some of the changes marqs mentioned a while back. I’m not sure whether 4x or 5x filtering is even possible without more room. scale4x is probably scale2x applied twice. Not sure if scale5x is even defined or if I would need to make it up. I took some shortcuts in hq3x to use a lot of the hq2x code (which was already based on code with shortcuts).

[BuckoA51]

It’s time I tried these and wrote an article, let me know when the 0.77fw is available (hopefully with an audio version too?) 🙂

[redguy_]

Here’s the last version ported to the latest 0.77 source code. It includes audio. I use it without issues on my audio-less ossc. It’s still experimental and has several bugs.

https://github.com/RedGuyyyy/ossc/blob/master/fw/ossc_0.77-rg0d.bin

Not much has changed in the filters from the previous FW. For those who haven’t tried it before:
– [Post-proc] -> [Filter/Filter str] control the filters. You can choose between ‘scale’ or ‘hq’. Filter strength only affects scale. There’s also a test filter called ‘diff’ that shows which edges scale applies changes to.
– It works best at 240P x2 or x3 in the optimized format for your input. E.g. 8:7/4:3.
– 480i 2x (bob de-interlacing) and 480p 2x are also supported, but they don’t add much other than the flashing pixels (see below).
– Due to the nature of going through DAC (snes) -> ADC (ossc) conversion, applying a filter that was intended for digital input (emulators) results in artifacts that are most noticeable on static images. These artifacts are flashing pixels that jump between the original pixel and the filtered pixel. I’m not sure of a solution without having access to a frame buffer. Reducing the filter strength with scale will lessen the flashing.
– The filters add one scanline of latency over the base ossc FW.

[BuckoA51]

Nice! I’ll check that out as soon as I get a moment and do a little write up.

[mattw77777]

I think these filters offer a nice subtle blur to the jaggy pixels from the retro age. I can’t wait to see the super eagle or de-dither filters if they become reality. Games like donkey king country really need more blur to smooth out the graphics. I really like the retroarch and snes classic’s bilinear filter for smoothing out the pixels. The horizontal linear filter on the snes classic actually looks better than the crt modes linear filter because it only blurs the screen about half as much. It looks a lot like svideo on a crt with the pixels being sharp but still blurring together just enough to smooth out the picture.

My question for you guys is this: How hard would it be to implement a simple bilinear filter on the OSSC that would be like the one on the snes classic or retroarch? Thanx a lot guys and good job getting some filters to work with the OSSC! I can’t wait to see where this heads!

[Author]

Posts