scaling filters

[redguy_]

Since I’ve never worked with a FPGA or written RTL before, I decided to do a small project with the OSSC. Some basic pixel scaling filters seemed like a good idea. Unfortunately, the FPGA is nearly out of memory so I temporarily removed the menu in order to fit 4x 2K RAMs in place of the existing 1x 4K RAM. It’s a bit of a hack right now, but I managed to get something working today. The changes add eagle2x/3x (eagle 3x is something I made up that has some bugs), lq2x/3x, and scale2x/3x.

Here’s a short video demonstrating it. Make sure to watch full screen and in high resolution as eagle and lq are both subtle. If you turn up the volume you will hear the remote click as I cycle through the 4 different modes (no filtering, eagle, lq, scale) @ line triple, 256×240-optimized. The complete cycle is repeated 3 times. Changes are visible at edges and are most notable at the ghost house. Sorry for the phone camera video of my projector. I need to invest in a better capture card that can handle the higher bandwidth at 240P from the nes/snes.

My preference used to be to disable as many filters as I could when converting nes/snes to HD, but I really like the subtle changes that lq3x makes.

[gruntbuggly]

“This video is private.”

Edit: It works now. The filters are interesting and seem to work as well as can be expected as a way of making the image less pixely for larger screens.

[BuckoA51]

Nice proof of concept! While the end result isn’t to my taste, I still commend your work, this is exactly the sort of innovation we hoped to see from the community.

[marqs]

That’s nice. FPGA memory is definitely thin at the momemt, but there are ways around it. If more memory is needed for line buffers, one option is to locate soft-CPU instruction memory to flash (as specified in Altera AN736). Execution is then a bit slower (not necessarily noticeable) and firmware update implementation gets more complicated, but that might be something I have to look into sooner or later.

[redguy_]

I reduced the size of the four RAMs from 2k each to 1k and added a temporary check to see if a 256×240 source is going to run off the end of the current RAM to stop the write pointer from wrapping. I think this is functionally ok for lower width sources. The image doesn’t show any problems. Now I need to make some simple changes to the read/write indexing logic to support an effective 2K row for higher resolution sources (detected using HSYNC?) to make it function like the original 4K buffer. I believe the RAMs can now be single ported which supports 3x read for the filter logic plus 1x write for the incoming row. The menu is able to fit back in.

[paulb_nl]

Nice work. So you have it working with the menu now? How does it look in linedouble mode?

[redguy_]

It works with the menu now, but I still need to fix the higher resolution modes to wrap into the next buffer.

It has a similar looking affect in line2x mode. The only problem I noticed is line2x mode is more sensitive to the timing violations I introduced. I’m going to make another attempt at trying to work around the phase offset problem and then rewrite this code to fix timing and remove some hacks.

[redguy_]

I rewrote the sampling phase alignment and filtering code this past weekend along with merging it with the latest from the release tree. This included changes to support a 4x 1K line buffer with filtering or an effective 2x 2K line buffer without filtering. linex3 M2 and M3 modes work great with and without filtering. Reset or reacquiring sync doesn’t result in a random 180 degree phase shift. But I’ve introduced some bugs:

1) linex2 and linex3 M0/M1 modes randomly drop frames (flicker) on a xcapture-1. my projector doesn’t exhibit the problem
2) linex2 scaling filters have flickering pixels.
3) 2x 2K linebuffer mode draws a column of seemingly random pixels (usually all black) when it crosses into the next physical 1k buffer.

For (1), the flickering doesn’t coincide with a tvp sync loss. I’m pretty sure the problem was introduced by the phase alignment changes.

I’m not sure what to do about (2). I allow a programmable amount of difference between pixels for comparison in the filtering logic, but changing this doesn’t result in a reduction in the amount or degree pixels flicker.

(3) seems like an off-by-one error. What’s interesting is sometimes the column of pixels isn’t black and it extends through the masked regions.

I may also need to further pipeline the filtering logic to fix timing problems.

[BuckoA51]

Very impressive work, hope that you’re able to squash those last few bugs. How severe is the flickering in 2?

[redguy_]

The flicker in (2) only affects a small number of pixels, but it happens at edges where the filters take effect. The pixels look like they are blinking between neighboring values at a high rate. I suspect it has to do with how the image is sampled by the TVP from frame to frame and the scaling filter amplifies the differences. Changing the sampling phase and LPF helps a little but really just moves where the problem pixels are at. It’s not a problem when things are moving. Pause screens and menus with non-black backgrounds are where it’s most noticeable. The lq filter uses averages rather than direct copy of pixels and it’s less noticeable with it than with scale. Maybe a (user-controlled) weighted interpolation to control filter strength is a good way to work around the problem.

I’ll post a video when I get home tonight.

[redguy_]

I made a workaround for the flickering in (1) and fixed the missing column of pixels in (3). I’m still not sure how to fix the flickering pixels in linex2 mode with filters enabled (2), but I’ll keep looking at it.

Anyone interesting in trying a new firmware out? It hopefully fixes the sampling phase 180 degree shift problem after reset/desync and also adds some preliminary scaling filters for linex2 and linex3 optimized modes. I don’t want to post a public link because it includes 0.75 changes from marqs that he may not be ready to widely distribute.

[paulb_nl]

Sure I would like to try it out. Firmware 0.75 has been released so posting a link shouldn’t be a problem. Thanks!

[redguy_]

https://drive.google.com/file/d/0Bx9gzQZCkH8qVkszel9oblYtZ3M/view?usp=sharing

FW 0.75-rg0
– Added preliminary scaling filters for linex2 and linex3 M2/M3 optimized video modes.
– Fixed (workaround) for sampling phase 180 degree shift in M2/M3 optimized video modes.

* I normally flash with jtag, but I tested the SD card and it was able to correctly flash to an unmodified firmware from 0.75-aud-rg0 and also flash from an unmodified fw to 0.75-aud-rg0.
* The scaling filters selection is available in the post-processing menu. It defaults to none. Eagle3x is an odd combination of eagle2x and lq3x which doesn’t always look very good. I usually use lq when I’m in the mood for filters.
* Filter delta controls the difference allowed when comparing two pixels. It’s best to leave this at 32 right now.
* Selecting a filter adds an additional 1 scanline of latency.
* Selecting a filter results in the image shifting down a scanline due to delayed data read relative to sync/data_enable. That’s a bug I have yet to fix.
* linex2 mode still has the flickering pixel problem when filters are enabled

I tested all basic functionality that I could think of (only on snes and nes), but please understand that this is an alpha, custom firmware and you should only try it if you understand the implications.

[paulb_nl]

Could you share a build for an OSSC without audio? I am not sure what will happen if I flash the audio firmware on OSSC without audio addon.

[redguy_]

I updated the link to include FW without the audio features compiled in.

FYI, I also don’t have the audio board add-on and, as far as I can tell, it functions correctly with the audio compiled into the FW.

[Author]

Posts