I have a Yamaha AVR (different model) myself and I also get the same rolling problem but only if I have the ‘VIDEO CONV’ setting turned ON. With my AVR, when VIDEO CONV is on it actively converts all input formats (Composite, s-video, component) and spits them out of the component and HDMI outputs.
Again, with my AVR, when VIDEO CONV is OFF, the video inputs act purely as switchers and the video enhancement mod comes through clean and stable. All the video formats are separated (IE, no cvbs>s-vid>ypbpr>hdmi conversion) and the AVR spits out the signal with no processing. I don’t normally make use of my AVR’s video conversion function (to minimise video lag) so I never knew that there was a incompatibility until I decided to see how good a job it would do converting the mod’s component to hdmi.
Turning the conversion off does mean a separate component connection from your AVR to your display is required.