понеделник, 28 март 2016 г.

RFC: Are Instant-refresh Displays Possible?

In brief, I'd love it if an electrical engineer who knows how computer displays (and phone screens, TV sets etc.) work on the hardware level and what their technological and physical limitations are would tell me why it takes so long (a whole refresh cycle of 7 to 16 ms) for an update of the frame buffer to be applied to all pixels, instead of that taking say a few microseconds - even on a TFT LCD screen where each pixel is addressed and controlled independently.

The longer version starts almost 11 years ago when I switched my 17" CRT monitor with a 17" LCD one. One of the first changes I noticed was the appearance of some strange horizontal lines in my games. I looked up the issue and found out it was called tearing and was fixed by enabling v-sync. I did that, it did fix the issue, and I forgot all about it - until some more years later, when I started noticing some weird stuttering in some new games. Every now and then, the picture would seemingly freeze for about a frame and then go on as usual. I looked that up and it turned out it was caused by the combination of v-sync and my computer not being able to produce frames as fast as the display could show them. It was a mathematical necessity that a frame would have to be repeated from time to time, which caused the perceived stuttering of the games.

Finding a solution to stuttering, however, proved tricky. At first I discovered triple buffering, but it turned out to have two major problems: most games didn't support it because they were using DirectX, which didn't support it; and it increased power consumption significantly* for less intensive games that were able to produce frames way faster than the display could show them. Then John Carmack requested - and received - a new driver feature from GPU vendors for the idTech 5 engine used for Rage: being able to dynamically turn v-sync on and off depending on load. Nvidia then decided to release this feature to the general population as a control-panel option and market it as adaptive v-sync. Impatient to finally have tearing and stuttering solved at the same time, I got a GeForce after having used AMD for many years - and was immediately disappointed when I turned on adaptive v-sync and saw tear lines. It turned out tearing doesn't just happen when frames are being produced too fast, but any time when rendering and displaying aren't in sync. Still, I left the question at that and accepted stuttering for a few more years.

Two years and change ago, Nvidia announced one more technology, called G-sync, which supposedly solved both tearing and stuttering once and for all. In short, it makes displays wait for GPUs, not the other way around. Shortly thereafter, AMD followed with a simpler and cheaper solution of their own, based on the exact same principle. Come this year, I started actively playing the third Witcher game and was somewhat disappointed I couldn't run it at Ultra; not wanting to buy a new GPU before the new series were out, I somehow thought removing v-sync would improve my frame rate, so I bought a g-sync monitor. Once again, I was disappointed - not only didn't the frame rate improve**, but I noticed a significant degradation of image quality and bad flickering that started hurting my eyes. After some experimentation, I discovered that just switching back from g-sync to a fixed refresh rate fixed the issues. However, I decided to give adaptive v-sync another try. And, voila, it worked! I didn't see any tearing. I thought I was once again unable to see it because I'd finally brought my display's refresh rate back to the hundreds, years after replacing the CRT. But this time, I started thinking.

Why would tear lines appear at all in the first place? I know just copying the back buffer to the frame buffer is nearly instantaneous - at FullHD and 24-bit pixels, that's about 6MB, when my GPU's slowest memory has a bandwidth of 160 GB/s. Even with some latency added, this copy should take less than 40 microseconds. In addition to that, I'd always thought the display would simultaneously tell all of its pixels their new values, as soon as it read them from the frame buffer at the beginning of a refresh. I thought this operation would also take microseconds. After that, no synchronization would be necessary - the capacitors of the pixels themselves would act as a buffer holding the color information, while the frame buffer would be free to be overridden. And on a regular 60 Hz LCD, this process would repeat once every 16.67 milliseconds. So, if the whole update took less than 100 microseconds, statistically a frame would be produced and sent to the frame buffer at exactly the same time as the display was reading from that frame buffer less than once every 166 refreshes, so I figured no one would be able to notice one short-lived horizontal tear line once every three seconds, very short-lasted each time, and at a different place each time. Therefore, my assumption about the duration of a screen refresh had to be wrong. In fact, it had to be very wrong - a refresh would have to take an amount of time comparable to the production of the frame itself for either tearing or stuttering*** to be such prevalent, noticeable problems.

I started using my limited Google-fu and spent a whole day gathering the few clues freely available among the flame wars and ignorance of the open net. From what I've been able to find (though no official statements or technical documents), it seems I was right, and in fact all modern screens still refresh like CRTs: there's a virtual scan line that traverses the screen, rightwards and downwards, and takes almost the whole time between two refreshes to consecutively visit all the pixels - one at a time, - update them with the new target colors and tell them to start changing.**** However, for the consecutive update of 2 million pixels to be able to happen in 7 ms on my new display, each individual pixel update must take less than 3.5 nanoseconds! If only we could somehow batch these updates and update a whole horizontal line at a time, the whole refresh would be sped up one thousandfold... which would bring us to the way I'd thought updates happened and make tearing, stuttering, v-sync, g-sync, freesync etc. completely obsolete.

Therefore, my question to electrical engineers - why aren't pixel updates batched?


* V-sync, or synchronization of game render calls with the monitor's refresh, has the side effect of limiting the game's frame rate: a new frame doesn't start being rendered by the engine until the previous one starts being drawn by the display. Without that, the computer might work a lot more, increasing power consumption. External frame-rate limiting tools are cumbersome, and most games don't expose such options - and none of the older ones that I'd been playing at the time did.

** I'd been playing The Witcher 3 in borderless-windowed mode, recommended by their support as a way to fix the frequent crashes. It did fix them - and as a side effect, it activated triple buffering, because that's always on for all Windows windows when Aero is on. So my frame rate had already been as good as it could have been.

*** If the GPU has to synchronize with the display - leading to a repeated/dropped frame, but not to a slowdown of the game - only once every few seconds, I doubt it would be very noticeable.

**** I also wondered a lot why the scan line would then not be visible on LCDs like it was on CRTs at 60 Hz. I couldn't find anything on this one, but I have a hypothesis - it works like soft shadows because of the pixel response times. I.e., there's always a band of pixels that takes up a large portion of the screen - even at the extreme case of a screen that takes a whole 16ms to refresh but the pixels themselves only need 1, that band would be 1/16th of the screen - where one frame is very gradually transforming into the next one; and that band is moving down the screen, very quickly, all the time, so this is imperceptible to the human eyes - or maybe brain - even when recorded on video. If my hypothesis is true, this band should be visible on exactly such a screen captured on a very high-speed camera when a sequence of completely opposite colors (white and black) is being flashed.