A common trope we’ve seen in numerous films and TV shows is the ability to zoom in on heavily pixelated footage or pictures, state the magic word “enhance” and see said image or footage suddenly become much clearer. While it’s still a far off concept, a few Google researchers working for the Google Brain team have developed a system that is close to hitting the mark.
The system uses two different neural networks to “enhance” low-resolution images by “filling in the details.” The first network is a “conditioning” element that maps the lower-resolution 8×8 images to existing images of similar examples that are in higher resolution. As a result, the first network will compare the two images to figure out how the low-res image should appear at a higher resolution.
Meanwhile, the other neural network is called the “prior” network which will add realistic details to the final output, after it has gone through the conditioning network. It does this by learning what each pixel in a low-res sample would generally correspond to in a high-res image.
Of course, it is nowhere near a complete and picture perfect state; after real-world tests, human observers were fooled 10 percent of the time, when shown a real high-res image versus the neural network version of a celebrity’s face. However, that’s still impressive considering that 50% is a perfect score. Meanwhile, images of a bedroom scored 28 percent.
The new developments aren’t the only way Google have invented to super-enhance an image. A few months ago, they revealed their RAISR program. The full research by the Google Brain team can be found here.
