Sharpening or enhancing low-res images has always been a popular Hollywood trademark in movies. You have the blurred out image of a culprit taken during the twilight hours but the police/spy agency technical expert says a bunch of technical jargon and voila, you know have a high res image letting you nab the criminal. Alas, such tricks were just tricks. Getting more data from a RAW image is simply impossible, but Google may have the next best thing: RAISR.
RAISR, or Rapid and Accurate Image Super-Resolution, is the latest software from Google that incorporates machine learning into its design, letting a lot of that movie magic we saw growing up actually become reality. RAISR works similarly to current methods of upsampling, the process of turning a low-res image into a high res image by inserting pixels into it. However, RAISR differs from current methods by adapting its methods to the type of image it’s looking at. The results are comparable to, or better than, the currently available super-resolution methods, and it does this roughly 10 to 100 times faster, allowing it to be run on a typical mobile device in real-time. Furthermore, the technique is able to avoid recreating the aliasing artefacts that may exist in the lower resolution image. Existing upsampling methods also produce blurry images as can be seen below.
RAISR can be trained in two ways. The first is the “direct” method, where filters are learned directly from low and high-resolution image pairs. The other method involves first applying a computationally cheap upsampler to the low-resolution image and then learning the filters from the upsampled and high-resolution image pairs. While the direct method is computationally faster, the second method allows for non-integer scale factors and better leveraging of hardware-based upsampling. You can see a diagram of the second sampling image below, as well as some RAISR enhanced pictures:
For either method, RAISR filters are trained according to “edge features” found in small patches of images, – brightness/color gradients, flat/textured regions, etc. – characterized by direction (the angle of an edge), strength (sharp edges have a greater strength) and coherence (a measure of how directional the edge is).
