Learning Continuous Image Representation
with Local Implicit Image Function


Yinbo Chen,  Sifei Liu,  Xiaolong Wang

Paper Code

How to represent an image? While the visual world is presented in a continuous manner, machines store and see the images in a discrete way with 2D arrays of pixels. In this paper, we seek to learn a continuous representation for images. Inspired by the recent progress in 3D reconstruction with implicit function, we propose Local Implicit Image Function (LIIF), which takes an image coordinate and the 2D deep features around the coordinate as inputs, predicts the RGB value at a given coordinate as an output. Since the coordinates are continuous, LIIF can be presented in an arbitrary resolution. To generate the continuous representation for pixel-based images, we train an encoder and LIIF representation via a self-supervised task with super-resolution. The learned continuous representation can be presented in arbitrary resolution even extrapolate to ×30 higher resolution, where the training tasks are not provided. We further show that LIIF representation builds a bridge between discrete and continuous representation in 2D, it naturally supports the learning tasks with size-varied image ground-truths and significantly outperforms the method with resizing the ground-truths.

Zooming in on Continuous Images


We generate LIIF representation for a given input image. On a continuous representation with infinite resolution, we can zoom in on the image while maintaining high fidelity. We compare LIIF with the raw pixels and bilinear interpolation in the following.

Input (360px)

Pixels

Bilinear resize

LIIF (ours)




Changing Resolution


LIIF representation is resolution-free, it can be presented in arbitrary resolution. We present the generated LIIF representation in changing resolution from 32×32 to 640×640. The following results are visualized by nearest up-sampling.

Input (32px)

LIIF in changing resolution (32px-640px)


BibTeX


@article{chen2020learning, title={Learning Continuous Image Representation with Local Implicit Image Function}, author={Chen, Yinbo and Liu, Sifei and Wang, Xiaolong}, journal={arXiv preprint arXiv:2012.09161}, year={2020} }