Publications

Intrinsic Images in the Wild

Sean Bell, Kavita Bala, Noah Snavely
Cornell University

ACM Transactions on Graphics (SIGGRAPH 2014)

Paper (44MB PDF)  Supplemental (27MB PDF)  Slides (309M Keynote) 

Abstract:

Intrinsic image decomposition separates an image into a reflectance layer and a shading layer. Automatic intrinsic image decomposition remains a significant challenge, particularly for real-world scenes. Advances on this longstanding problem have been spurred by public datasets of ground truth data, such as the MIT Intrinsic Images dataset. However, the difficulty of acquiring ground truth data has meant that such datasets cover a small range of materials and objects. In contrast, real-world scenes contain a rich range of shapes and materials, lit by complex illumination.

In this paper we introduce Intrinsic Images in the Wild, a large-scale, public dataset for evaluating intrinsic image decompositions of indoor scenes. We create this benchmark through millions of crowdsourced annotations of relative comparisons of material properties at pairs of points in each scene. Crowdsourcing enables a scalable approach to acquiring a large database, and uses the ability of humans to judge material comparisons, despite variations in illumination. Given our database, we develop a dense CRF-based intrinsic image algorithm for images in the wild that outperforms a range of state-of-the-art intrinsic image algorithms. Intrinsic image decomposition remains a challenging problem; we release our code and database publicly to support future research on this problem, available online at http://intrinsic.cs.cornell.edu/.

Video:

BibTeX:

@article{bell14intrinsic,
	author = "Sean Bell and Kavita Bala and Noah Snavely",
	title = "Intrinsic Images in the Wild",
	journal = "ACM Trans. on Graphics (SIGGRAPH)",
	volume = "33",
	number = "4",
	year = "2014",
}

Code and Data

Dataset: We include all collected data as well as a Python implementation of our WHDR metric.

Full dataset (release 0, 1.5G)  Judgement data only (release 0, 97M)


Crowdsourcing pipeline: We extended the OpenSurfaces pipeline to collect reflectance judgements.

Code (Github repository)  Documentation


Decomposition code: We release both our code, as well as pre-computed decompositions for all images and all algorithms in our dataset. Note that the decompositions are distributed as a script that downloads the actual PNG images.

Code (Github repository)  Pre-computed decompositions (release 0, 4.5M)


MTurk Tasks

We include previews of our instructions, tutorials, and tasks that were shown to online workers.

Flag transparent/mirror points

Preview:  Intructions  Tutorial  Task

Compare surface reflectance

Preview:  Intructions  Tutorial  Task


License

The annotations are licensed under a Creative Commons Attribution 4.0 International License. The photos have their own licenses.


Acknowledgements

We would like to thank Kevin Matzen for his invaluable help in putting together our submission. This work was supported in part by a NSERC PGS-D scholarship, the National Science Foundation (grants IIS-1149393, IIS-1011919, IIS-1161645), and by the Intel Science and Technology Center for Visual Computing. In the supplemental material, we acknowledge the Flickr users who released their images under Creative Commons licenses.

Header background pattern: courtesy of Subtle Patterns.