High-order Wavelets for Hierarchical Refinement in Inverse Rendering

Abstract

It is common to use factored representation of visibility, lighting and BRDF in inverse rendering. Current techniques use Haar wavelets to calculate these triple product integrals efficiently [Ng et al. 2004]. Haar wavelets are an ideal basis for the piecewise constant visibility function, but suboptimal for the smoother lighting and material functions. How can we leverage compact high-order wavelet bases to improve efficiency, memory consumption and accuracy of an inverse rendering algorithm? If triple product integrals can be efficiently calculated for higher-order wavelets, the reduction in coefficients will reduce the number of calculations, therefore improving performance and memory usage. Some BRDFs can be stored 5x more compactly.
Current inverse rendering algorithms rely on solving large systems of bilinear equations [Haber et al. 2009]. We propose a hierarchical refinement algorithm that exploits the tree structure of the wavelet basis. By only splitting at interesting nodes in the hierarchy, large portions of less important coefficients can be skipped. The key of this algorithm is only splitting nodes of the wavelet tree that contribute to the solution of the system M (see Algorithm 1). It is critical to use high-order wavelets for this, as Haar wavelet can only introduce high frequencies which leads to blockiness.

Images

Iterative refinement of illumination by adding detail coefficients based on a splitting criterium. Reconstructions for both Haar (red) and the smoother Coiflet (green) wavelet bases are shown. Haar has a tendency to introduce disturbing high frequencies around edges.	Reconstruction of a temporal face dataset under different lighting conditions and estimated with the hierarchical refinement method. Ray traced occlusion maps, BRDF slices and lighting environment map are combined in the triple product integral calculation.
Iterative refinement of illumination by adding detail coefficients based on a splitting criterium. Reconstructions for both Haar (red) and the smoother Coiflet (green) wavelet bases are shown. Haar has a tendency to introduce disturbing high frequencies around edges.	Reconstruction of a temporal face dataset under different lighting conditions and estimated with the hierarchical refinement method. Ray traced occlusion maps, BRDF slices and lighting environment map are combined in the triple product integral calculation.
Preprocessed binding coefficients. The Haar binding coefficients can be identified by 3 simple cases [Ng et al. 2004] (left). On the other hand, higher-order wavelets have overlapping support, resulting in a more complex tensor of binding coefficients. By exploiting the hierarchical nature and vanishing moments of wavelets we can calculate efficiently the sparse tensor for high-order wavelets (right).
Preprocessed binding coefficients. The Haar binding coefficients can be identified by 3 simple cases [Ng et al. 2004] (left). On the other hand, higher-order wavelets have overlapping support, resulting in a more complex tensor of binding coefficients. By exploiting the hierarchical nature and vanishing moments of wavelets we can calculate efficiently the sparse tensor for high-order wavelets (right).

Paper and Video

The videos can be watched in VLC media player

surreyFaceUffiziFrame18_slow.avi (1,0 MB) Relighting of one frame with uffizi environment map.	surreyVideoUffiziHighRes.avi (2.6 MB) Relighting the temporal sequence with uffizi environment map.
surreyFacePool6Frame190.avi (1,1 MB) Relighting of one frame with a pool environment map.	surreyFaceVideoPool6.avi (0,9 MB) Relighting the temporal sequence with a pool environment map.

References

HABER, T., FUCHS, C., BEKAERT, P., SEIDEL, H.-P., GOESELE, M., AND LENSCH, H. P. A. 2009. Relighting objects from image collections.

NG, R., RAMAMOORTHI, R., AND HANRAHAN, P. 2004. Triple product wavelet integrals for all-frequency relighting. ACM Trans. Graph. 23, 3 (Aug.), 477–487.

PEERS, P., AND DUTR´E , P. 2005. Inferring reflectance functions from wavelet noise. In Proceedings of the Sixteenth Eurographics conference on Rendering Techniques, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, EGSR’05, 173– 182.

© Nick Michiels 2013