Snap Angle Prediction for 360° Panoramas

Bo Xiong, Kristen Grauman; The European Conference on Computer Vision (ECCV), 2018, pp. 3-18

360° panoramas are a rich medium, yet notoriously difficult to visualize in the 2D image plane. We explore how intelligent rotations of a spherical image may enable content-aware projection with fewer perceptible distortions. Whereas existing approaches assume the viewpoint is fixed, intuitively some viewing angles within the sphere preserve high-level objects better than others. To discover the relationship between these optimal emph{snap angles} and the spherical panorama's content, we develop a reinforcement learning approach for the cubemap projection model. Implemented as a deep recurrent neural network, our method selects a sequence of rotation actions and receives reward for avoiding cube boundaries that overlap with important foreground objects. Our results demonstrate the impact both qualitatively and quantitatively.