General Object Pose Transformation Network from Unpaired Data

Yukun Su, Guosheng Lin, Ruizhou Sun, Qingyao Wu ;

"Object pose transformation is a challenging task. Yet, most existing pose transformation networks only focus on synthesizing humans. These methods either rely on the keypoints information or rely on the manual annotations of the paired target pose images for training. However, collecting such paired data is laboring and the cue of keypoints is inapplicable to general objects. In this paper, we address a problem of novel general object pose transformation from unpaired data. Given a source image of an object that provides appearance information and the desired pose image as a reference in the absence of paired examples, we produce a depiction of that object in that pose, retaining the appearance of both the object and background. Specifically, to preserve the source information, we propose an adversarial network with $\textbf{S}$patial-$\textbf{S}$tructural (SS) block and $\textbf{T}$exture-$\textbf{S}$tyle-$\textbf{C}$olor (TSC) block after the correlation matching module that facilitates the output to be semantically corresponding to the target pose image while contextually related to the source image. In addition, we can extend our network to complete multi-object and cross-category pose transformation. Extensive experiments demonstrate the effectiveness of our method which can create more realistic images when compared to those of recent approaches in terms of image quality. Moreover, we show the practicality of our method for several applications."