• 

  Paper

• 

  Video

• 

  Code

• 

  Model

We introduce a novel Garment Animation NeRF that generates character animations directly from body motion sequences, eliminating the need for an explicit garment proxy. Upon training, our network produces garment animations with intricate wrinkle details, ensuring plausible body-and-garment occlusions and maintaining structural consistency across views and frames. We demonstrate the network's generalization capabilities across unseen body motions and camera views, while also enabling color editing for garment appearance. Notably, our method is applicable to both synthetic and real-capture garment data.

Given a sequence of character's body motion, we construct a neural radiance field to render animation of the character dressed in the target garment. We first employ a dynamic feature encoder &mathcal;E to infer garment dynamic feature map F_t^s from the information textures V_t and N_t of the body motion B_t. Simultaneously, taking body neural texture images Q_t of the body at front c^fr and back views c^ba, we use a pre-trained image generator &mathcal;G;_img to predict reference images &hat;I;_t(c^fr) and &hat;I;_t(c^ba). Subsequently, we use the detail feature encoder &mathcal;G;_en to generate the detail feature maps F_t^d(c^fr) and F_t^d(c^ba). Then, we obtain body-aware geometric information x_t^b by calculating the distance h between sampling points and the body surface, and finding b_o on the canonical body shape. Finally, we utilize a NeRF network &mathcal;M;_NeRF to render garment appearance feature image F_t^ζ. To enable color editing, we introduce a network &mathcal;M;_D to decompose the garment appearance into a front mask M_t, a color offset map O_t, a radiance map R_t and a blending weight map W_t. By linearly recombining those visual elements, we synthesize the final frame image I_t. Except for the generator &mathcal;G;_img, we jointly train the networks of &mathcal;E;, &mathcal;G;_en, &mathcal;M;_NeRF and &mathcal;M;_D in an end-to-end manner.