DualReal: Adaptive Joint Training for
Lossless Identity-Motion Fusion

Wenchuan Wang, Mengqi Huang, Yijing Tu, Zhendong Mao^*

University of Science and Technology of China,

^*Corresponding author

Generated customization results of our proposed novel paradigm DualReal. Given subject images and motion videos, DualReal generates high-quality customized identity and motion simultaneously, without compromising the consistency of either dimension.

Abstract

Customized text-to-video generation with pre-trained large-scale models has recently garnered significant attention through focusing on identity and motion consistency. Existing works typically follow the isolated customized paradigm, where the subject identity or motion dynamics are customized exclusively. However, this paradigm completely ignores the intrinsic mutual constraints and synergistic interdependencies between identity and motion, resulting in identity-motion conflicts throughout the generation process that systematically degrades. To address this, we introduce DualReal, a novel framework that, employs adaptive joint training to collaboratively construct interdependencies between dimensions. Specifically, DualReal is composed of two units: (1) Dual-aware Adaptation dynamically selects a training phase (i.e., identity or motion), learns the current information guided by the frozen dimension prior, and employs a regularization strategy to avoid knowledge leakage; (2) StageBlender Controller leverages the denoising stages and Diffusion Transformer depths to guide different dimensions with adaptive granularity, avoiding conflicts at various stages and ultimately achieving lossless fusion of identity and motion patterns. We constructed a more comprehensive evaluation benchmark than existing methods. The experimental results show that DualReal improves CLIP-I and DINO-I metrics by 21.7% and 31.8% on average, and achieves top performance on nearly all motion quality metrics.

How does it work?

Training Paradigm: At each training step, we first dynamically select the training phase Z(i.e., identity or motion) to determine the data processing path. The specific data undergoes noise injection and combines with the text embeddings. StageBlender Controller governs two-dimension adapters' contributions in Dual-Aware Block (DA-Block) through time-aware conditioning of current denoising step and fused feature representations. In DA-Block, the training-stage(Z) adapter learns the current information guided by the frozen dimension prior, and employs a regularization strategy to avoid dimensional knowledge leakage, achieving joint training. Both branches engage in residual connections with DiT outputs.

StageBlender Controller employs Adaptive LayerNorm mechanism modulates text-visual feature based on timestep-conditional embeddings, then maps the feature to multiple groups after residual gated connections. These scaled weight are subsequently routed to their respective DA-Blocks for processing.

Comparison Results

Qualitative comparison with existing methods. Compared with other methods, DualReal achieves high identity consistency with coherent motion, demonstrating the advantage of joint training in balancing pattern conflicts.

Quantitative comparison of personalization video generation for customized subject and motion. "T.Cons" and "T.Flickering" denotes Temporal Consistency and Temporal Flickering, respectively. Compared with other methods, DualReal achieved average improvements of 21.7% on CLIP-I and 31.8% on DINO-I, recorded the best results on three motion quality metrics (T.Cons, Motion Smoothness, and Temporal Flickering), and ranked second on CLIP-T. The motion datasets achieve an average Dynamic Degree of 12.02 and parenthetical values quantify the current method’s deviation from this benchmark to determine the intensity consistency of movement.