登录 EN

添加临时用户

高表现力数字人舞蹈生成的研究与应用

Research and Application on Expressive Digital Human Dance Generation

作者:庄昊霖
  • 学号
    2021******
  • 学位
    硕士
  • 电子邮箱
    zhl******com
  • 答辩日期
    2024.05.08
  • 导师
    吴志勇
  • 学科名
    电子信息
  • 页码
    99
  • 保密级别
    公开
  • 培养单位
    599 国际研究生院
  • 中文关键词
    高表现力舞蹈生成; 数字人舞蹈; 舞蹈流派; 舞动律动; 整体性舞蹈
  • 英文关键词
    Expressive Dance Generation; Digital Human Dance; Dance Genre; Dance Grooving Movement; Holistic Dance

摘要

随着国民物质生活水平提升,公众对于文化艺术和舞蹈艺术的追求日益增长,促使了基于数字人的舞蹈艺术技术方案不断涌现。虽然这些技术方案能够根据音乐生成数字人舞蹈,但其舞蹈表现力往往不尽人意。在舞蹈艺术中,舞蹈表现力扮演着极为关键的角色,直接左右舞蹈作品的艺术价值。因此,深入探讨如何提升舞蹈表现力、开发高表现力的数字人舞蹈生成技术、并将这些高表现力的数字人舞蹈应用于实际中,对于推动舞蹈艺术的数字化进程具有重要意义。本研究的主要研究内容包括:第一,在中国传统舞蹈的表现力提升要素基础上,结合人工智能技术可行性,识别并总结出三个关键且能通过深度学习实现的舞蹈表现力提升因素,包括流派一致性、律动可控性和全身整体性。第二,提出了三个数字人舞蹈生成方案,包括音乐驱动的数字人流派一致舞蹈生成、音乐驱动的数字人律动可控舞蹈生成、数字人全身整体性舞蹈生成。第三,整合了三个高表现力数字人舞蹈生成的推理方案,对所提出的交叉创新成果进行了系统部署等工程实践,便于用户使用高表现力数字人舞蹈生成技术进行创新应用,提升了本研究的应用实践价值。结果表明,本研究显著提升了数字人舞蹈的表现力,在用户测评的表现力评分上提升了16.38%。针对流派一致性任务,实现了从音乐推断舞蹈流派,实现了具备一定泛化能力的流派一致数字人舞蹈生成,在舞蹈动作质量上提升了12.26%,流派一致性上提升了12.18%。针对律动可控性任务,实现了对舞蹈中律动动作的提取,并验证了律动提取方案的有效性,继而对于律动进行生成及可控融合,在节拍对齐度上达到了最优(BAS=0.2543)。针对全身整体性任务,实现了舞蹈中面部动作的生成,使数字人在舞蹈过程中能做到表情丰富及口型同步,并结合工程方案实现了数字人舞蹈的全身参数统一,在肢体动作、面部动作质量上皆达到了最优(FID_k=20.34,LVD=0.126),在全身整体性上提升了37.67%。本研究的主要贡献点如下:第一,提出了三个数字人舞蹈生成的表现力提升因素,为提升数字人舞蹈生成表现力开辟了新的路径。第二,提出了三个数字人舞蹈生成方案,实现了数字人舞蹈的流派一致性、律动可控性、全身整体性,提升了舞蹈表现力。第三,实现了高表现力数字人舞蹈生成的工程实践应用,为舞蹈艺术数字化提供了一种新的解决方案。

With the improvement of the national material life, the public‘s pursuit of cultural and dance arts is growing, leading to the continuous emergence of dance art technology solutions based on digital humans. Although these technological solutions can generate digital human dances driven by music, the dancing expressiveness is often unsatisfactory. Dance expressiveness plays an essential role, which determines the artistic value of the dance. Therefore, it is vital to explore how to improve dance expressiveness, to achieve expressive digital human dance generation, and to apply expressive digital human dances technology in practice, with the intention of contributing to the digitisation process of dance art.The main research contents of this thesis are summarised as follows. First, based on the factors for enhancing expressiveness in Chinese traditional dance, this thesis combines the technological feasibility of artificial intelligence to identify and summarise three essential factors for enhancing dance expressiveness which can be practiced through deep learning method: genre consistency, grooving controllability, and holism. Second, three digital human dance generation schemes are proposed, including genre-consistent dance generation, grooving-controllable dance generation, and holistic dance generation. Third, this thesis integrates expressive digital human dance generation inference schemes, carries out system and model deployment for the proposed cross-innovation research achievements, which facilitates users in utilizing expressive digital human dance generation technology for innovative applications, increases the practical value for this thesis.The experimental results show this thesis significantly improves the expressiveness of digital human dance. According to the user study, the dance expressiveness is improved by 16.38%. In the genre-consistent approach, this thesis achieves to infer dance genres from music and genre-consistent dance generation with a certain generalization capability. The quality of dance movement is improved by 12.26% and genre consistency is improved by 12.18%. In the grooving-controllable approach, this thesis achieves the extraction of grooving movements and demonstrates the effectiveness of the extraction scheme. The grooving movements are then generated and fused in a controlled mechanism. It reaches the maximum in beat alignment score (BAS=0.2543). In the holistic approach, this thesis achieves the generation of facial expressions, enables the digital human to have rich emotions and lip-sync while dancing, and achieves the whole-body parameters unification for digital human dance. Body movements and facial expressions reach the optimum in terms of quality (FID_k=20.34, LVD=0.126), and dance holism is improved by 37.67%. The main research contributions of this thesis are summarised as follows. First, three factors for enhancing expressiveness in digital human dance generation are proposed, which provides a potential solution for expressive digital human dance generation. Second, three digital human dance generation schemes are proposed, achieving genre consistency, grooving controllability, and holism, enhancing dance expressiveness. Third, a practical engineering application of expressive digital human dance generation is realised, provides a new solution for the digitisation of dance art.