This plugin uses voice recognition engine to generate lip-sync animation from SoundWave assets or PCM audio data. Animation is saved as curves in data assets and can be played in runtime together with audio. This approach allows to achieve well-looking lip-sync animation easily without subtitles.
Additional feature: recognize input from microphone (speech-to-text) in runtime.
Technical Details
Features:
subtitles aren't needed;
completely offline;
can generate lip-sync in runtime for loaded/TTS audio;
can generate Anim Sequence assets with lip-sync in curves;
lip-sync for animation curves (universal) or morph targets (when possible);
asynchronous audio recognition and building lip-sync;
(beta!) create lip-sync in runtime on PC and Android using remote server;
additional feature: recognize microphone input (Speech-to-text) in runtime (Windows only).
Code Modules:
YnnkVoiceLipsync (Runtime)
YnnkVoiceLipsyncUncooked (UncookedOnly)
Number of Blueprints: 0
Number of C++ Classes: 8
Network Replicated: No
Supported Development Platforms: Windows x64
Supported Target Build Platforms: Windows x64, Android
Documentation: [Doc]
Example Project: 5.3 | 5.2 | MetaHuman 5.1 with Enhancer Plugin
Executable Demo: [ZIP]
Enhancer Plugin: [5.1 - 5.3]
