• Philipp Lechner

Getting started with Auto-Lip-Sync

Updated: Apr 28



Install Auto-Lip-Sync


You can download the latest version from your dashboard.


To install Auto-Lip-Sync in Blender

1. Open Blender

2. Open the "Preferences" via (Edit -> Preferences)

3. Navigate to "Add-ons"


4. Click "Install"




5. Select the downloaded ZIP file

6. Active the addon by ticking the checkbox


Getting Started

You can find Auto-Lip-Sync in the context menu which can be opened by pressing "N" on your keyboard.




Next, log in your credentials. To start synchronizing select your face rig and activate pose mode via the drop-down in the top left corner or by pressing Ctrl + Tab.


Rig Configuration

Create a pose library

If you haven't already, you need to create a pose library for your face-rig to use Auto-Lip-Sync.

  1. Select all the bones of your face-rig armature

  2. Activate "Pose Mode" via the drop-down in the top left corner or by pressing Ctrl + Tab.

  3. In the "Properties Panel" navigate to the "Object Data Properties"

  4. Within the section "Pose Library" click "New" and name the pose library appropriately



5. Click on the shield icon right next to the pose library name to save the library


6. Move your rig in the desired pose

7. Click on the "+" to save this pose and name the pose accordingly



8. Repeat the process until you saved all of your poses


Setup your face-rig

Next, go to "Rig Configuration" in the Auto-Lip-Sync area. Here you can define a pose for every phoneme. On the left, you can see the phoneme and an example word with that phoneme. Based on this information you can choose the pose of your face rig which best fits that phoneme. Note that for most face rigs you will use a pose several times.


Example configuration for the first ten phonemes

Settings


Audio Path

Please provide the path to the audio file by clicking on the folder symbol on the right-hand side. You can use audio files in the most common formats like WAV, MP3, MP4, OGG, WMA, ACC of FLAC.


Transcript Path

Please provide the path to the transcript by clicking on the folder symbol on the right-hand side.


Note: This file should be saved as a plain file (TXT file).


Start frame

This setting allows you to manually set the beginning of your facial animation. By default, the animation will start at the frame selected by your cursor in the timeline.



Interpolation

You can choose how to interpolate between the different poses:

  • Bezier: smooth transition between poses (Default)

  • Linear: smooth transition between poses with a sudden start and stop

  • Constant: no transition phase between poses