The Lipz file format is a simple description of lip animation and subtitles corresponding to an audio file for use in games and other software that performs character animation. The format has the assumption that the game will be responsible in choosing assets and how to animate, and concerns itself with providing just enough information as to be useful for the game.
Here’s a quick example of a Lipz file to give you the gist:
{
"text-en_us" : "Now, boys and girls, I'd like you to put away your readers|and get ready for a very special treat.",
"visemes" : "ABBBCABBFGGBBCAABBCDFFGBAAAABAACDEFFFCCBBBABBCCCDB|CCCBBAAGGBBCCDDDBBAACCBBBDDBBCCAAABCCC---",
"viseme_type" : "toonboom",
"fps" : 24
}
The Lipz file above is associated with one audio file containing the spoken dialogue it describes. The game uses information in the Lipz file to show subtitles and synchronize lip animation of the character as the audio file plays.
The text element contains text for the game to display on the screen as subtitles. There is a special “segment delimiter” character (|) that shows a point in the text that can be used to segment the text into separate parts for subtitle display. In this case, first a subtitle with “Now, boys and girls, I’d like you to put away your readers” would be shown, followed by a subtitle with “and get ready for a very special treat.”
The visemes element contains codes that indicate specific visemes (lip frames) to show at different points in time. The segment delimiter (|) appears here to mark the point in the audio that corresponds to the same point in the text element marked by a segment delimiter.
viseme_type and fps are used to interpret the codes in the visemes element correctly.
The full Lipz specification can be downloaded from the link below: