Python SDK

simli-ai runs Simli sessions from Python. You send the audio the avatar should speak, and get back its lip-synced video and audio as PyAV frames, to save to a file, show in a window or process however you like.

Building a voice agent? LiveKit Agents and Pipecat have Simli integrations that handle the audio and video for you.

What you need

A Simli API key, from your Simli dashboard
Python 3.11 or later

The SDK runs where your API key is safe, so there’s no session token step.

Install

pip install simli-ai

To show the avatar in a window with LocalRenderer, install the local extra instead: pip install "simli-ai[local]".

1. Start a session

import asyncio

from simli import SimliClient, SimliConfig
from simli.renderers.renderers import FileRenderer


async def main():
    with open("speech.pcm", "rb") as f:  # PCM16, 16 kHz, mono
        audio = f.read()

    async with SimliClient(
        api_key="YOUR_SIMLI_API_KEY",
        config=SimliConfig(
            faceId="bf13113a-355a-4d97-8b90-4edccd1da7b2",  # a Trinity-2 preset face
            maxSessionLength=120,  # seconds
            maxIdleTime=60,  # seconds
        ),
    ) as connection:
        await connection.send(audio)
        await FileRenderer(connection).render()  # writes output.mp4


asyncio.run(main())

Replace "YOUR_SIMLI_API_KEY" with your Simli API key. To use another avatar, replace the face ID with one from your dashboard or the default faces. For the video size, the avatar’s position, the camera and backgrounds, see Trinity-2. async with starts the session and stops it when the block ends. Without it, call await connection.start() first and await connection.stop() when you’re done.

2. Send audio

send() takes raw PCM16 audio, 16 kHz, mono, as bytes. It plays after any audio already queued.

await connection.send(audio)
await connection.clearBuffer()  # drop the queued audio, e.g. when the user interrupts
await connection.sendImmediate(audio)  # drop the queued audio and play this instead

handleSilence in SimliConfig (default True) decides what Simli does when your audio runs out. With True, Simli fills the gaps with silence, so the avatar keeps moving between sentences. Set it to False when you stream continuous audio that already has its own silences: Simli then plays only the audio you send, and the video freezes when you stop sending.

3. Get the video and audio

The renderers in simli.renderers.renderers take the stream for you:

FileRenderer(connection) writes it to output.mp4.
LocalRenderer(connection) shows it in a window and plays it on your speakers. It needs the local extra and a display.
NDArrayRenderer(connection) returns the video and audio as NumPy arrays once the session ends.

To handle the frames yourself, iterate over them:

async for frame in connection.getVideoStreamIterator("rgb24"):
    ...  # a PyAV VideoFrame

getAudioStreamIterator() does the same for audio. Run the two loops as separate tasks to receive both at once.

Errors and retries

start() returns once Simli has started the session. If it can’t, it raises one of these exceptions, all exported from simli and all subclasses of SimliError:

SimliAPIError: a request to the Simli API failed, for example an invalid API key, an unknown face ID or no minutes left. status_code and detail hold the HTTP status and Simli’s message.
SimliSessionError: Simli refused the session, for example an expired session token or the rate limit. Also a message from Simli the SDK doesn’t recognise, sent while the session is starting. That one is logged as Unknown error from Simli: <message>, and its code is UNKNOWN_ERROR.
SimliConnectionError: the network failed. A request or the websocket couldn’t connect, timed out or dropped.

start() retries only network failures: SimliConnectionError and HTTP 5xx responses. It tries up to retry_count times in all (3 by default), with exponential backoff: it waits 1, 2, 4 and 8 seconds before the first four retries, then 16 seconds before each one after that, each wait plus a little random jitter. Everything else raises at once.

from simli import SimliAPIError, SimliError

try:
    await connection.start()
except SimliAPIError as e:
    print(e.status_code, e.detail)  # e.g. 400 INVALID_FACE_ID
except SimliError as e:
    print(e)

Once the session has started, a message the SDK doesn’t recognise no longer ends it: it’s logged as a warning on the simli_client logger. Once the session has ended, connection.stopReason holds the reason Simli gave (such as "IDLE" or "MAX"), connection.error holds the error that ended it, if one did, and send() raises instead of failing silently.

Next steps

Trinity-2

Video size, the avatar’s position, the camera, backgrounds and foregrounds.

API reference

Every class, method, event and exception.

Questions? Ask on Discord, or fork simli-ai for your own use case.

Get Started

Integrations

Simli SDKs

Starter repos

Advanced: Custom Simli Client

Links

What you need

Install

1. Start a session

2. Send audio

3. Get the video and audio

Errors and retries

Next steps

Trinity-2

API reference

​What you need

​Install

​1. Start a session

​2. Send audio

​3. Get the video and audio

​Errors and retries

​Next steps

Trinity-2

API reference

What you need

Install

1. Start a session

2. Send audio

3. Get the video and audio

Errors and retries

Next steps