Skip to main content
The Simli WebRTC SDK is a powerful tool for integrating real-time audio and video streaming capabilities into your python applications using WebRTC. You can use this SDK to display a Simli outside the browser, save the stream into a file, or integrate your stream in your program however you want!

​
Getting Started

​
Prerequisites

Before you begin, make sure you have:
  1. A Simli account with an API key
  2. Python3.9 or higher

​
Installation

Install simli-ai package: 
pip install simli-ai

​
Usage

​
Step 1: Creating the SimliClient

Create an instance of SimliClient and initialize it with your configuration: 
import asyncio
from simli import SimliClient, SimliConfig
from simli.renderers import FileRenderer
async def main():
    connection =  SimliClient(
        api_key="YOUR_SIMLI_API_KEY",  # API Key
        SimliConfig(
            faceId="YOUR_FACE_ID",  # Face ID
            maxSessionLength=20,
            maxIdleTime=10,
        )
    )
    await connection.start()

asyncio.run(main())
Make sure to replace "YOUR_SIMLI_API_KEY" with your actual Simli API key, and "YOUR_FACE_ID" with the desired face ID for your application.

​
Step 2: Send Audio Data

Once the connection is established, you can start sending audio data: 
# Example: sending audio data (should be PCM16 format, 16KHz sample rate)
await connection.sendSilence(1) # to send 1 second of silence to the API
await connection.send(audio) # where audio is a bytes object representing the raw audio data
Ensure that your audio data is in PCM16 format with a 16KHz sample rate.

​
simli-ai API reference

​
simli.SimliConfig

This is a dataclass that represents the config sent to this endpoint to get a session_token
  • faceId: str # FaceID you can get from default avatars or create your own on the simli website
  • handleSilence: bool. Specify whether you want to keep the Simli Avatar Moving when you don’t send in Audio default true. Set to false for absolute control (difficult to get response time right)
  • maxSessionLength: int. A hard time limit (in seconds) after which the session will disconnect
  • maxIdleTime: int. A soft time limit (in seconds) that disconnects the session after a period of not sending in data

​
simli.SimliClient

  • class SimlClient(api_key: str, config: SimliConfig, simliURL: str = "https://api.simli.ai",retry_count: int = 3, retry_timeout: float = 15.0, enableSFU: bool = True, transport_mode: TransportMode = TransportMode.P2P)
    • SimliClient constructor
      • api_key: str. Your API key you get from this page
      • config: refer to SimliConfig
      • simliURL: str. Used if you’re setting up a proxy for Simli Backend. Not useful for most
      • retry_count: int = 3. How many reconnection attempts before an error is thrown
      • retry_timeout: float. Time between retry attempts
      • enableSFU: bool. Only works for TransportMode.P2P. Specifies whether traffic should run through cloudflare network or not
      • transport_mode: TransportMode. Transport Mode. Livekit or Peer to Peer
  • async def start(): Initializes the SimliClient with the provided configuration and starts the WebRTC Connection
  • async close(): Closes the WebRTC connection and cleans up resources.
  • registerEventCallback(event: SimliEvent,async_callback: Callable[[], Awaitable[None]]). Register a callback to a SimliEvent
  • async send(data: bytes | string): Sends audio data or control signals to the server.
  • async sendImmediate(data: bytes): Sends audio data for immediate playback. Clears all previous audio in the buffer.
  • async sendSilence(duration: float= 0.1875): Sends the specified duration to the server as silent audio, used mostly in Idle mode.
  • async clearBuffer(): Clears the audio buffer, best used when you want the avatar to stop talking.
  • async getNextVideoFrame(): returns the following PyAV VideoFrame.
  • async getNextAudioFrame(): returns the following PyAV AudioFrame.
  • async getVideoStreamIterator(targetFormat:str): Creates an async iterator that retrieves all the video frames in order. Can be used in an async for loop.
    • targetFormat: specify what format you want the frames to be in, defaults to RGB.
  • async getAudioStreamIterator(targetSampleRate): Creates an async iterator that retrieves all the video frames in order. Can be used in an async for loop.
    • targetSampleRate: int: specify the sampling rate of the output audio, specify if you want to use the audio in something else that has specific sampling rate constraints

​
SimliEvent

Possible signals that are sent from the server. Can be used for callbacks. You can refer to JS SDKfor an explanation of the events 
class SimliEvent(StrEnum):
    START = auto()
    STOP = auto()
    ERROR = auto()
    SILENT = auto()
    SPEAK = auto()
    ANSWER = auto()
    VIDEO_META = auto()
    DESTINATION = auto()
    UNKNOWN = auto()
    ACK = auto()

​
Fork and Contribute to simli-client

simli-ai

Fork and contribute to the simli-client repository on GitHub or clone for your own usecase.

​
Troubleshooting

If you encounter issues:
  1. Ensure your API key is correct and active.
  2. Verify that your face ID is valid and associated with your account.
  3. Check that your audio data is in the correct format (PCM16, 16KHz).
  4. Verify that you have the necessary permissions for accessing the user’s media devices.
  5. Review the console logs for any error messages or warnings.
Reach out to our support team for further assistance on Discord.