SimliClient is awesome on it’s own. However, it uses the API key directly on client-side which isn’t so great for security as someone with a little bit of knowledge can get your API key. With SimliClient 1.2.7+, you can handle auth on your own backend by making the authentication request there and passing the session_token and ICE Config back to the client.

​
Getting Started

​
Usage

​
Step 0: Standard SimliClient

First, import the SimliClient class into your project:

import { SimliClient } from "simli-client";

function YourComponent() {
  const videoRef = useRef < HTMLVideoElement > null;
  const audioRef = useRef < HTMLAudioElement > null;

  return (
    <div>
      <video ref={videoRef} autoPlay playsInline></video>
      <audio ref={audioRef} autoPlay></audio>
    </div>
  );
}

const simliClient = new SimliClient();

const simliConfig = {
  apiKey: "YOUR_SIMLI_API_KEY",
  faceID: "YOUR_FACE_ID",
  handleSilence: true, // keep the face moving while in idle
  maxSessionLength: 3600, // in seconds
  maxIdleTime: 600, // in seconds
  videoRef: videoRef.current,
  audioRef: audioRef.current,
  enableConsoleLogs: true, // enables Simli console logs
};

simliClient.Initialize(simliConfig);
simliClient.start();

Make sure to replace 'YOUR_SIMLI_API_KEY' with your actual Simli API key, and 'YOUR_FACE_ID' with the desired face ID for your application.

​
Step 1: Creating our authentication backend:

This is a really simple backend with one goal, proxying /startAudioToVideoSession and /getIceServer to Simli servers and keeping the API key safe on your backend. This sample backend will now have any auth on it’s own as there’s no one good solution for that and you can pick whatever you like for your auth!

For this example, we’re using FastAPI with python but as we mentioned, you can use anything you like.

import os
from dotenv import load_dotenv
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import httpx

# Load environment variables from the .env file
load_dotenv()
SIMLI_API_KEY = os.getenv("SIMLI_API_KEY")
if not SIMLI_API_KEY:
    raise ValueError("SIMLI_API_KEY is not set in the environment")

# External API endpoints
EXTERNAL_AUDIO_TO_VIDEO_URL = "https://api.simli.ai/startAudioToVideoSession"
EXTERNAL_ICE_SERVER_URL = "https://api.simli.ai/getIceServer"

# Request model for the audio-to-video session.
# Note: The apiKey is no longer expected from the client.
class StartAudioToVideoSessionRequest(BaseModel):
    faceId: str
    handleSilence: bool = True
    maxSessionLength: int = 3600
    maxIdleTime: int = 300

app = FastAPI()

@app.post("/myAudioToVideoSession")
async def my_audio_to_video_session(request: StartAudioToVideoSessionRequest):
    """
    Proxy endpoint that forwards a POST request to the external
    startAudioToVideoSession API with the API key injected from the .env file.
    """
    # Convert the incoming request to a dict and inject the API key.
    payload = request.dict()
    payload["apiKey"] = SIMLI_API_KEY
    async with httpx.AsyncClient() as client:
      response = await client.post(EXTERNAL_AUDIO_TO_VIDEO_URL, json=payload)
    return response.json()

@app.post("/myIceServer")
async def my_ice_server():
    """
    Proxy endpoint that forwards a POST request to the external
    getIceServer API with the API key injected from the .env file.
    """
    # Build the payload using the API key from the environment.
    payload = {"apiKey": SIMLI_API_KEY}

    async with httpx.AsyncClient() as client:
      response = await client.post(EXTERNAL_ICE_SERVER_URL, json=payload)
    return response.json()

# To run the app, use:
# uvicorn proxy:app --reload
if __name__ == "__main__":
    import uvicorn
    uvicorn.run("app:app", host="0.0.0.0", port=8000, reload=True)

with the .env file being

SIMLI_API_KEY=your_simli_api_key_here

and the requirements.txt file

fastapi
python-dotenv
httpx

​
Step 2: Modify Simli-Client usage

We can make a call to our backend (which we’re assuming is on 127.0.0.1:8000 for now), get the session token and IceServers config and give them to the Simli-Client instance

import { SimliClient } from "simli-client";

function YourComponent() {
  const videoRef = useRef < HTMLVideoElement > null;
  const audioRef = useRef < HTMLAudioElement > null;

  return (
    <div>
      <video ref={videoRef} autoPlay playsInline></video>
      <audio ref={audioRef} autoPlay></audio>
    </div>
  );
}

async function startSimliConnection() {
  const simliClient = new SimliClient();
  const simliConfig = {
    faceID: "YOUR_FACE_ID",
    handleSilence: true, // keep the face moving while in idle
    maxSessionLength: 3600, // in seconds
    maxIdleTime: 600, // in seconds
    videoRef: videoRef.current,
    audioRef: audioRef.current,
    enableConsoleLogs: true, // enables Simli console logs
  };

  const session_token = await fetch(
    "http://localhost:8000/myAudioToVideoSession",
    {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        faceId: simliConfig.faceID,
        handleSilence: simliConfig.handleSilence,
        maxSessionLength: simliConfig.maxSessionLength,
        maxIdleTime: simliConfig.maxIdleTime,
      }),
    }
  );
  simliConfig.session_token = (await session_token.json()).session_token;

  const iceConfig = await (
    await fetch("http://localhost:8000/myIceServer", {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
      },
    })
  ).json();

  simliClient.Initialize(simliConfig);
  simliClient.start(iceConfig);
}
Suggest edits
Javascript SDKPython SDK