JavaScript SDK

​

What you need

• A Simli API key, from your Simli dashboard
• A backend route that creates session tokens. Your API key stays there and never goes in the browser.

​

Install

npm install simli-client

​

1. Add video and audio elements

import { useRef } from "react";

function YourComponent() {
	const videoRef = useRef<HTMLVideoElement>(null);
	const audioRef = useRef<HTMLAudioElement>(null);

	return (
		<div>
			<video ref={videoRef} autoPlay playsInline></video>
			<audio ref={audioRef} autoPlay></audio>
		</div>
	);
}

const videoElement = document.getElementById("video") as HTMLVideoElement;
const audioElement = document.getElementById("audio") as HTMLAudioElement;

​

2. Create a session token on your server

import { generateSimliSessionToken } from "simli-client";

// Runs on your server, so SIMLI_API_KEY never reaches the browser.
export async function POST() {
	const { session_token } = await generateSimliSessionToken({
		apiKey: process.env.SIMLI_API_KEY!,
		config: {
			faceId: "bf13113a-355a-4d97-8b90-4edccd1da7b2", // a Trinity-2 preset face
			maxSessionLength: 600, // seconds
			maxIdleTime: 180, // seconds
		},
	});
	return Response.json({ session_token });
}

​

3. Connect

import { SimliClient, LogLevel } from "simli-client";

async function startSimli(videoElement: HTMLVideoElement, audioElement: HTMLAudioElement): Promise<SimliClient> {
	const res = await fetch("/api/simli-token", { method: "POST" });
	if (!res.ok) throw new Error(`Couldn't create a Simli session token: ${res.status}`);
	const { session_token } = await res.json();

	const simliClient = new SimliClient(session_token, videoElement, audioElement, null, LogLevel.INFO);
	simliClient.on("stop", () => console.log("Session ended"));
	await simliClient.start(); // resolves once the avatar's first video frame is on screen
	return simliClient;
}

​

4. Send audio

// audioData: a Uint8Array of PCM16, 16 kHz, mono audio
simliClient.sendAudioData(audioData);

simliClient.ClearBuffer();

​

Choosing handleSilence

• true: Simli fills the gaps with silence, so the avatar keeps moving naturally between sentences and picks up again when you send more. Keep it on when you send audio in bursts with sendAudioData, such as one TTS response at a time.
• false: Simli plays only the audio you send, and the video freezes when you stop sending. Turn it off when you stream continuous audio with listenToMediastreamTrack or listenToAudioElement. Those already send their own silence, and Simli’s gap filling would cut into it and cause audio artifacts.

​

5. Stop

await simliClient.stop();

​

Errors and retries

• Simli refuses the session, for example because the session token has expired, the face ID is unknown, your account is out of minutes, or you hit a rate limit. The error event fires with Simli’s message, and start() rejects at once with the same message, a string such as "ERROR: INVALID_FACE_ID". It doesn’t retry, since a retry would fail the same way.
• The network fails: the connection can’t be opened or drops, or an attempt takes more than 15 seconds. These are the only failures start() retries, up to 10 times, with exponential backoff: it waits 1, 2, 4 and 8 seconds before the first four retries, then 16 seconds before each one after that, each wait plus a little random jitter. Then it rejects with Error("Too Many Retry Attempts Failed to connect"). Your event handlers stay registered across retries.
• Anything else goes wrong before the first frame, such as a message from Simli the SDK doesn’t recognise. start() rejects at once with that message as is, and the SDK logs it as Unknown error from Simli: <message>. Once the session has started, an unrecognised message doesn’t end it: it fires the unknown event and is logged as a warning.

try {
	simliClient = await startSimli(videoRef.current!, audioRef.current!);
} catch (reason) {
	console.error("Simli couldn't start:", reason);
}

​

Peer-to-peer mode (optional)

import { generateIceServers } from "simli-client";

export async function GET() {
	// Falls back to a public STUN server if Simli's can't be fetched
	return Response.json(await generateIceServers(process.env.SIMLI_API_KEY!));
}

const iceServers: RTCIceServer[] = await (await fetch("/api/simli-ice")).json();
const simliClient = new SimliClient(session_token, videoElement, audioElement, iceServers, LogLevel.INFO, "p2p");
await simliClient.start();

​

Next steps

​

Troubleshooting

1. Check the reason start() rejected with, or the error event’s message.
2. Check that your API key is correct and active, and that your face ID is valid.
3. Check that your audio is PCM16, 16 kHz, mono.
4. Check that the page has permission to use the microphone, if you stream it.