JavaScript SDK

​

Getting Started

​

Prerequisites

1. A Simli account with an API key
2. Node.js and npm installed in your development environment

​

Installation

npm install simli-client

​

Usage

​

Step 1: Get the Video and Audio element references

function YourComponent() {
	const videoRef = useRef<HTMLVideoElement>(null);
	const audioRef = useRef<HTMLAudioElement>(null);

	return (
		<div>
			<video ref={videoRef} autoPlay playsInline></video>
			<audio ref={audioRef} autoPlay></audio>
		</div>
	);
}

// assuming we have only one video element and one audio element
const videoElement: HTMLVideoElement = document.getElementById("video");
const audioElement: HTMLAudioElement = document.getElementById("audio");

​

Step 2: Create a session token

import {generateSimliSessionToken, SimliSessionRequest} from "simli-client";

async function getSessionToken(): Promise<string> {
	const SimliConfig: SimliSessionRequest = {
		faceId: "YOUR_FACE_ID",
		maxSessionLength: 600, // in seconds
		maxIdleTime: 180, // in seconds
	};
	const session_token = (await generateSimliSessionToken({apiKey: "YOUR_SIMLI_API_KEY", config: SimliConfig})).session_token;
	return session_token;
}

​

(Optional) Step 2.5: Generate Ice Server Config (For running Simli in peer to peer mode)

async function getIceServers(): Promise<RTCIceServers[]> {
	return await getIceServers("YOUR_SIMLI_API_KEY");
}

​

Step 3: Initialize SimliClient

import {SimliClient, LogLevel} from "simli-client";
async function InitializeSimliClient(): SimliClient {
	simliClient = new SimliClient(
		await getSessionToken(),
		videoRef.current,
		audioRef.current,
		await getIceServers(),
		LogLevel.DEBUG, // Optional Paramter, in production it's recommended to set it to info
		"p2p",
	);
	await simliClient.start();
	return simliClient;
}

import {SimliClient, LogLevel} from "simli-client";
async function InitializeSimliClient(): SimliClient {
	simliClient = new SimliClient(
		await getSessionToken(),
		videoRef.current,
		audioRef.current,
		null,
		LogLevel.DEBUG, // Optional Paramter, in production it's recommended to set it to info
		"livekit",
	);
	await simliClient.start();
	return simliClient;
}

​

Step 4: Send Audio Data

// Example: sending audio data (should be PCM16 format, 16KHz sample rate)
const audioData = new Uint8Array(6000).fill(0); // Replace with your actual audio data
simliClient.sendAudioData(audioData);

​

SimliClient API reference

​

Types

interface SimliSessionRequest {
	faceId: string;
	handleSilence: boolean;
	maxSessionLength: number;
	maxIdleTime: number;
	model?: "fasttalk" | "artalk";
}
interface TokenRequestData {
	config: SimliSessionRequest;
	apiKey: string;
}
interface SimliSessionToken {
	session_token: string;
}

type TransportMode = "livekit" | "p2p";
type SignalingMode = "websockets";
type session_token = string;

​

Methods

• async generateSimliSessionToken( request: TokenRequestData, SimliURL: string = "https://api.simli.ai",): Generate new session token given a TokenRequestData object. You can also redirect that request to your own backend.
• new SimliClient(session_token: string, videoElement: HTMLVideoElement, audioElement: HTMLAudioElement, iceServers: RTCIceServer[] | null, logLevel: LogLevel = LogLevel.DEBUG, transport_mode: TransportMode = "p2p", signaling: SignalingMode = "websockets", SimliWSURL: string = "wss://api.simli.ai", audioBufferSize: number = 3000,): Initializes the SimliClient with the provided configuration.
• async SimliClientInstance.start(): Connects everything to Simli Servers with automatic retry on failure
• async SimliClientInstance.stop(): Closes the WebRTC connection and cleans up resources.
• SimliClientInstance.sendAudioData(audioData: Uint8Array): Sends audio data to the server.
• SimliClientInstance.listenToMediastreamTrack(stream: MediaStreamTrack): Listens to a MediaStreamTrack and sends audio data to the server. Can be used as an alternative to sendAudioData.
• SimliClientInstance.ClearBuffer(): Clears the audio buffer, best used when you want the avatar to stop talking.

​

Advanced Usage

​

Events

• start when the webrtc connection is successful and avatar is shown on screen
• stop when the simli server terminates a connection normally
• error when the webRTC connection fails to connect or crashes in the middle
• speaking when the avatar starts speaking
• silent when the avatar stops speaking
• ack when server acknowledges an audio segment has been received
• connection_info(serialized_info: string) when rtc answer or livekit session info is received not useful in most applications and is meant for primarily internal use
• video_info(serialized_info: string) shows video FPS, height, and width
• destination(serialized_info: string) debugging information useful when reporting issues to simli
• unknown(message: string) when the server sent a message that couldn’t be parsed. Usually future messages that have been added to later versions of the client
• startup_error(message: string) when there was a fundamental error causing simli to be unable to connect. This is usually triggered for invalid FaceIDs or depleted minutes. Message should be the cause of this startup failure. Automatic retries are disabled on startup_error

simliClient.on("EVENT", callbackFunction);

​

Error Handling

​

Fork and Contribute to simli-client

​

Troubleshooting

1. Ensure your API key is correct and active.
2. Verify that your face ID is valid and associated with your account.
3. Check that your audio data is in the correct format (PCM16, 16KHz).
4. Verify that you have the necessary permissions for accessing the user’s media devices.
5. Review the console logs for any error messages or warnings.