SimliClient is a powerful tool for integrating real-time audio and video streaming capabilities into your web applications using WebRTC. This guide will walk you through the process of setting up and using SimliClient in your project.

​
Getting Started

​
Prerequisites

Before you begin, make sure you have:
  1. A Simli account with an API key
  2. Node.js and npm installed in your development environment

​
Installation

Install the simli-client package in your project: 
npm install simli-client

​
Usage

​
Step 1: Import SimliClient

First, import the SimliClient class into your project: 
import { SimliClient } from "simli-client";

​
Step 2: Create HTML Elements

In your React component, create video and audio elements with refs: 
import React, { useRef } from "react";

function YourComponent() {
  const videoRef = useRef<HTMLVideoElement>(null);
  const audioRef = useRef<HTMLAudioElement>(null);

  return (
    <div>
      <video ref={videoRef} autoPlay playsInline></video>
      <audio ref={audioRef} autoPlay></audio>
    </div>
  );
}

​
Step 3: Initialize SimliClient

Create an instance of SimliClient and initialize it with your configuration: 
const simliClient = new SimliClient();

const simliConfig = {
  apiKey: "YOUR_SIMLI_API_KEY",
  faceID: "YOUR_FACE_ID",
  handleSilence: true, // keep the face moving while in idle
  maxSessionLength: 3600, // in seconds
  maxIdleTime: 600, // in seconds
  videoRef: videoRef.current,
  audioRef: audioRef.current,
  enableConsoleLogs: true, // enables Simli console logs
};

simliClient.Initialize(simliConfig);
Make sure to replace 'YOUR_SIMLI_API_KEY' with your actual Simli API key, and 'YOUR_FACE_ID' with the desired face ID for your application.
If you’re using listenToMediastreamTrack() set handleSilence: false to avoid audio artifacts

​
Step 4: Start the WebRTC Connection

Call the start method to set up the WebRTC connection: 
simliClient.start();

​
Step 5: Send Audio Data

Once the connection is established, you can start sending audio data: 
// Example: sending audio data (should be PCM16 format, 16KHz sample rate)
const audioData = new Uint8Array(6000).fill(0); // Replace with your actual audio data
simliClient.sendAudioData(audioData);
Ensure that your audio data is in PCM16 format with a 16KHz sample rate.

​
simli-client API reference

​
Methods

  • Initialize(config: SimliClientConfig): Initializes the SimliClient with the provided configuration.
  • start(): Sets up the WebRTC connection and prepares for streaming.
  • close(): Closes the WebRTC connection and cleans up resources.
  • sendAudioData(audioData: Uint8Array): Sends audio data to the server.
  • listenToMediastreamTrack(stream: MediaStreamTrack): Listens to a MediaStreamTrack and sends audio data to the server. Can be used as an alternative to sendAudioData.
  • ClearBuffer(): Clears the audio buffer, best used when you want the avatar to stop talking.

​
Advanced Usage

​
Events

  • connected when the data channel is open and ready to use
  • disconnected when the data channel is closed
  • failed when the webRTC connection fails to connect
  • speaking when the avatar starts speaking
  • silent when the avatar stops speaking
simliClient.on("connected", () => {
  console.log("SimliClient is now connected!");
});

simliClient.on("disconnected", () => {
  console.log("SimliClient has disconnected!");
});

simliClient.on("failed", () => {
  console.log("SimliClient has failed to connect!");
});

simliClient.on("speaking", () => {
  console.log("Agent is now speaking!");
});

simliClient.on("silent", () => {
  console.log("Agent is now silent!");
});

​
Error Handling

SimliClient provides console logging for various events and errors. It’s recommended to implement proper error handling in your application to manage potential issues, such as network disconnections or initialization failures.

​
Customizing WebRTC Configuration

The SimliClient uses a default STUN server for ICE candidate gathering. If you need to use custom ICE servers or other WebRTC configurations, you may need to modify the createPeerConnection method in the SimliClient class.

​
Fork and Contribute to simli-client

simli-client

Fork and contribute to the simli-client repository on GitHub or clone for your own usecase.

​
Troubleshooting

If you encounter issues:
  1. Ensure your API key is correct and active.
  2. Verify that your face ID is valid and associated with your account.
  3. Check that your audio data is in the correct format (PCM16, 16KHz).
  4. Verify that you have the necessary permissions for accessing the user’s media devices.
  5. Review the console logs for any error messages or warnings.
Reach out to our support team for further assistance on Discord.