LiveKit Agents

​

What you need

• A LiveKit server that’s reachable from the internet: a LiveKit Cloud project (the free tier works), or a self-hosted LiveKit server with a public address. You need its URL, API key and API secret.
• A Simli API key, from your Simli dashboard
• A key for your model provider. The starter uses OpenAI’s Realtime API, so an OpenAI API key is all it needs: there’s no separate speech-to-text or text-to-speech provider.
• Python 3.10 or later (the starter uses uv), and Node.js 20.9 or later for the starter’s web page

​

Run the starter

git clone https://github.com/simliai/create-simli-app-livekit-agents
cd create-simli-app-livekit-agents
uv sync
cp .env.example .env                          # then paste in your keys
cp frontend/.env.example frontend/.env.local  # then copy in the same three LIVEKIT_ values
uv run agent.py dev

cd frontend && npm install && npm run dev

​

Build it yourself

​

1. Install the packages

pip install "livekit-agents[openai,simli]" python-dotenv

​

2. Set your keys

LIVEKIT_URL=wss://your-project.livekit.cloud
LIVEKIT_API_KEY=...
LIVEKIT_API_SECRET=...
SIMLI_API_KEY=...
OPENAI_API_KEY=...

​

3. Add the avatar to your agent

import asyncio
import os

from dotenv import load_dotenv
from livekit.agents import Agent, AgentServer, AgentSession, JobContext, cli
from livekit.plugins import openai, simli

load_dotenv()

server = AgentServer()


# Explicit dispatch: the agent only joins rooms whose token names it.
@server.rtc_session(agent_name=os.getenv("LIVEKIT_AGENT_NAME") or "simli-agent")
async def entrypoint(ctx: JobContext) -> None:
    session = AgentSession(llm=openai.realtime.RealtimeModel(model="gpt-realtime", voice="marin"))

    # Connect first: wait_for_join() below only waits once the room is connected.
    await ctx.connect()

    avatar = simli.AvatarSession(
        simli_config=simli.SimliConfig(
            api_key=os.environ["SIMLI_API_KEY"],
            face_id=os.getenv("SIMLI_FACE_ID") or "bf13113a-355a-4d97-8b90-4edccd1da7b2",
            max_session_length=600,  # seconds; the avatar leaves after this
            max_idle_time=300,  # seconds without speaking before the avatar leaves
        ),
    )
    await avatar.start(session, room=ctx.room)
    # If Simli refuses the session (a bad key or face ID, or no free session), the plugin
    # logs the error and returns without connecting the agent's voice to the avatar.
    if session.output.audio is None:
        ctx.delete_room()
        return

    # Don't start talking until the avatar is in the room, so its first words aren't lost.
    try:
        await avatar.wait_for_join(timeout=30)
    except asyncio.TimeoutError:
        ctx.delete_room()
        return

    await session.start(agent=Agent(instructions="You are a friendly voice assistant."), room=ctx.room)
    session.generate_reply(instructions="Greet the user in one short sentence.")


if __name__ == "__main__":
    cli.run_app(server)

​

4. Run it

python agent.py dev     # while developing, with debug logs
python agent.py start   # in production

​

5. Talk to it

​

Troubleshooting

• The Simli avatar didn’t start: the Simli error logged just before says why. 401 means SIMLI_API_KEY is wrong, 400 means the face ID wasn’t found, and 429 means your Simli account is already running as many sessions as it can.
• The Simli avatar didn’t join the room: usually Simli couldn’t reach your LiveKit server. LIVEKIT_URL must be reachable from the internet, not localhost.
• The page says the avatar didn’t join: check that the agent is running, that .env and frontend/.env.local hold the same LIVEKIT_* values, and that LIVEKIT_AGENT_NAME matches in both if you set it.

​

Before you go to production

• Anyone who can join a room in your LiveKit project starts a conversation, which uses your Simli minutes and model credit. The starter page’s token route hands a token to anyone who opens the page, so put it behind your own sign-in before you deploy it.
• The avatar leaves the room as soon as its agent does, so ending the agent’s session, or deleting the room, ends the Simli session and its billing. A brief network blip on the agent’s side doesn’t count: LiveKit resumes the connection.
• To deploy the agent, run python agent.py start on any server, or see LiveKit’s deployment guide. The page builds with npm run build and runs with npm start, or deploys to any Next.js host.

​

Learn more

• The starter’s README: changing the face, model, voice and prompt
• LiveKit’s guide to Simli
• LiveKit Agents docs: tools, other models and more