Pipecat

Pipecat is an open-source Python framework for voice and multimodal agents. Simli’s video service drops into a Pipecat pipeline: it takes the bot’s voice and returns the avatar’s lip-synced video and voice, in sync.

What you need

A Simli API key, from your Simli dashboard
An OpenAI API key. The starter uses OpenAI’s Realtime API, speech to speech, so there’s no separate speech-to-text or text-to-speech provider.
Python 3.11 or later (the starter uses uv), and Node.js 20.9 or later for the starter’s web page

Everything runs on your machine: Pipecat connects the browser to the bot over peer-to-peer WebRTC, so there are no other accounts or media servers to set up.

Run the starter

create-simli-app-pipecat

A Pipecat voice agent on OpenAI’s Realtime API, with a Simli Trinity-2 avatar and a small web page to talk to it. Its lockfile pins the versions it was tested with.

git clone https://github.com/simliai/create-simli-app-pipecat
cd create-simli-app-pipecat
uv sync
cp .env.example .env   # then paste in your keys
uv run bot.py

In a second terminal, start the web page:

cd frontend && npm install && npm run dev

Open http://localhost:3000, press Start and allow the microphone. The page holds no keys: it posts its WebRTC offer to its own /api/offer, which Next.js forwards to the bot. If the bot isn’t on http://localhost:7860, start the page with BOT_URL set to where it is. Pipecat’s prebuilt client at http://localhost:7860 works too.

Build it yourself

1. Install the packages

pip install "pipecat-ai[openai,runner,simli,webrtc]" python-dotenv

2. Set your keys

Put them in a .env file next to your bot:

.env

SIMLI_API_KEY=...
OPENAI_API_KEY=...

3. Add the avatar to your pipeline

This is the core of the starter’s bot: the transport, the Simli service and the pipeline. The starter’s bot.py has the complete file, with the imports, the OpenAI Realtime service and the runner.

bot.py

transport = SmallWebRTCTransport(
    webrtc_connection=runner_args.webrtc_connection,
    params=TransportParams(
        audio_in_enabled=True,
        audio_out_enabled=True,
        video_out_enabled=True,
        video_out_is_live=True,
        video_out_width=768,  # Simli's default video size
        video_out_height=768,
        video_out_framerate=25,  # Simli streams at 25 fps
    ),
)

# Simli gets the bot's voice and sends back the avatar's video and voice, in sync.
simli = SimliVideoService(
    api_key=os.environ["SIMLI_API_KEY"],
    face_id=os.getenv("SIMLI_FACE_ID") or "bf13113a-355a-4d97-8b90-4edccd1da7b2",
    max_session_length=600,  # seconds, so a forgotten tab doesn't use up your minutes
    max_idle_time=120,
)

pipeline = Pipeline(
    [
        transport.input(),  # the user's microphone
        user_aggregator,
        llm,  # OpenAIRealtimeLLMService: hears the user and answers in voice
        simli,
        transport.output(),  # the avatar's video and voice
        assistant_aggregator,
    ]
)

Simli goes right after the service that produces the bot’s voice, and before the transport’s output.

This gives a Trinity-2 face Simli’s default 768x768 video. For a 16:9 video and the avatar’s position, and for known issues with the avatar’s first video and with GPT-Live, see Trinity-2 in LiveKit Agents and Pipecat (temporary).

4. Run it

python bot.py

Then open http://localhost:7860, where Pipecat’s runner serves a ready-made browser client, press Connect and allow the microphone.

Before you go to production

The runner is for development. Anyone who can reach port 7860 can start a conversation, which uses your Simli minutes and OpenAI credit, so keep it on localhost or put it behind your own sign-in. To deploy, see Pipecat’s deployment guide.

Learn more

The starter’s README: changing the face, voice and prompt, and how it works
Pipecat’s guide to Simli
Pipecat docs: tools, other models and more

Get Started

Integrations

Simli SDKs

Starter repos

Advanced: Custom Simli Client

Links

What you need

Run the starter

create-simli-app-pipecat

Build it yourself

1. Install the packages

2. Set your keys

3. Add the avatar to your pipeline

4. Run it

Before you go to production

Learn more

​What you need

​Run the starter

create-simli-app-pipecat

​Build it yourself

​1. Install the packages

​2. Set your keys

​3. Add the avatar to your pipeline

​4. Run it

​Before you go to production

​Learn more

What you need

Run the starter

Build it yourself

1. Install the packages

2. Set your keys

3. Add the avatar to your pipeline

4. Run it

Before you go to production

Learn more