{
"type": "<string>",
"sdp": "<string>"
}{}{}{}{}{
"type": "<string>",
"sdp": "<string>"
}{}{}{}{}{}{}Peer_to_peer
Peer to Peer WebRTC
Compose Session Token is sent as query parameter string
enableSFU is a query parameter boolean to opt into sending the traffic through cloudflare network for improved reliability
First message must be offer. Offer must be provided within 30 seconds of opening the websocket. Once offer is sent, a corresponding answer will be sent from the server which can be used as described here
{
"type": "<string>",
"sdp": "<string>"
}{}{}{}{}{
"type": "<string>",
"sdp": "<string>"
}{}{}{}{}{}{}RTC Answer as generated by https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection/createAnswer or compatible. Will be received AFTER an offer has been sent
Server replies to a camera message, sent as text. None of them end the session.
Server replies to a background or foreground message, sent as text; a message that is applied gets none. None of them end the session, and the avatar keeps what it had.
Server sent signals
Termination messages that specify the reason for immediate termination, including a routing error (Routing error, no EU Sessions available. EU ONLY PROCESSING enabled) when a region-pinned session lands on a machine outside its region
RTC Offer as generated by https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection/createOffer or compatible
Binary stream of the audio that the Avatar will lipsync to. Sent as a binary stream (in JS/TS UInt8Arr)
Client Signal to make the avatar stop speaking immediately and ignore all previously sent audio
Client Signal to tell the server that the final segment of audio has been already sent and that it should close after playing it back
Moves the camera, Trinity-2 faces only. Sent as binary: the ASCII prefix CAM_, then little-endian float32 camera offsets on top of the avatar's own framing, in this order (y comes before x):
| # | Field | Meaning | Supported range |
|---|---|---|---|
| 1 | pos_y_off | Moves the avatar down, in half canvas heights | -10 to 10 |
| 2 | pos_x_off | Moves the avatar right, in half canvas widths | -10 to 10 |
| 3 | pos_z_off | Metres the camera backs away from the avatar (negative moves in) | -5 to 5 |
| 4 | zoom | Multiplies the avatar's zoom; 1 is no change | above 0, up to 4 |
| 5 | rot_x_off | Degrees the camera tilts up (pitch) | -180 to 180 |
| 6 | rot_y_off | Degrees the camera pans right (yaw) | -180 to 180 |
| 7 | rot_z_off | Degrees the camera rolls clockwise | -180 to 180 |
| 8 | orbit_x_off | Degrees the camera orbits up over the avatar's head (elevation) | -80 to 80 |
| 9 | orbit_y_off | Degrees the camera orbits round to its right (azimuth) | -180 to 180 |
Send the first 4, the first 7 or all 9 values; the ones left off stay neutral (0).
Each message replaces the whole camera, including the one the session started with (the session token's camera): the values are offsets from the avatar's own framing, not from the previous message. 0, 0, 0, 1, 0, 0, 0, 0, 0 returns to the avatar's own framing.
The camera cuts to the new setting. To glide instead, append the ASCII tag LERP and one more float32: the seconds the move takes, from wherever the camera is now (part-way through an earlier glide, if one is running). 0 or less is a cut.
A message is 20, 32 or 40 bytes, or 28, 40 or 48 with LERP. A value outside its supported range is still applied, with a WARNING: reply. Any other length, NaN or infinity is rejected and the camera stays where it is. See camera_notice.
Puts one of your uploaded backgrounds behind the avatar, Trinity-2 faces only: the ASCII prefix BG_ and the background's id, as POST /assets/backgrounds returned it, 36 characters (hyphens included, either case). Text or binary; 39 characters in all, nothing else is accepted. An image can't be sent: upload it first. The background is cropped to fill the session's canvas.
Only your own backgrounds: an id that isn't one of your account's is ignored. It replaces the one the session started with (the session token's background_id) a moment later, once loaded; audio keeps playing meanwhile, and the last one sent wins. See asset_notice.
BG_00000000-0000-0000-0000-000000000000 (the nil UUID) goes back to the avatar's own background, at once.
Lays one of your uploaded foregrounds over the avatar through its transparency, Trinity-2 faces only: the ASCII prefix FG_ and the foreground's id, as POST /assets/foregrounds returned it, 36 characters. Text or binary; 39 characters in all, nothing else is accepted, an image least of all. Otherwise as background, in place of the session token's foreground_id. FG_00000000-0000-0000-0000-000000000000 (the nil UUID) takes the foreground away.