Pipecat C++ Client SDK 1.3.0
Connect native apps to Pipecat bots
Loading...
Searching...
No Matches
pipecat::SmallWebRTCTransport Class Reference

#include <pipecat/smallwebrtc/transport.h>

Inheritance diagram for pipecat::SmallWebRTCTransport:
pipecat::Transport

Public Member Functions

 SmallWebRTCTransport (SmallWebRTCTransportOptions options={})
 Creates a transport.
 
 ~SmallWebRTCTransport () override
 Disconnects if needed.
 
 SmallWebRTCTransport (const SmallWebRTCTransport &)=delete
 Transports can't be copied.
 
SmallWebRTCTransport & operator= (const SmallWebRTCTransport &)=delete
 Transports can't be copied.
 
void initialize (TransportObserver *observer) override
 Prepares the transport.
 
void set_start_bot_params (const APIRequest &request) override
 
void connect (const nlohmann::json &params) override
 
void disconnect () override
 Closes the connection.
 
void send_ready_message (const rtvi::Message &message) override
 Sends the client-ready message.
 
void send_message (const rtvi::Message &message) override
 Sends a message to the bot.
 
int32_t send_user_audio (const int16_t *frames, size_t num_frames) override
 
int32_t read_bot_audio (int16_t *frames, size_t num_frames) override
 
- Public Member Functions inherited from pipecat::Transport
virtual size_t max_message_size () const
 The largest message, in bytes, that send_message() accepts.
 

Detailed Description

Connects to Pipecat bots over WebRTC, peer to peer, with Pipecat's SmallWebRTC transport.

Give it to PipecatClientOptions::transport. To connect, it sends a WebRTC offer to the bot's server. With PipecatClient::start_bot(), it finds out where on its own: start endpoints like Pipecat's development runner (python bot.py -t webrtc) and Pipecat Cloud answer with a session ID when asked for the webrtc transport. Without it, give it the offer endpoint, see connect().

You can use several at the same time, e.g. one per client to talk to several bots.

Member Function Documentation

◆ connect()

void pipecat::SmallWebRTCTransport::connect ( const nlohmann::json &  params)
overridevirtual

Connects to the bot.

params says where to send the offer, with one of:

  • sessionId: the session a start endpoint created, after PipecatClient::start_bot(). The offer goes to the start endpoint's URL, with /start at the end of its path replaced by /sessions/<sessionId>/api/offer, and with the same headers.
  • webrtcRequestParams: the offer endpoint, as a JSON object with its URL in endpoint, and optionally HTTP headers in headers and data for the bot in requestData.

If both are there, webrtcRequestParams is used. Optionally, iceConfig has STUN and TURN servers in iceServers, each with its URLs in urls, and username and credential for TURN. Every key can also be in snake case, e.g. session_id.

Throws InvalidTransportParamsError if the params are wrong, and TransportStartError if connecting fails.

Implements pipecat::Transport.

◆ disconnect()

void pipecat::SmallWebRTCTransport::disconnect ( )
overridevirtual

Closes the connection.

Implements pipecat::Transport.

◆ initialize()

void pipecat::SmallWebRTCTransport::initialize ( TransportObserver *  observer)
overridevirtual

Prepares the transport.

Implements pipecat::Transport.

◆ read_bot_audio()

int32_t pipecat::SmallWebRTCTransport::read_bot_audio ( int16_t *  frames,
size_t  num_frames 
)
overridevirtual

Reads num_frames frames of 16-bit PCM bot audio into frames, converted to the format of the options. Waits until it's time to play them, or until the transport disconnects. Returns the number of frames read, or 0 if it disconnected.

The bot's audio plays at the pace the bot sent it, 80 ms behind, so the network's ups and downs don't reach your app: audio that arrives late or out of order still plays in its turn, and lost audio is concealed. Read it from your audio thread, as you play it.

Implements pipecat::Transport.

◆ send_message()

void pipecat::SmallWebRTCTransport::send_message ( const rtvi::Message &  message)
overridevirtual

Sends a message to the bot.

Implements pipecat::Transport.

◆ send_ready_message()

void pipecat::SmallWebRTCTransport::send_ready_message ( const rtvi::Message &  message)
overridevirtual

Sends the client-ready message.

Implements pipecat::Transport.

◆ send_user_audio()

int32_t pipecat::SmallWebRTCTransport::send_user_audio ( const int16_t *  frames,
size_t  num_frames 
)
overridevirtual

Sends num_frames frames of 16-bit PCM user audio, in the format of the options. Returns the number of frames sent.

Send the audio continuously, as it's captured, and send silence while the user is muted: the bot needs it to tell when the user stops speaking. The audio is sent as is, without echo cancellation, and in mono, which is what the bot listens to.

Implements pipecat::Transport.

◆ set_start_bot_params()

void pipecat::SmallWebRTCTransport::set_start_bot_params ( const APIRequest &  request)
overridevirtual

Keeps the request that started the bot, to send the offer to the same server.

Reimplemented from pipecat::Transport.


The documentation for this class was generated from the following file: