pipecat::rtvi Namespace Reference

Classes
struct	AboutClientData
	Information about the client, sent to the bot when connecting. More...
struct	BotLLMSearchResponseData
	Search results used by the bot's LLM. More...
struct	BotOutputData
struct	BotReadyData
	Sent by the bot when it's ready. More...
struct	ClientMessageData
	An app-defined message between the client and the bot. More...
struct	ErrorData
	An error. More...
struct	LLMFunctionCallInProgressData
	A function call is running. More...
struct	LLMFunctionCallResultData
	The result of a function call, sent to the bot. More...
struct	LLMFunctionCallStartedData
	The bot's LLM started a function call. More...
struct	LLMFunctionCallStoppedData
	A function call finished or was cancelled. More...
struct	LLMSearchOrigin
	Where search results come from, e.g. a website. More...
struct	LLMSearchResult
	A search result. More...
struct	Message
struct	MetricData
	A single measurement. More...
struct	MetricsData
	Performance metrics from the bot. More...
struct	SendTextOptions
	Options for sending text to the bot. More...
struct	SpokenProgressData
	How much of a text the bot has spoken. More...
struct	TextData
	Some text. More...
struct	TranscriptData
	What the user said. More...

Typedefs
using	UserLLMTextData = TextData
	User text sent to the bot's LLM.
using	BotLLMTextData = TextData
	Text from the bot's LLM, as it streams.
using	BotTTSTextData = TextData
	Text the bot is speaking.

Enumerations
enum class	MessageType {   ClientReady , DisconnectBot , ClientMessage , SendText ,   DTMF , LLMFunctionCallResult , BotReady , Error ,   ErrorResponse , ServerMessage , ServerResponse , Metrics ,   UserStartedSpeaking , UserStoppedSpeaking , BotStartedSpeaking , BotStoppedSpeaking ,   BotInterrupted , UserMuteStarted , UserMuteStopped , UserTranscription ,   UserLLMText , BotOutput , BotLLMText , BotLLMStarted ,   BotLLMStopped , BotLLMSearchResponse , BotTTSText , BotTTSStarted ,   BotTTSStopped , LLMFunctionCallStarted , LLMFunctionCallInProgress , LLMFunctionCallStopped }
	Types of RTVI messages. More...

Functions
const char *	to_string (MessageType type)
	The RTVI name of a message type, e.g. "bot-ready".
std::optional< MessageType >	parse_message_type (const std::string &type)
	The message type with an RTVI name, or std::nullopt if it's unknown.
AboutClientData	default_about_client ()
	Information about this library and the platform it runs on.

Variables
constexpr const char *	PROTOCOL_VERSION = "2.1.0"
	The RTVI protocol version this library speaks.
constexpr const char *	MESSAGE_LABEL = "rtvi-ai"
	The label of every RTVI message.

Detailed Description

RTVI messages and their data.

RTVI is the protocol clients and Pipecat bots use to talk to each other. PipecatClient sends messages for you and passes the data of the messages it receives to PipecatClientCallbacks.

Missing fields in messages from the bot get default values, or std::nullopt if they're optional. All the data types convert to and from nlohmann::json, e.g. message.data.get<BotOutputData>().

Enumeration Type Documentation

MessageType

enum class pipecat::rtvi::MessageType

strong

Types of RTVI messages.

Enumerator
ClientReady	The client is ready.
DisconnectBot	Asks the bot to leave.
ClientMessage	An app-defined message to the bot.
SendText	Text for the bot's LLM.
DTMF	DTMF keys.
LLMFunctionCallResult	The result of a function call.
BotReady	The bot is ready.
Error	An error in the bot.
ErrorResponse	The bot couldn't handle a message.
ServerMessage	An app-defined message from the bot.
ServerResponse	The bot's answer to a request.
Metrics	Performance metrics.
UserStartedSpeaking	The user started speaking.
UserStoppedSpeaking	The user stopped speaking.
BotStartedSpeaking	The bot started speaking.
BotStoppedSpeaking	The bot stopped speaking.
BotInterrupted	The bot was interrupted.
UserMuteStarted	The bot started ignoring user audio.
UserMuteStopped	The bot stopped ignoring user audio.
UserTranscription	What the user said.
UserLLMText	User text sent to the bot's LLM.
BotOutput	What the bot says or outputs.
BotLLMText	Text from the bot's LLM, as it streams.
BotLLMStarted	The bot's LLM started responding.
BotLLMStopped	The bot's LLM finished responding.
BotLLMSearchResponse	Search results used by the bot's LLM.
BotTTSText	Text the bot is speaking.
BotTTSStarted	The bot's text-to-speech started.
BotTTSStopped	The bot's text-to-speech stopped.
LLMFunctionCallStarted	The bot's LLM started a function call.
LLMFunctionCallInProgress	A function call is running.
LLMFunctionCallStopped	A function call finished.