Shared Endpoint Settings

This page describes the shared configuration settings for all endpoints. The settings highlighted here are available to all configurations under the “Endpoints” field unless noted otherwise.

Example Configuration

endpoints:
  openAI:
    streamRate: 25
  anthropic:
    streamRate: 25
  google:
    streamRate: 1
  azureOpenAI:
    streamRate: 20
  assistants:
    streamRate: 30
  azureAssistants:
    streamRate: 30
  # the `all` setting would override all the above values, making them unnecessary to be set
  all:
    streamRate: 20

streamRate

Key:

Key	Type	Description	Example
streamRate	Number	The rate at which data is streamed from the endpoint. Useful for controlling the pace of streaming data.	streamRate: 25

Default: 1

Allows for streaming data at the fastest rate possible while allowing the system to wait for the next tick

Notes:

The all setting would override all individual endpoint values, making those specific settings unnecessary if used.
The value can be customized for each endpoint or set globally using the all key.
Recommended values are between 25-40 for a smooth streaming experience
Using a higher rate is a must when serving the app to many users at scale.

Endpoint Settings

Actions Intro