Docs
⚙️ Configuration
Tools and Plugins
Intro

Tools and Plugins

Note: A new approach has been devised to revamp the handling of tools and plugins from scratch. The plan is to prioritize implementation for Assistants initially, followed by creating a new agent system that seamlessly integrates with various endpoints such as Anthropic, Google, and others.

Setup Instructions:

Azure AI Search

Google Search

OpenWeather

Stable Diffusion

Wolfram|Alpha

DALL-E

  • You just need an OpenAI key, and it’s made distinct from your main API key to make Chats but it can be the same one

Zapier

  • You need a Zapier account. Get your API key from here after you’ve made an account
    • Create allowed actions - Follow step 3 in this Start Here guide from Zapier
      • ⚠️ NOTE: zapier is known to be finicky with certain actions. I found that writing email drafts is probably the best use of it

Browser/Scraper

  • This is not to be confused with ‘browsing’ on chat.openai.com (which is technically a plugin suite or multiple plugins)
    • This plugin uses OpenAI embeddings so an OpenAI key is necessary, similar to DALL-E, and it’s made distinct from your main API key to make Chats but it can be the same one
    • This plugin will simply scrape html, and will not work with dynamic Javascript pages as that would require a more involved solution
    • A better solution for ‘browsing’ is planned but can’t guarantuee when
    • This plugin is best used in combination with google so it doesn’t hallucinate webpages to visit

SerpAPI

  • An alternative to Google search but not as performant in my opinion
    • You can get an API key here: https://serpapi.com/dashboard
    • For free tier, you are limited to 100 queries/month
    • With google, you are limited to 100/day for free, which is a better deal, and any after may cost you a few pennies

YouTube

  • Requires a YouTube API key from the Google Cloud Console
    • Create a new project or select an existing one
    • Enable the YouTube Data API v3
    • Create credentials (API key)
    • Set the API key in your environment variables as YOUTUBE_API_KEY

The YouTube tool provides four main operations:

  1. Search Videos

    • Search for YouTube videos with customizable results
    • Example: Search for “cooking pasta” videos, limit to 5 results

  2. Get Video Info

    • Retrieve detailed information about a specific video
    • Includes title, description, views, likes, and comment count

  3. Get Comments

    • Fetch comments from a specific video
    • Customizable number of comments (1-50, default: 10)
    • Returns author, text, and like count

  4. Get Transcript

    • Retrieve video transcripts
    • Attempts to fetch English transcript first
    • Finally tries any available language if neither is found
OpenaiAzure AI Search