MCP HubMCP Hub
nbonamy

witsy

by: nbonamy

Witsy: desktop AI assistant

977created 25/04/2024
Visit
AI
desktop

📌Overview

Purpose:
Witsy is designed to provide a powerful, customizable desktop AI assistant that integrates with major LLM providers and enhances productivity across any desktop environment.

Overview:
Witsy is a BYOK (Bring Your Own Keys) AI assistant application for desktop. It supports a wide array of leading AI models and services, allowing users to utilize both cloud-based and locally running models. Witsy facilitates seamless integration into desktop workflows, enabling content generation, chat, transcription, document interaction, and process automation directly within any application.

Key Features:

  • Multi-Provider & Local Model Support:
    Connects to major LLM providers (OpenAI, Anthropic, Google, xAI, MistralAI, Azure, DeepSeek, Ollama, and more), and allows local model usage via Ollama for privacy and cost-saving.

  • Integrated Productivity Tools:
    Offers features like "Prompt Anywhere" for cross-application AI content generation, AI commands on highlighted text, scratchpad for collaborative content creation, chat with documents (RAG), transcription/dictation, voice mode, and long-term memory plugins for enhanced relevance.

Witsy: Desktop AI Assistant

Downloads

Download Witsy from witsyai.com or from the releases page.

Overview

Witsy is a BYOK (Bring Your Own Keys) AI application for desktop. To use it, you need API keys from LLM providers you want to access, or you can run local models via Ollama.

Features

  • Support for OpenAI, Ollama, Anthropic, MistralAI, Google, xAI, Azure, OpenRouter, DeepSeek, Groq, Cerebras, and more
  • Connect other providers through OpenAI compatibility layer
  • Chat completion with vision models (describe images)
  • Text-to-image and text-to-video with OpenAI, Google, xAI, Replicate, fal.ai, HuggingFace
  • Image-to-image editing and image-to-video with Google, Replicate, fal.ai
  • Interactive Scratchpad for content creation
  • Prompt Anywhere: generate content in any application
  • AI Commands: run AI on selected text in any application
  • Experts: specialized prompts for specific tasks or topics
  • LLM plugins: extend LLM capabilities (e.g., run Python code, search Internet)
  • Long-term memory plugin to retain conversation context
  • Read aloud for assistant messages and other text (requires OpenAI or ElevenLabs API key)
  • Chat with local files and documents (RAG)
  • Speech-to-text transcription/dictation
  • Realtime chat and voice mode
  • Support for Anthropic Computer Use protocol
  • Local chat history with automatic titles
  • Code formatting and clipboard integration
  • Conversation PDF export
  • Image copy and download

Usage Highlights

Prompt Anywhere

  • Works in any editable content field in any application
  • Shortcut: Shift+Control+Space
  • Enter a prompt and Witsy generates and enters the text in your application
  • On Mac, you can define experts to auto-select based on the active application

AI Commands

  • Select text in any application
  • Shortcut: Alt+Control+Space
  • Pick or create quick actions powered by LLMs
  • Customize your own commands with preferred prompts

Inspired by https://the.fibery.io/@public/Public_Roadmap/Roadmap_Item/AI-Assistant-via-ChatGPT-API-170.

Experts

Based on https://github.com/f/awesome-chatgpt-prompts.

Scratchpad

Watch overview: https://www.youtube.com/watch?v=czcSbG2H-wg

Chat with Documents (RAG)

  • Attach a local document repository to a chat
  • Witsy searches your files and provides relevant context to the model
  • Set up document repositories and select them as needed

Transcription / Dictation (Speech-to-Text)

  • Record audio via microphone and transcribe to text
  • Use OpenAI Whisper API (online) or local Whisper model
  • Copy transcribed text to clipboard or insert into originating application

Anthropic Computer Use

Watch overview: https://www.youtube.com/watch?v=vixl7I07hBk

Setup

Download Witsy from witsyai.com, from the releases page, or build it yourself:

npm install
npm start

Prerequisites

Development

To build and run Witsy locally:

npm install
npm start

Roadmap

  • Proper database (SQLite3) storage

Recently Completed

  • Azure AI support
  • Brave Search plugin
  • User-input models for embeddings
  • User-defined parameters for custom engines
  • Direct speech-to-text checkbox
  • Quick access buttons on home
  • fal.ai support (speech-to-text, text-to-image, text-to-video)
  • Debug console
  • Design Studio
  • i18n support
  • Mermaid diagram rendering
  • Model Context Protocol
  • Local Web Search
  • Model defaults
  • Speech-to-text language option
  • Favorite models
  • ElevenLabs Text-to-Speech
  • Custom OpenAI-compatible engines
  • Long-term memory plugin
  • OpenRouter support
  • DeepSeek support
  • Folder mode for files
  • Fork chat with optional LLM switch
  • Realtime chat
  • Video generation with Replicate
  • Together.ai compatibility
  • Gemini 2.0 Flash support
  • Groq LLama 3.3 support
  • xAI Grok Vision model support
  • Ollama function-calling
  • Hugging Face image generation
  • Redesigned Prompt window
  • Anthropic Computer Use support
  • Auto-update improvements
  • Dark mode
  • Conversation mode
  • Function calling for Google and Anthropic
  • Enhanced Scratchpad
  • Dictation: OpenAI Whisper + Whisper WebGPU
  • Auto-select expert based on active app (Mac only)
  • Cerebras support
  • Local files RAG
  • PDF export for chats
  • Editable Experts (prompts)
  • Read aloud feature
  • Import/Export commands
  • Anthropic Sonnet 3.5 support
  • Customizable base URLs for Ollama and OpenAI
  • DALL-E integration
  • Google Gemini API support
  • Prompt anywhere
  • Cancel commands
  • GPT-4o model support
  • Attachments (TXT, PDF, DOCX, PPTX, XLSX)
  • MistralAI function calling
  • Model and command management improvements
  • Shortcuts, menu, search, and history features
  • Copy code button, image tools, and many other productivity improvements

For more information and updates, visit witsyai.com or the GitHub repository.