📌Overview

Purpose: To create a seamless connection between local Large Language Models (LLMs) and Model Context Protocol (MCP) servers, enabling rich functionalities for open-source models.

Overview: The MCP-LLM Bridge allows local LLMs to leverage advanced capabilities similar to those utilized by Claude, such as filesystem operations and web interactions. It facilitates interaction through a translation layer between the LLM's outputs and the MCP's JSON-RPC protocol, making various tools accessible to any Ollama-compatible model.

Key Features:

• Multi-MCP Support: Enables dynamic routing to multiple MCPs for diverse task handling, including filesystem operations, web searches, and email management.

• Structured Output Validation: Ensures that all tool calls from the LLM are validated against expected formats, enhancing reliability and reducing errors.

• Automatic Tool Detection: Smart recognition of user prompts allows for efficient and intuitive operation routing to the correct tool based on context and keywords.

• Robust Process Management: Ensures efficient handling of interactions with the Ollama LLM, allowing for seamless operation and communication.

• Detailed Logging and Error Handling: Provides comprehensive logs for tracking operations and managing errors effectively, promoting easier debugging and maintenance.

---

MCP-LLM Bridge

A TypeScript implementation that connects local LLMs (via Ollama) to Model Context Protocol (MCP) servers. This bridge enables open-source models to use powerful tools similar to Claude, facilitating local AI assistants.

Overview

This project bridges local Large Language Models with MCP servers to provide capabilities such as:

• Filesystem operations
• Web search
• GitHub interactions
• Google Drive & Gmail integration
• Memory/storage
• Image generation

The bridge translates LLM outputs to MCP's JSON-RPC protocol, allowing any Ollama-compatible model to utilize these tools.

Current Setup

• LLM: Using Qwen 2.5 7B (qwen2.5-coder:7b-instruct) through Ollama
• MCPs:
  • Filesystem operations
  • Brave Search
  • GitHub
  • Memory
  • Flux image generation
  • Gmail & Drive

Architecture

• Bridge: Manages tool registration and execution
• LLM Client: Handles Ollama interactions and formats tool calls
• MCP Client: Manages MCP server connections and JSON-RPC communication
• Tool Router: Routes requests to appropriate MCP based on tool type

Key Features

• Multi-MCP support with dynamic tool routing
• Output validation for tool calls
• Automatic tool detection from user prompts
• Robust process management for Ollama
• Detailed logging and error handling

Setup

1. Install Ollama and required model:

   ollama pull qwen2.5-coder:7b-instruct
   

2. Install MCP servers:

   npm install -g @modelcontextprotocol/server-filesystem
   npm install -g @modelcontextprotocol/server-brave-search
   npm install -g @modelcontextprotocol/server-github
   npm install -g @modelcontextprotocol/server-memory
   npm install -g @patruff/server-flux
   npm install -g @patruff/server-gmail-drive
   

3. Configure credentials:

   • Set BRAVE_API_KEY for Brave Search
   • Set GITHUB_PERSONAL_ACCESS_TOKEN for GitHub
   • Set REPLICATE_API_TOKEN for Flux
   • Run Gmail/Drive MCP auth.

Configuration

The bridge is configured through bridge_config.json, including:

• MCP server definitions
• LLM settings (model, temperature, etc.)
• Tool permissions and paths

Example:

{
  "mcpServers": {
    "filesystem": {
      "command": "node",
      "args": ["path/to/server-filesystem/dist/index.js"],
      "allowedDirectory": "workspace/path"
    }
  },
  "llm": {
    "model": "qwen2.5-coder:7b-instruct",
    "baseUrl": "http://localhost:11434"
  }
}

Usage

1. Start the bridge:

   npm run start
   

2. Available commands:

   • list-tools: Show available tools
   • Send prompts to the LLM
   • quit: Exit the program

Technical Details

Tool Detection

The bridge includes smart tool detection based on user input for various operations.

Response Processing

Responses are processed through multiple stages involving validation, routing, and formatting for user display.

Extended Capabilities

This bridge brings Claude's tool capabilities to local models, enabling:

• Filesystem manipulation
• Web search
• Email and document management
• Code and GitHub interactions
• Image generation
• Persistent memory

Future Improvements

• Add support for more MCPs
• Implement parallel tool execution
• Add streaming responses
• Enhance error recovery
• Add conversation memory
• Support more Ollama models

Related Projects

This bridge integrates with the broader Claude ecosystem, enhancing the functionalities of local AI assistants.