📌Overview

Purpose: To create a seamless connection between local Large Language Models (LLMs) and Model Context Protocol (MCP) servers, enabling rich functionalities for open-source models.

Overview: The MCP-LLM Bridge allows local LLMs to leverage advanced capabilities similar to those utilized by Claude, such as filesystem operations and web interactions. It facilitates interaction through a translation layer between the LLM's outputs and the MCP's JSON-RPC protocol, making various tools accessible to any Ollama-compatible model.

Key Features:

Multi-MCP Support: Enables dynamic routing to multiple MCPs for diverse task handling, including filesystem operations, web searches, and email management.
Structured Output Validation: Ensures that all tool calls from the LLM are validated against expected formats, enhancing reliability and reducing errors.
Automatic Tool Detection: Smart recognition of user prompts allows for efficient and intuitive operation routing to the correct tool based on context and keywords.
Robust Process Management: Ensures efficient handling of interactions with the Ollama LLM, allowing for seamless operation and communication.
Detailed Logging and Error Handling: Provides comprehensive logs for tracking operations and managing errors effectively, promoting easier debugging and maintenance.

MCP-LLM Bridge

A TypeScript implementation that connects local LLMs (via Ollama) to Model Context Protocol (MCP) servers. This bridge allows open-source models to use the same tools and capabilities as Claude, enabling powerful local AI assistants.

Overview

This project bridges local Large Language Models with MCP servers providing capabilities such as:

Filesystem operations
Brave web search
GitHub interactions
Google Drive & Gmail integration
Memory/storage
Image generation with Flux

The bridge translates between LLM outputs and the MCP's JSON-RPC protocol, allowing any Ollama-compatible model to use these tools like Claude.

Current Setup

LLM: Qwen 2.5 7B (qwen2.5-coder:7b-instruct) through Ollama
MCPs:
- Filesystem operations (@modelcontextprotocol/server-filesystem)
- Brave Search (@modelcontextprotocol/server-brave-search)
- GitHub (@modelcontextprotocol/server-github)
- Memory (@modelcontextprotocol/server-memory)
- Flux image generation (@patruff/server-flux)
- Gmail & Drive (@patruff/server-gmail-drive)

Architecture

Bridge: Manages tool registration and execution
LLM Client: Handles Ollama interactions and formats tool calls
MCP Client: Manages MCP server connections and JSON-RPC communication
Tool Router: Routes requests to appropriate MCP based on tool type

Key Features

Multi-MCP support with dynamic tool routing
Structured output validation for tool calls
Automatic tool detection from user prompts
Robust process management for Ollama
Detailed logging and error handling

Setup

Install Ollama and required model:
```
ollama pull qwen2.5-coder:7b-instruct
```

Install MCP servers:

npm install -g @modelcontextprotocol/server-filesystem
npm install -g @modelcontextprotocol/server-brave-search
npm install -g @modelcontextprotocol/server-github
npm install -g @modelcontextprotocol/server-memory
npm install -g @patruff/server-flux
npm install -g @patruff/server-gmail-drive

Configure credentials:
- Set BRAVE_API_KEY for Brave Search
- Set GITHUB_PERSONAL_ACCESS_TOKEN for GitHub
- Set REPLICATE_API_TOKEN for Flux
- Run Gmail/Drive MCP auth:
```
node path/to/gmail-drive/index.js auth
```

Configuration

The bridge is configured through bridge_config.json:

MCP server definitions
LLM settings (model, temperature, etc.)
Tool permissions and paths

Example:

{
  "mcpServers": {
    "filesystem": {
      "command": "node",
      "args": ["path/to/server-filesystem/dist/index.js"],
      "allowedDirectory": "workspace/path"
    }
  },
  "llm": {
    "model": "qwen2.5-coder:7b-instruct",
    "baseUrl": "http://localhost:11434"
  }
}

Usage

Start the bridge:
```
npm run start
```
Available commands:
- list-tools: Show available tools
- Regular text: Send prompts to the LLM
- quit: Exit the program

Example interactions:

> Search the web for "latest TypeScript features"
[Uses Brave Search MCP to find results]

> Create a new folder called "project-docs"
[Uses Filesystem MCP to create directory]

> Send an email to user@example.com
[Uses Gmail MCP to compose and send email]

Technical Details

Tool Detection

The bridge includes smart tool detection based on user input:

Email operations: Detected by email addresses and keywords
Drive operations: Detected by file/folder keywords
Search operations: Contextually routed to appropriate search tool

Response Processing

Response processing stages:

LLM generates structured tool calls
Bridge validates and routes to appropriate MCP
MCP executes operation and returns result
Bridge formats response for user

Extended Capabilities

This bridge brings Claude's tool capabilities to local models with:

Filesystem manipulation
Web search and research
Email and document management
Code and GitHub interactions
Image generation
Persistent memory

All running locally with open-source models.

Future Improvements

Support for more MCPs
Parallel tool execution
Streaming responses
Enhanced error recovery
Conversation memory
Support for more Ollama models

Related Projects

This bridge integrates with the broader Claude ecosystem, Model Context Protocol (MCP), Ollama Project, and various MCP server implementations, creating a powerful local AI assistant capable of matching many Claude capabilities on your hardware.

ollama-mcp-bridge