web-browser-mcp-server
by: blazickjp
A Minimum Control Program (MCP) server implementation for web browsing capabilities using BeautifulSoup4
πOverview
Purpose: Enable AI assistants to browse the web and extract content through a Message Control Protocol (MCP) interface.
Overview: The Web Browser MCP Server enhances AI models' capabilities by allowing them to access and interpret web content. It ensures efficient content extraction and robust error handling, making it an essential tool for AI-driven applications.
Key Features:
-
Smart Content Extraction: Utilizes CSS selectors to precisely target and extract required data from web pages, enhancing data relevance.
-
Lightning Fast Performance: Designed with asynchronous processing for optimal speed, ensuring quick responses during content fetching.
-
Rich Metadata Capture: Gathers comprehensive information including titles, links, and structured content for in-depth analysis.
-
Robust & Reliable Error Handling: Incorporates built-in error management and timeout control for dependable operation.
-
Cross-Platform Compatibility: Functions seamlessly on any platform where Python is supported, ensuring broad usability.
Web Browser MCP Server
β¨ Features
Enable AI assistants to browse and extract content from the web through a simple MCP interface.
The Web Browser MCP Server provides AI models with the ability to browse websites, extract content, and understand web pages through the Message Control Protocol (MCP). It enables smart content extraction with CSS selectors and robust error handling.
π€ Contribute β’ π Report Bug
β¨ Core Features
- π― Smart Content Extraction: Target exactly what you need with CSS selectors
- β‘ Lightning Fast: Built with async processing for optimal performance
- π Rich Metadata: Capture titles, links, and structured content
- π‘οΈ Robust & Reliable: Built-in error handling and timeout management
- π Cross-Platform: Works everywhere Python runs
π Quick Start
Installing via Smithery
To install Web Browser Server for Claude Desktop automatically via Smithery:
npx -y @smithery/cli install web-browser-mcp-server --client claude
Installing Manually
Install using uv:
uv tool install web-browser-mcp-server
For development:
# Clone and set up development environment
git clone https://github.com/blazickjp/web-browser-mcp-server.git
cd web-browser-mcp-server
# Create and activate virtual environment
uv venv
source .venv/bin/activate
# Install with test dependencies
uv pip install -e ".[test]"
π MCP Integration
Add this configuration to your MCP client config file:
{
"mcpServers": {
"web-browser-mcp-server": {
"command": "uv",
"args": [
"tool",
"run",
"web-browser-mcp-server"
],
"env": {
"REQUEST_TIMEOUT": "30"
}
}
}
}
For Development:
{
"mcpServers": {
"web-browser-mcp-server": {
"command": "uv",
"args": [
"--directory",
"path/to/cloned/web-browser-mcp-server",
"run",
"web-browser-mcp-server"
],
"env": {
"REQUEST_TIMEOUT": "30"
}
}
}
}
π‘ Available Tools
The server provides a powerful web browsing tool:
browse_webpage
Browse and extract content from web pages with optional CSS selectors:
# Basic webpage fetch
result = await call_tool("browse_webpage", {
"url": "https://example.com"
})
# Target specific content with CSS selectors
result = await call_tool("browse_webpage", {
"url": "https://example.com",
"selectors": {
"headlines": "h1, h2",
"main_content": "article.content",
"navigation": "nav a"
}
})
βοΈ Configuration
Configure through environment variables:
Variable | Purpose | Default |
---|---|---|
REQUEST_TIMEOUT | Webpage request timeout in seconds | 30 |
π§ͺ Testing
Run the test suite:
python -m pytest
π License
Released under the MIT License. See the LICENSE file for details.
Made with β€οΈ by the Pear Labs Team