📌Overview

Purpose: The MCP Server aims to facilitate fast interactions between AI agents and web content, enabling seamless browsing and information retrieval for language models.

Overview: The Model Context Protocol (MCP) Server for the RAG Web Browser Actor is designed to allow AI agents and large language models to perform web searches, scrape data, and extract information efficiently. Operating locally, it communicates with the RAG Web Browser Actor to enhance AI capabilities by enabling access to real-time web data.

Key Features:

Search Tool: Enables querying Google Search, scraping results, and returning cleaned content in Markdown format. It allows specification of search terms, the number of results, output formats, and scraping techniques.
Integration with AI Agents: Provides a straightforward API over standard input/output (stdio) for easy integration with AI Agents, supporting responsive web browsing and content extraction.

Model Context Protocol (MCP) Server for the RAG Web Browser Actor

This document details the implementation of an MCP server for the RAG Web Browser Actor. The Actor functions as a web browser for large language models (LLMs) and RAG pipelines.

What does this MCP server do?

This server is designed to provide fast responses to AI agents and LLMs, enabling interaction with the web and extraction of information from web pages. It operates locally and communicates with the RAG Web Browser Actor in Standby mode, sending search queries and receiving extracted web content.

Features

Perform web searches and scrape the top N URLs, returning cleaned content as Markdown.
Fetch a single URL and return its content as Markdown.

Components

Tools

search: Queries Google Search, scrapes the top N URLs, and returns their cleaned content as Markdown.
- query (string, required): Search term or URL.
- maxResults (number, optional): Maximum number of search results to scrape (default: 1).
- scrapingTool (string, optional): Select a scraping tool for extracting web pages. Options: 'browser-playwright' or 'raw-http' (default: 'raw-http').
- outputFormats (array, optional): Output formats. Options: 'text', 'markdown', 'html' (default: ['markdown']).
- requestTimeoutSecs (number, optional): Maximum time for the request in seconds (default: 40).

What is the Model Context Protocol?

The Model Context Protocol (MCP) enables AI applications to connect with external tools and data sources. More details can be found on the Model Context Protocol website or in the blog post What is MCP and why does it matter?.

How does the MCP Server integrate with AI Agents?

The MCP Server allows AI Agents to conduct web searches using the RAG Web Browser Actor. For further insights, refer to our blog post: What are AI Agents?, and explore Apify's Agents.

If you're interested in building your own AI agent on Apify, check out our step-by-step guide.

Related MCP servers and clients by Apify

Apify offers several MCP-related tools for ease of integration:

Server Options

This MCP Server – A local stdio-based server for direct integration with Claude Desktop.
RAG Web Browser Actor via SSE – Access the RAG Web Browser directly via Server-Sent Events.
MCP Server Actor – Provides AI agents with access to over 4,000 specialized Apify Actors.

Client Options

Tester MCP Client – A user-friendly UI for interacting with any SSE-based MCP server.

Configuration

Prerequisites

MacOS or Windows
Latest version of Claude Desktop or another MCP client
Node.js (v18 or higher)
Apify API Token (APIFY_TOKEN)

Install

To set up the server locally:

Clone the repository:

git clone git@github.com:apify/mcp-server-rag-web-browser.git

Navigate to the project directory and install dependencies:
```
cd mcp-server-rag-web-browser
npm install
```
Build the project:
```
npm run build
```

Configuration in Claude Desktop

Update Claude Desktop configuration:

Edit the following file:

On macOS: ~/Library/Application\ Support/Claude/claude_desktop_config.json
On Windows: %APPDATA%/Claude/claude_desktop_config.json

Include the following configuration:

"mcpServers": {
  "rag-web-browser": {
    "command": "npx",
    "args": [
      "@apify/mcp-server-rag-web-browser"
    ],
    "env": {
       "APIFY_TOKEN": "your-apify-api-token"
    }
  }
}

Restart Claude Desktop.

Examples

Example queries you can use with Claude:

What is an MCP server and how can it be used?
What is an LLM, and what are the recent news updates?
Find and analyze recent research papers about LLMs.

Development

Local client (stdio)

To test the server locally, use example_client_stdio.ts:

export APIFY_TOKEN=your-apify-api-token
node dist/example_client_stdio.js

Direct API Call

To call the RAG Web Browser Actor directly:

export APIFY_TOKEN=your-apify-api-token
node dist/example_call_web_browser.js

Debugging

For debugging, use the MCP Inspector:

Build the MCP server:
```
npm run build
```

Launch the MCP Inspector:

export APIFY_TOKEN=your-apify-api-token
npx @modelcontextprotocol/inspector node dist/index.js

The Inspector will display a URL for debugging.