MCP Directory

pragmar/mcp-server-webcrawl

GitHub
Web Utilities

Advanced search and retrieval for web crawler data. Supports WARC, wget, Katana, SiteOne, and InterroBot crawlers.

Overview

Direct install

Run the underlying MCP server directly — without Agent-CoreX routing.

npx -y mcp-server-webcrawl

Use with Agent-CoreX

Agent-CoreX intelligently routes which tools from pragmar/mcp-server-webcrawl your AI agent actually needs per request — reducing token usage and cost.

agent-corex.json

{
  "mcpServers": {
    "mcp-server-webcrawl": {
      "command": "npx",
      "args": [
        "-y",
        "mcp-server-webcrawl"
      ]
    }
  }
}

Route pragmar/mcp-server-webcrawl intelligently

Connect in 2 minutes. Only pay for the tools your agent actually uses.

Get started free Browse all servers

More Web Utilities servers

Fetch

Web content fetching and conversion for efficient LLM usage; converts HTML to markdown

Brave Search

Web search using Brave Search API

Puppeteer

Browser automation and web scraping with headless Chrome

Installation Guide

Use with Agent-CoreX

Recommended: intelligent tool routing, lower costs

uvx agent-corex mcp add mcp-server-webcrawl

Agent-CoreX selects only the tools your agent needs, cutting token usage by up to 60%.

Direct Install

Use pragmar/mcp-server-webcrawl directly without routing

npx -y mcp-server-webcrawl

New to Agent-CoreX? View setup guide →

pragmar/mcp-server-webcrawl MCP Server — Agent-CoreX