test(fetch): add unit tests for fetch MCP server

Browse Source

Add comprehensive test coverage for the fetch server:
- TestGetRobotsTxtUrl: 6 tests for URL parsing
- TestExtractContentFromHtml: 3 tests for HTML-to-markdown conversion
- TestCheckMayAutonomouslyFetchUrl: 5 tests for robots.txt handling
- TestFetchUrl: 6 tests for URL fetching with various scenarios

Total: 20 tests covering:
- URL parsing and robots.txt URL generation
- HTML content extraction and markdown conversion
- robots.txt permission checking (401, 403, 404, allow/disallow)
- HTTP response handling (success, errors, raw mode)
- Proxy support

Also adds pytest and pytest-asyncio as dev dependencies.

...

This commit is contained in:

Piyush Tripathi

2026-01-28 10:20:49 -08:00

parent e6b0b0f5d3

commit 71d5d71a80

4 changed files with 1120 additions and 656 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

1444

src/fetch/uv.lock generated

File diff suppressed because it is too large Load Diff