test(fetch): add unit tests for fetch MCP server

Add comprehensive test coverage for the fetch server:
- TestGetRobotsTxtUrl: 6 tests for URL parsing
- TestExtractContentFromHtml: 3 tests for HTML-to-markdown conversion
- TestCheckMayAutonomouslyFetchUrl: 5 tests for robots.txt handling
- TestFetchUrl: 6 tests for URL fetching with various scenarios

Total: 20 tests covering:
- URL parsing and robots.txt URL generation
- HTML content extraction and markdown conversion
- robots.txt permission checking (401, 403, 404, allow/disallow)
- HTTP response handling (success, errors, raw mode)
- Proxy support

Also adds pytest and pytest-asyncio as dev dependencies.
This commit is contained in:
Piyush Tripathi
2026-01-28 10:20:49 -08:00
parent e6b0b0f5d3
commit 71d5d71a80
4 changed files with 1120 additions and 656 deletions

View File

@@ -33,4 +33,8 @@ requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.uv]
dev-dependencies = ["pyright>=1.1.389", "ruff>=0.7.3"]
dev-dependencies = ["pyright>=1.1.389", "ruff>=0.7.3", "pytest>=8.0.0", "pytest-asyncio>=0.21.0"]
[tool.pytest.ini_options]
testpaths = ["tests"]
asyncio_mode = "auto"