fix(url_safety): dial a declared local-proxy fake-ip block instead of blocking it

A TUN proxy in fake-ip mode (Mihomo/Clash `fake-ip`, Surge enhanced) answers DNS with an
address from its own block — `198.18.0.0/15` by default — for every name outside its filter.
The SSRF guard resolves names with the local resolver and reads that sentinel as a private
destination, so on such a host every fetch fails before the request leaves the machine:
`web_extract`, gateway media downloads (`is_safe_url` has ~20 call sites, including the
Feishu/WeCom/Telegram/Slack/Discord attachment paths) and the browser relay all report
"URL targets a private or internal network address".

The existing hostname allowlist (`_TRUSTED_PRIVATE_IP_HOSTS`, the QQ multimedia case) does not
generalise to this: the sentinel is a property of the host's resolver, not of one name.

Fix: `security.fake_ip_ranges` declares the CIDR blocks the local proxy owns. Answers inside a
declared block are dialable even with private-IP blocking on. Empty by default, so no existing
host changes behaviour. Loopback, RFC 1918, link-local, CGNAT and the cloud-metadata floor are
untouched. Pre-flight and connect-time checks both go through `_resolved_ip_block_reason`, so one
exemption covers the class instead of the ~20 call sites.

Tests: `TestDeclaredFakeIpSentinelRanges` in tests/tools/test_url_safety.py — red on the unfixed
module (declared sentinel rejected, "Blocked request to private/internal address during connect:
example.com -> 198.18.0.55"), green after. The pre-existing benchmark/QQ hostname tests are
unchanged and still pass (66 passed).

Docs: website/docs/user-guide/security.md + zh-Hans translation. The key is registered in
hermes_cli/config_defaults.py so `hermes config set security.fake_ip_ranges` is not flagged as
unknown.
This commit is contained in:
AYin-Z
2026-09-16 10:29:00 +08:00
committed by Teknium
parent 9f48f4ed04
commit 0f870069ea
5 changed files with 153 additions and 1 deletions

View File

@@ -1624,6 +1624,10 @@ DEFAULT_CONFIG = {
"personalities": {},
"security": { # Security: pre-exec scanning via tirith plus related guards.
"allow_private_urls": False, # allow requests to private/internal IPs (OpenWrt, VPNs)
# CIDR blocks a local TUN proxy answers DNS with (Mihomo/Clash fake-ip, Surge enhanced).
# Answers inside these blocks are the proxy's sentinels, not internal hosts, so the guard
# dials them instead of rejecting them as private. Empty = normal private-address verdict.
"fake_ip_ranges": [],
"redact_secrets": True,
# Persisted acknowledgement for unattended model overrides whose tier lets the vendor train
# on prompts. The startup guard still warns every run; cost guards are unaffected.

View File

@@ -19,6 +19,7 @@ from tools.url_safety import (
_is_blocked_ip,
_global_allow_private_urls,
_reset_allow_private_cache,
_reset_fake_ip_cache,
)
import ipaddress
@@ -496,3 +497,44 @@ class TestRedirectTargetFromResponse:
next_request=_FakeNextRequest("http://10.0.0.1/meta"),
)
assert redirect_target_from_response(resp) == "http://10.0.0.1/meta"
class TestDeclaredFakeIpSentinelRanges:
"""A local TUN proxy answers DNS with a fake-ip block — declared in ``security.fake_ip_ranges``.
On such a host every name outside the proxy's filter resolves into that block, so keying the
guard on the resolver's answer blocked every outbound fetch (web_extract, platform attachment
downloads, the browser relay) while the request never reached the network at all. The exemption
is per-host opt-in and scoped to the declared block — an undeclared host keeps the ordinary
private-address verdict (see TestProxyEnvironmentDnsDelegation).
"""
@pytest.fixture
def declared(self, monkeypatch):
monkeypatch.setattr(
"hermes_cli.config.read_raw_config",
lambda: {"security": {"fake_ip_ranges": ["198.18.0.0/15"]}},
)
_reset_fake_ip_cache()
yield
_reset_fake_ip_cache()
def test_undeclared_host_is_unaffected(self):
with _resolves_to("198.18.0.23"):
assert is_safe_url("https://example.com/file.jpg") is False
def test_declared_sentinel_is_dialable_with_private_blocking_on(self, declared):
with _resolves_to("198.18.1.125"):
assert is_safe_url("https://example.com/") is True
def test_declared_sentinel_passes_the_connect_time_check_too(self, declared):
with _resolves_to("198.18.0.55"):
assert _resolved_http_connect_ips("example.com", 443, "https") == ["198.18.0.55"]
def test_declaration_does_not_excuse_real_private_answers(self, declared):
with _resolves_to("192.168.99.99"):
assert is_safe_url("https://example.com/") is False
def test_metadata_floor_outranks_the_declaration(self, declared):
with _resolves_to("169.254.169.254"):
assert is_safe_url("http://example.com/") is False

View File

@@ -1,7 +1,10 @@
"""URL safety checks — blocks requests to private/internal network addresses (SSRF).
``security.allow_private_urls: true`` disables private-IP blocking (DNS that resolves public
names to private ranges); cloud metadata hostnames/IPs are **always** blocked. DNS rebinding
names to private ranges); cloud metadata hostnames/IPs are **always** blocked. A local TUN proxy
that answers DNS with a fake-ip block (Mihomo/Clash fake-ip, Surge enhanced) declares that block
in ``security.fake_ip_ranges`` so its sentinel answers are dialable instead of looking private;
the list is empty by default, so the sentinel stays blocked for everyone else. DNS rebinding
(TOCTOU) is closed for Hermes-owned httpx paths by ``create_ssrf_safe_[async_]client()``, which
re-apply the policy at TCP connect and dial the validated IP while preserving Host/SNI. Redirect
bypass is mitigated by response hooks re-validating each target (``redirect_target_from_response``).
@@ -120,6 +123,7 @@ _CGNAT_NETWORK = ipaddress.ip_network("100.64.0.0/10")
# Global toggle cache (process lifetime; see _global_allow_private_urls).
_allow_private_resolved, _cached_allow_private = False, False
_fake_ip_resolved, _cached_fake_ip_ranges = False, ()
def _global_allow_private_urls() -> bool:
@@ -160,6 +164,50 @@ def _reset_allow_private_cache() -> None:
_allow_private_resolved = _cached_allow_private = False
def _resolve_fake_ip_ranges() -> tuple:
"""CIDR blocks this host's local proxy answers DNS with (``security.fake_ip_ranges``).
A TUN proxy in fake-ip mode answers every non-filtered name with an address from its own
block; that answer is the proxy's sentinel, not an internal host's, so treating it as a
private target blocks every outbound fetch on such a host (web_extract, platform attachment
downloads, the browser relay). Empty by default: no host gets the exemption unless it
declares one, and the sentinel range keeps the ordinary private-address verdict otherwise.
"""
try:
from hermes_cli.config import read_raw_config
block = read_raw_config().get("security", {})
raw = block.get("fake_ip_ranges") if isinstance(block, dict) else None
if isinstance(raw, str):
raw = [part.strip() for part in raw.split(",")]
if not isinstance(raw, (list, tuple)):
return ()
networks = []
for entry in raw:
if not str(entry).strip():
continue
try:
networks.append(ipaddress.ip_network(str(entry).strip(), strict=False))
except ValueError:
logger.warning("Ignoring unparseable security.fake_ip_ranges entry: %r", entry)
return tuple(networks)
except Exception:
return () # config unavailable (tests, early import) — keep the secure default
def _global_fake_ip_ranges() -> tuple:
"""Process-lifetime cache, same shape as the allow_private toggle."""
global _fake_ip_resolved, _cached_fake_ip_ranges
if not _fake_ip_resolved:
_fake_ip_resolved, _cached_fake_ip_ranges = True, _resolve_fake_ip_ranges()
return _cached_fake_ip_ranges
def _reset_fake_ip_cache() -> None:
"""Reset the cached sentinel ranges — only for tests."""
global _fake_ip_resolved, _cached_fake_ip_ranges
_fake_ip_resolved, _cached_fake_ip_ranges = False, ()
def _normalize_hostname(host: Optional[str]) -> str:
return (host or "").strip().lower().rstrip(".")
@@ -189,6 +237,21 @@ def _is_always_blocked_ip(ip: _IPAddress) -> bool:
return ip in _ALWAYS_BLOCKED_IPS or any(ip in net for net in _ALWAYS_BLOCKED_NETWORKS)
def _is_local_proxy_sentinel(ip: _IPAddress) -> bool:
"""True when *ip* is in a fake-ip block this host declared in ``security.fake_ip_ranges``.
The dial still goes to the local proxy, which resolves and connects to the real target, so
exempting a declared block grants no reach an attacker lacks through the proxy's own DNS.
Undeclared ranges keep the ordinary private-address verdict.
"""
networks = _global_fake_ip_ranges()
if not networks:
return False
if isinstance(ip, ipaddress.IPv6Address) and ip.ipv4_mapped is not None:
ip = ip.ipv4_mapped
return any(ip in net for net in networks)
def _is_blocked_ip(ip: _IPAddress) -> bool:
"""Return True if the IP should be blocked for SSRF protection."""
# IPv4-mapped IPv6 (``::ffff:x.x.x.x``) is classified by its embedded IPv4.
@@ -248,6 +311,10 @@ def _resolved_ip_block_reason(ip: _IPAddress, allow_private: bool) -> Optional[s
if _is_always_blocked_ip(ip):
return "cloud metadata address"
if not allow_private and _is_blocked_ip(ip):
# A fake-ip sentinel is the local proxy's own address, not an internal target — dialable either way.
if _is_local_proxy_sentinel(ip):
logger.debug("Allowing local-proxy fake-ip sentinel address: %s", ip)
return None
return "private/internal address"
return None

View File

@@ -746,6 +746,27 @@ When on, web tools, the browser, vision URL fetches, and gateway media downloads
The host-substring guard (which blocks lookalike Unicode domain tricks even when the underlying IP is public) stays on regardless of this setting.
#### Local proxy fake-ip ranges
A TUN proxy in fake-ip mode (Mihomo/Clash `fake-ip`, Surge enhanced mode) answers DNS with an
address from its own block — `198.18.0.0/15` (RFC 2544 benchmarking) by default — for every name
outside its filter. Those answers are the proxy's sentinel, not an internal host, so the
private-IP guard otherwise rejects every outbound fetch on such a host: `web_extract`, platform
attachment downloads and the browser relay all fail with *URL targets a private or internal
network address* while the request never reaches the network. Declare the block to let the
sentinel through:
```yaml
security:
fake_ip_ranges:
- 198.18.0.0/15
```
Empty by default, and narrower than `allow_private_urls`: only the declared blocks get the
exemption, they should be ranges the local proxy owns (the dial still goes to the proxy, which
resolves the real target itself), and loopback, RFC 1918, link-local, CGNAT and cloud-metadata
destinations stay blocked.
### Tirith Pre-Exec Security Scanning
Hermes integrates [tirith](https://github.com/sheeki03/tirith) for content-level command scanning before execution. Tirith detects threats that pattern matching alone misses:

View File

@@ -523,6 +523,24 @@ security:
主机子字符串防护(即使底层 IP 是公共的,也能阻止 Unicode 同形字域名欺骗)无论此设置如何均保持开启。
#### 本地代理的 fake-ip 地址段
以 fake-ip 模式工作的 TUN 代理(Mihomo/Clash `fake-ip`、Surge 增强模式)会对不在其过滤器内的
每个域名返回自己地址段中的地址——默认是 `198.18.0.0/15`(RFC 2544 基准测试段)。这些地址是代理
的哨兵地址,而不是内网主机,因此私网 IP 守卫会在这种机器上拦掉全部出网抓取:`web_extract`、平台
附件下载、浏览器链路都会以 *URL targets a private or internal network address* 失败,而请求根本
没有发出。声明该地址段即可放行哨兵地址:
```yaml
security:
fake_ip_ranges:
- 198.18.0.0/15
```
默认为空,且比 `allow_private_urls` 更窄:只有被声明的地址段获得豁免,且应当是本地代理自己拥有的
地址段(连接仍然发往代理,由代理自行解析真实目标),回环、RFC 1918、链路本地、CGNAT 和云元数据
目标依然被拦截。
### Tirith 预执行安全扫描
Hermes 集成了 [tirith](https://github.com/sheeki03/tirith) 用于在执行前进行内容级命令扫描。Tirith 能检测单纯模式匹配所遗漏的威胁: