f6f6befbcf
Completes the managed-endpoint rung for every caller that resolves a bare
llamacpp / llama.cpp / llama-cpp alias outside the main ladder:
- The rung moves from _resolve_custom_branch to the resolve_provider_client
entry and to _get_cached_client, via agent/auxiliary_local_runtime.py.
The `main_runtime is None` gate is gone: the cached auxiliary path always
passes a normalized dict, so the rung never ran there, and a task pinned to
llamacpp inherited a foreign main runtime's URL instead.
- No running server now means (None, None) — the same answer the main ladder's
_resolve_llamacpp_runtime gives — instead of continuing into
_try_custom_endpoint / _resolve_api_key_provider, which posted the local GGUF
slug (and, for compression, the conversation) to whichever cloud provider held
a key (#119227) or to the primary's own endpoint. OPENAI_BASE_URL is not
consulted either: the alias table entry (b7e0d715be) exists for explicit
base_url lanes, and the main ladder never reads it for a bare alias.
- The cached client keys on the live endpoint (base_url + key), read at call
time, so a server restarted on a new port/key gets a fresh client and a
stopped one resolves to nothing instead of a stale client.
- The endpoint is resolved with the calling profile's config (load_config() at
call time), matching the main ladder; a configured provider entry named like
the alias still wins.
- Vision keeps the raw provider name for its final resolve so the alias is not
flattened to bare `custom` first.
- The aux "unavailable" message names the stopped local server instead of an
invented LLAMACPP_API_KEY.
Co-authored-by: Gille <4317663+helix4u@users.noreply.github.com>
36 lines
1.8 KiB
Python
36 lines
1.8 KiB
Python
"""Managed llama.cpp endpoint for auxiliary and fallback client resolution.
|
|
|
|
A bare ``llamacpp`` / ``llama.cpp`` / ``llama-cpp`` alias (no base_url, no configured provider of
|
|
that name) means "the local server Hermes manages for this profile" — exactly what the main ladder
|
|
resolves in ``hermes_cli.runtime_provider_custom._resolve_llamacpp_runtime``. With no server
|
|
running the answer is "unavailable", never the generic custom/API-key discovery: that handed the
|
|
local model slug to whatever cloud provider held a key (#119227) or to the primary's own endpoint.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from typing import Any, Optional, Tuple
|
|
|
|
from hermes_cli.local_runtime.endpoint import LLAMACPP_ALIASES
|
|
|
|
|
|
def bare_llamacpp_endpoint(provider: Optional[str], base_url: Optional[str],
|
|
api_key: Any) -> Optional[Tuple[str, Any]]:
|
|
"""``(base_url, api_key)`` of the profile's local llama.cpp server for a bare alias.
|
|
|
|
``base_url`` is ``""`` when nothing is serving. ``None`` when the request is not a bare alias:
|
|
an explicit base_url or a configured provider entry under the alias name resolves as before.
|
|
Read on every call so a restarted server's new port/key is picked up.
|
|
"""
|
|
alias = (provider or "").strip().lower()
|
|
if alias not in LLAMACPP_ALIASES or base_url:
|
|
return None
|
|
from hermes_cli.config import load_config
|
|
from hermes_cli.local_runtime.endpoint import resolve_llamacpp_endpoint
|
|
from hermes_cli.runtime_provider import _get_named_custom_provider
|
|
if _get_named_custom_provider(alias):
|
|
return None
|
|
endpoint = resolve_llamacpp_endpoint(config=load_config()) or {}
|
|
return (str(endpoint.get("base_url") or "").strip(),
|
|
api_key or endpoint.get("api_key") or "no-key-required")
|