Every Linux lane that does real work ran on a 4-core `ubuntu-latest`. The
Python suite and the JS checks were split into many small jobs to make that
size usable. Each split job repeated the full setup. In most of the JS jobs
the repeated setup cost more than the work.
The work lanes move to larger runners. Then the splits that existed only to
make small runners usable go away.
Python tests: 12 slices become 1 job on a 96-core runner. Slicing cost a
matrix job, a duration cache, a per-slice artifact and a merge job. 96 cores
clear the floor that the slowest single test file sets, which is about 82s. A
second slice divides work that is already at that floor, and adds a second
setup. Duration data from run 32522943054 gives the numbers behind this: 3178
files, 11645s in series.
The worker count is explicit, because `run_tests.sh` defaults to twice the
core count. A later commit sets it from a measurement on this hardware.
JS checks: 14 jobs become 1. The matrix paid about 371s of repeated setup to
spread about 612s of work. One larger runner installs one time. The three UI
shard scripts and `run-ui-shard.mjs` are therefore removed, because the
unsharded `test:ui` covers the same tests.
The unit of parallel work inside that job is a CHECK, and not a workspace.
apps/desktop is most of the payload, and its own `check` is a serial && chain.
A spread across workspaces alone therefore leaves that chain as the long pole.
A package that declares `check:*` sub-scripts gives one unit for each
sub-script. That is the same selection rule the matrix used.
The loop lives in `.github/scripts/run-workspace-checks.mjs`, so the same
sequence runs on a laptop. It runs 11 units together, buffers the output of
each one, and fails at the end with the full list. Children that share one
stdout interleave their lines and make a failure hard to read.
`npm run --ws check` stops at the first workspace that fails.
`check:test:plugins` joins the desktop `check` script. The matrix prefers
`check:*` sub-scripts over the plain `check` script, so `check:test:plugins`
ran only as its own leg. Without this change the merge drops that suite and
the job stays green.
node_modules is cached on the lockfile, and `npm ci` is skipped on an exact
hit. The `cache: npm` option of `setup-node` caches only the ~/.npm tarball
cache, which leaves the extract and the postinstalls to pay again.
The arm64 image build stays on a native arm64 runner. A build of linux/arm64
on an x64 host uses emulation.
The docker test lane caps its workers at the core count. Each of those tests
drives a container, so the docker daemon sets the limit and not the processor.
`.github/actionlint.yaml` declares the runner labels. actionlint knows the
GitHub-hosted labels only, and an undeclared label reads as an error that
hides the real findings.
The `detect` job checks out one file through a sparse checkout, and its
timeout drops to 1 minute. It reads
`scripts/ci/classify_changes.py` and nothing else.
Verification:
- actionlint reports 9 findings across all workflows. An unmodified HEAD with
the same config reports the same 9. This change adds none.
- A wrong label still fails. actionlint reports `ubuntu-latest-32-cor` and
`ubuntu-latest-32-arm-cores`.
- Every changed workflow parses, and `name` parses as a string.
- A replay of the `save-durations` merge step against a three-artifact layout
returns all 3178 entries.
- An expansion of the npm script graph gives the same leaf commands for the
parallel units and for a plain `npm run check`, in both directions. Against
the 13-leg matrix the count is 13 to 11, and the whole difference is the
three UI shards that collapse into one unsharded `check:test:ui`.
- `--list` reports the 11 units, and a full local run completes and reports
the time of each unit.
- The runner labels cannot be verified here. The first real run is the test.
Review follow-ups on the compositor spinner and the invalidation scoping.
Spinner CSS:
- Clip each frame to its own box. Braille renders from a system fallback
face (JetBrains Mono has no U+2800 block), whose metrics are not
guaranteed to fit the 1em frame, so neighbouring ink could bleed into
the viewport.
- Name descendants explicitly in the selection guard. The competing
`[data-selectable-text='true'] *` rule has the same (0,1,0)
specificity, so relying on inheritance made the winner depend on
stylesheet order.
- Scope the compositor promotion to spinners that are actually running.
A permanently promoted layer per parked spinner is pure memory at
fan-out breadth, where many sit mounted and paused at once.
- Give every var() the braille default as its fallback, so a missing
custom property degrades to a working spinner rather than an invalid
declaration.
Spinner component: replace the bare `as CSSProperties` cast on the inline
style with an exported GlyphSpinnerVars contract, so a typo in a custom
property name is a compile error rather than a silently dead declaration.
Assistant message:
- Render the inter-agent collapse as a CHILD of the normal body instead
of a competing root. The settled case previously returned a different
element type than the running case, so settling unmounted the whole row
and mounted a fresh one — discarding the DOM the scroll anchor held.
One component, one root, children vary; the truth table is unchanged,
including the collapsed row carrying no tapback listener.
- Collapse AssistantStatusSlot's separate subscriptions into one selector
returning a stable string. The inputs always move together on a status
flip, so reading them separately just multiplied the wake-ups.
- Give StreamingMarker a stable `data-slot` and assert on that rather
than on `span.hidden`.
Repro script: count settled rows by subtracting streaming markers from
message roots instead of `:not(:has(...))`. The selector walked every
row's subtree on each evaluation, inside the very latency window the
probe measures.
Comments: drop the stale translateY(-100%) description, name both pause
triggers, replace hard-coded line-number citations with selector/symbol
ones, note that only the primary window arms the renderer-pause
attribute, and move the forensic trace numbers out of source comments
into the PR.
Delete the three tests that asserted on stylesheet TEXT. AGENTS.md bans
reading source in tests outright, and they demonstrated exactly why: a
var()-fallback edit that changed no rendered pixel broke one of them.
Replacements that exercise the CSS in a real browser follow.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QwTc9XqUjhbay446VjugHZ
Removes the two residual invalidators left by the previous commit.
data-streaming: gone from the message root. The flag is not dead -- it is the
settled-row signal for scripts/run-short-session-hang-repro.mjs -- so it moved
to a permanently-mounted, display:none leaf that is a ROOT-LEVEL sibling, and
the repro now matches on the descendant. Placement is load-bearing three ways:
a node inside [data-slot='aui_assistant-message-content'] would steal
:last-child from the stall indicator and change inter-bubble margins mid-stream
(styles.css:1995-2003); keeping it mounted and toggling only the attribute
keeps the per-flip write on a childless node instead of making it a DOM
structure change; display:none costs no layout or paint while querySelectorAll
and :has() still match it.
Renamed to data-message-streaming rather than reusing data-streaming: shiki
puts that exact attribute on deferred code cards, which are descendants of the
message root, so a descendant-matching selector sharing the name would report
any message holding a deferred code card as still streaming.
root isRunning: gone from the standard path. AssistantMessage now dispatches on
interAgentSender, so the collapse gate's live status subscription lives in
InterAgentAssistantMessage and only the rare inter-agent case pays it. The
enter animation captures its enabled flag once off the runtime, non-reactively,
because use-enter-animation.ts parks the value in a ref behind a useCallback([])
identity and consults it only when the callback ref fires at mount -- a live
subscription fed a value the hook already ignores.
Adds inter-agent-collapse.test.tsx: the collapse gate and the marker contract
both had zero coverage, and nothing in the app reads the marker, so a delete
would otherwise look free and silently regress the repro's response gate.
Behavior-identical; invalidation scope only.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
require.resolve returns the macOS realpath (/private/var/...) while
os.tmpdir() stays on the /var symlink, so a raw join() deepEqual failed
even though the spawn was correct.
With a GH_TOKEN/GITHUB_TOKEN in the environment, electron-builder auto-selects
the github provider and resolves owner/repo from the repository field, falling
back to reading <projectDir>/.git/config. projectDir is apps/desktop, which has
no .git of its own, and app-builder-lib does not walk up to the workspace root
-- so resolution returned null and threw "Cannot detect repository by
.git/config".
On Linux this fires from onAfterPack for a plain `dir` target: the darwin and
Windows branches return early for non-installer targets, Linux has no such
guard. That is why the same build worked elsewhere.
--publish never keeps `pack` from reaching this at all, but `dist:*` and
test-desktop.mjs still resolve publish config on a machine with a token, so
declare the field too.
Tests call the real app-builder-lib resolver rather than asserting on the text
of package.json, so they track electron-builder's behavior instead of our
formatting.
Co-authored-by: airo7 <airo7@users.noreply.github.com>
Co-authored-by: frankmendes1979 <frankmendes1979@users.noreply.github.com>
The desktop window opened blank white on a fresh install: React threw
"Minified React error #527" before the first paint, from the
`vendor-react-<hash>.js` chunk.
`apps/desktop` pins react and react-dom to the same exact version, but
`vite.config.ts` aliased both to a hardcoded `../../node_modules/<pkg>` —
straight into the monorepo root, where npm is free to hoist a different
react. `@streamdown/math` is a root dependency whose react peer is
`^18.0.0 || ^19.0.0` and which declares no react-dom peer, so npm hoists
the newest react (19.2.8) to the root while react-dom stays at the
version hoisted from the workspaces (19.2.7). react-dom's own peer is
`react: ^19.2.7`, which 19.2.8 satisfies, so the install reports success
and nothing warns. The bundle then shipped react 19.2.8 with react-dom
19.2.7 and React refused to run.
`npm ci` masks this because the lockfile pins the root react to 19.2.7,
which is why CI is green. The recurrence engine is
`_run_npm_install_deterministic()`: when `npm ci` fails it falls back to
`npm install --no-save`, which re-resolves the whole tree and never
records the result — so the split comes back on the next update and
leaves no trace.
Fix the resolution rather than the hoist. The aliases now resolve both
packages from the desktop workspace itself, where npm guarantees the
declared versions are reachable (it nests a copy under the workspace
exactly when the hoisted one differs), so the pair can only ever match.
Pinning react at the npm layer instead was rejected: every manifest-level
pin tried (root dependency, root `overrides`, a scoped override on
`@streamdown/math`) breaks a fresh install with
`ERESOLVE ... peer ink-text-input@"6.0.0" from @hermes/ink@0.0.1`.
Two guards keep it from silently returning:
- `assert-root-install.mjs` (the existing preflight of `build`,
`dev:renderer` and `preview`) now fails the build when the resolved
react and react-dom versions differ, so a split surfaces as an
actionable error instead of a white window.
- A `tests-js` contract test asserts every workspace pins the two to the
same exact version, and that the desktop bundler no longer points at a
hardcoded `node_modules` path.
Verified on a synthetic split tree (root react 19.2.8 / react-dom 19.2.7,
workspace react 19.2.7): the old aliases resolve 19.2.8 + 19.2.7, the new
ones resolve 19.2.7 + 19.2.7, and the preflight exits 1 when the
workspace itself resolves the mismatch.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The desktop journal synchronously read, parsed, cloned, and rewrote one
aggregate localStorage value while streamed turns were repainting. Large
tool results and multi-session state could therefore block the renderer and
leave the app unresponsive, while the existing macOS diagnostic path lacked
a real native hide/restore regression check.
Store bounded recovery projections under per-session keys, migrate legacy v1
data once, isolate quota and storage failures, and preserve the newest
recoverable tail without allowing oversized writes to replace valid state.
Add a real Electron/CDP macOS-arm64 A/B harness with native visibility control,
renderer heartbeat and Settings/composer/transcript checks, plus focused
regressions. Keep bulk tool payloads, diagnostics, and the existing recovery
merge behavior out of the hot path.
Fixes#63047
The bench timed two of the six RPCs the fix touches. Extend it to
image.attach, pdf.attach, clipboard.paste and image.detach so every changed
handler carries a number rather than an inference.
Also report which surfaces reach these RPCs at all, since "why was the GUI
special" is the first question the fix invites. CLI attaches inline in its
own turn path with the agent already built, so it cannot reach the stall;
the TUI calls the same RPCs and was equally exposed. The difference was hit
rate, not code path.
gateway_attach_bench.py drives the real dispatcher with a session whose agent
build is still running and times each attach RPC against prompt.submit as the
control — the harness that located the stall and measures it.
image-attach-bench.mjs times the renderer-side transforms (file read, base64,
RPC frame, embedded-image extraction, render-weight walk) across image sizes.
It is what ruled the renderer out: ~26ms total at 3MB.
Review findings: spawnSync('npm', ...) without shell fails on Windows
(npm is npm.cmd; Node >=18.20 throws EINVAL — same handling as
test-desktop.mjs and stage-native-deps.mjs), and a spawn-level failure
exited 1 with no diagnostic. CI is ubuntu-only but the desktop workspace
supports local Windows dev.
Closes the silent-skip hole reviewers flagged: with the index/count
hardcoded in three sibling strings, a copy-paste slip (shard-2of3 running
--shard=1/3) or a partial 3->4 migration would silently skip a third of
the 428-file suite while CI stays green.
scripts/run-ui-shard.mjs parses N/M from npm_lifecycle_event (the script
NAME is the single source of truth), validates the package's shard family
is exactly 1..M for one M, and delegates through 'npm run test:ui' so the
vitest command stays single-sourced.
Mutation-verified: shard-9of3 name -> exit 1 'index out of range';
adding shard-4of4 beside the 3-family -> exit 1 'must form exactly 1..M';
correct invocation runs shard 2/3 (142 files) identically to before.
* test(desktop): stress long agent sessions in the multitab perf scenario
--tools seeds every transcript with settled tool rounds and drives the live
stream as a working agent turn (tool calls opened and completed between text
chunks), and each tile reveal is timed to next paint (reveal_max_ms) so deep
transcripts report their mount cost.
* perf(desktop): hold the transcript window cut steady while streaming
A fresh weight-walk per store flush slid the cut forward one message at a
time, ~30x/s, and every slide re-indexed the whole windowed transcript —
each row rendered a different message and the runtime repository took its
O(window) rebuild path instead of the one-message update. advanceTranscriptWindow
anchors the cut to a message id and re-cuts once per ~half page of new
content instead of once per flush.
* perf(desktop): stable rows, stepped backfill, pane-shared render budget
Three thread-list fixes for long streaming sessions: memoize the visible-
groups slice and each turn row so a budget-cut advance no longer re-renders
every mounted turn per streamed token; raise the first-paint backfill in
BACKFILL_STEP slices (one bounded commit per frame) instead of a single
20-to-600 transition whose commit landed as a 780ms freeze mid-stream; and
share RENDER_BUDGET across mounted panes so a 4-way grid mounts a quarter
page per pane instead of 4x the fibers.
* fix(desktop): evict settled session states nothing on screen references
Closing a tile never removed its runtime's entry from $sessionStates, so
every tile ever closed parked its full transcript in the map for the life
of the process. Each leftover entry taxes every subsequent stream flush —
the map is spread-copied per delta and the busy/attention/draft projections
walk every entry per publish — so the app got slower the longer it ran,
which users read as "I need to clean my sessions/dbs".
Publish now evicts a settling state when no tile and not the primary view
holds its runtime (transition side effects still fire, so the settle keeps
its unread dot), and closing a tile drops an already-settled state on the
spot. Busy and needs-input states stay: background turns feed the sidebar
dots, and a first publish always lands because a resume can publish a beat
before the surface binds the runtime.
16 tiles streaming in a 2x2 grid with a day's worth of closed-tile residue:
worst-second 34 -> 58 fps, p99 frame 90 -> 28 ms, longtasks 37 -> 0.
* perf(desktop): index lineage aliases per sessions-list reference
lineageAliases scanned the whole recents list per call, and it is called
per cached session state per status projection per message delta — with a
populated sessions DB and a few busy sessions that multiplied out to
millions of row checks a second during streaming. Build the alias index
once per list reference (the list is replaced wholesale, never mutated)
and look aliases up in O(1).
* perf(desktop): journal each in-flight turn under its own storage key
The v1 journal kept every session's tail in one localStorage key, so each
throttled write re-parsed and re-stringified EVERY busy session's snapshot
— a grid of concurrent streams turned that into a whole-store JSON round
trip dozens of times a second, all on the main thread. Per-session keys
make a write O(own tail) no matter how many other sessions are streaming.
A v1 store migrates on first touch; expired/overflow crash residue is
pruned once per renderer.
* perf(desktop): stress the multitab scenario across grid/streaming/DB axes
The one-stack multitab run hid every cost this round of fixes removed: it
drove hook.publish (store only — no journal, no wiring cache), with an
empty recents list and no closed-tile residue. Streaming now routes through
hook.update (the real gateway write path), and the scenario grows axes for
the workloads users actually hit: --zones splits tiles across visible grid
zones, --streaming caps how many sessions are mid-turn (zone leaders
first), --sessions seeds a lived-in recents list, --dead models settled
sessions no surface references. launch.mjs pins HERMES_DESKTOP_CDP_PORT so
a non-default --port survives the app's own dev-CDP flag.
Fixing the allowlist only helps a fresh install. npm will not re-run an
install script for a package already on disk, so every checkout that
installed while get-windows was blocked stays bricked: `hermes update`
pulls the fix, `npm install` skips the script, and the build fails on the
same missing binding.
Run `npm rebuild get-windows` from the staging step when the binding is
absent, and if that still yields nothing, print the two commands that
recover the checkout by hand instead of the previous advice to reinstall
dependencies, which is exactly what the user already tried. Gated to a
win32 host building for win32, since no other host can produce the binding.
Co-authored-by: JoaoMarcos44 <JoaoMarcos44@users.noreply.github.com>
The published tarball ships lib/binding/napi-9-darwin-unknown-arm64 on every
platform, so a real Windows host has both it and the downloaded win32 binding
— the classify-everything gate threw on the darwin dir and killed every
Windows pack. Stage only bindings naming the target platform (classify still
rejects impostors), stop copyGlobByExt from recursing into lib/binding, and
add a version tripwire so a get-windows bump fails the build until the
lib/windows.js rewrite is re-verified.
Also from review: the renderer answers window.read.respond with empty text
when the IPC invoke rejects (older shell / main-side throw) instead of
stalling the tool's 30s timeout; the tool schema discloses that sibling
Hermes windows are skipped; docs gain read_window_below in both references.
get-windows@9.3.0 (MIT, zero runtime deps on macOS/Linux) is external to the
esbuild bundle and staged into dist/node_modules per target platform: the
universal Swift helper on macOS, the prebuilt N-API binding on Windows
(fail-closed magic-byte validation), nothing on Linux (xprop at runtime).
The staged lib/windows.js is rewritten to load the binding directly so
@mapbox/node-pre-gyp's tree stays out of the package.
⌘K is an overlay that is stateful to itself — pressing it owes the user a
frame immediately, whatever else the shell is doing. It was not built that
way.
`CommandPalette` is mounted for the life of the app, and its body ran
unconditionally: a dozen store subscriptions (connection, desktop version,
client + backend update status/apply, keybinds, worktrees, theme, i18n),
three `useQuery`s, and the group builders that assemble a few hundred rows.
`<Portal>` renders nothing while closed, so none of it was ever visible —
but all of it still ran. An in-flight update rewrites `$updateApply` on
every progress line, and each of those rebuilt the entire row set for a
surface nobody could see.
Split the body into `CommandPaletteBody`, mounted only while the palette is
on screen. A closed palette is now one store subscription. The body is keyed
by open count, so per-open state (search, sub-page) resets by remount and
the explicit close-reset effect goes away, and `mounted` lags `open` by the
150ms exit animation so Radix can still play `data-[state=closed]` instead
of the overlay vanishing.
Rows additionally move behind `useDeferredValue` in their own memo
component. Because that component mounts with the portal, the deferred
initial value applies per open: the first commit is the frame + input, and
the several-hundred-row list arrives in an interruptible follow-up render
rather than blocking the frame the keypress asked for. The empty state is
suppressed while rows are still pending so opening doesn't flash "no
results".
The `enabled: open` gates on the three queries are dropped — the component
only exists when open, so they are inherently lazy, and react-query still
serves a reopen from cache while revalidating.
`submit` measures the scroll jump when a turn is appended; nothing measured
the jump when a session is opened, which is the prepend/settle path. Clicks
sidebar rows and tracks how far the bottom turn moves after first paint.
The sidebar stays mounted beneath every overlay/page, and it subscribes to
$sessions + $workingSessionIds — both tick on every streaming token. An
unmemoized SidebarSessionRow re-rendered the whole list (Codicon, labels,
status dots) on each delta, and that churn bled into every overlay opened on
top: Cron, Profiles, Agents, Starmap, Webhooks, Command Center, Settings.
- SidebarSessionRow: memo() with a custom comparator that ignores the pure
id-forwarding callbacks (fresh closures by design) and compares only the
data that changes what the row paints. Rows bail out while siblings stream.
- SessionStatusDot: the 5 $...SessionIds arrays now read via useStoreSelector
returning this session's boolean, so a dot repaints only when ITS OWN
membership flips, not on every array tick.
- Artifacts: stable cellCtx (useMemo) + memoized Primary/Location/Session
cells so a link-title fetch on one row stops re-rendering the whole table.
Measured before/after (2s idle, sessions streaming), sidebar-fed overlays:
Cron 407->~30 wasted, Profiles 732->~80, Agents 132->9, Starmap 188->56,
Webhooks 154->22.
Gating this behind an opt-in was the wrong call. A dev server already
executes arbitrary local JS — vite's module graph, every postinstall in
node_modules — so a loopback debugging port does not meaningfully widen
what a `npm run dev` session can already do, and `perf:serve` has opened
one unconditionally all along.
Requiring the variable also defeated the point: the tooling exists to be
reached for mid-task, and a capability you must remember to enable before
launching is one you don't have when you need it.
So the port opens on 9222 — the same port scripts/eval.mjs and
scripts/perf/lib/cdp.mjs already default to — for any dev-server run.
HERMES_DESKTOP_CDP_PORT stops being an on-switch and becomes an
override: a different port, or `off` to disable.
The hard gate is unchanged and still checked first: a packaged build
never opens the port, and no env value talks it into it. Neither does an
unpackaged `electron .` against dist/, which is how the packaged app
gets smoke tested.
Refusals only log when they contradict something the developer asked for
(a typo'd port, an explicit `off`). Packaged and dist runs are closed by
design and stay quiet.
The renderer is a Chromium page, and apps/desktop already carries a whole
CDP toolkit for it — scripts/eval.mjs, scripts/perf/lib/cdp.mjs with its
shared SELECTORS map, and the diag-*/probe-* family. None of it can
attach to `hgui` or `npm run dev`, because neither passes
--remote-debugging-port. The only launcher that opens one is
`npm run perf:serve`, which is a separate isolated instance rather than
the app you're looking at.
Add HERMES_DESKTOP_CDP_PORT. When set, the shell opens a CDP port on
loopback so that existing tooling can read the live DOM: computed
styles, geometry, which rule actually won.
Three independent gates, all required, resolved by a pure function in
electron/dev-cdp.ts so the policy is testable without an Electron app:
1. not packaged — a shipped build never opens the port, and this is
checked first so no env combination can talk it into doing so;
2. HERMES_DESKTOP_DEV_SERVER present — an unpackaged `electron .`
against dist/ is how the packaged app gets smoke tested, so it
behaves like the packaged app here;
3. the port explicitly requested and a valid integer.
Default `npm run dev` is unchanged and silent: no port, no nag. An
opt-in that gets refused always logs why, so nobody loses an hour
wondering what isn't listening.
The address is pinned to 127.0.0.1 rather than left to Chromium's
default, and is deliberately not configurable — there's no reason to
expose a renderer debugger off-host and offering the knob invites
someone to try.
scripts/eval.mjs hardcoded :9222 and threw a raw ECONNREFUSED stack when
nothing was there. It now honours the same variable and explains itself.
Switching to a STREAMING session took ~1.4s to settle while an idle
session settled in ~50ms. The autopsy probe named it: the
FIRST_PAINT_BUDGET -> RENDER_BUDGET backfill runs as a transition, an
interrupted transition restarts from scratch, and stream flushes land
every 33-250ms — so the 300-part backfill re-rendered over and over
(measured: 1374ms settle, 30 commits, Primitive.div x2237 for one switch).
Gate the backfill on the thread being idle. The user lands on the live
tail immediately either way; older turns backfill the moment the run
ends, and 'Show earlier' remains the manual path meanwhile.
Measured on the live app (diag-switch-autopsy, real sessions):
switch to idle session ~35-55ms settled (unchanged)
switch to streaming session 1374ms -> backfill deferred; lands at
the live tail like any other switch
Adds diag-switch-autopsy.mjs (per-switch settle/commits/top-renders) and
live-drive.mjs (status/fps/drag one-liners against the running app).
Driving HER real instance (real profile, real transcripts, streams live)
via CDP instead of synthetic tiles finally exposed the remaining stall.
The timeline on a real 60-frame sash drag:
style recalc 2736ms | script 1027ms | layout 89ms
top callsite: pin @ fallback.tsx — 927ms
Two pin-to-bottom ResizeObservers (the bounded tool window's and the
reasoning preview's) pinned on EVERY resize delivery. A sash drag changes
every message's WIDTH once per frame, so each frame ran scrollTop write ->
scrollHeight read across every tool group: a forced write-read reflow
cascade that the render counters could never see (zero React involvement).
Both pins are now height-gated off the RO entry (reflow-free): only
content GROWTH pins. Width-only deliveries return immediately.
Measured on the live app, same drag, before -> after:
fps 11.5 -> 59-60
p95 101ms -> 18ms
slow>33 60/60 -> 1/60
Also in this batch (each was verified live before the next was attempted):
- thread/list: split messageSignature into STRUCTURAL (ids/roles — keys
boundaries + row identity) and WEIGHT (part counts — budget only), and
memoize groups + row JSX. A streamed part-append re-rendered every
turn's boundary via its resetKey prop; explain() measured 540-865
wasted Block renders per drag/stream sample, now {}.
- message-render-boundary: document the structural-only resetKey contract.
- tool/fallback: memoize ToolFallback's part object + ToolEntry/ToolTitle/
ToolGlyph (151 renders each, 100% wasted, on real transcripts).
- use-message-stream: ADAPTIVE flush floor — next flush waits 3x the
measured cost of the last one (33ms floor, 250ms cap), so multi-stream
load degrades text update rate instead of input latency.
- tree-split: preview sash drags with inline flex on the two seam
wrappers, committing the store ONCE on release (fixed-zone sides get
flexBasis only, so a hidden sidebar can't leave a phantom gap).
- debug/: perf-live LoAF long-frame attribution, explain() cascade walker
with changed-hook indices, diag-real-loop/key-latency/switch-trace
probes that drive the real app over CDP.
Typing during 2 live streams: keystroke->paint p50 3.3ms, p95 18.4ms,
zero frames over 33ms. Session switch p50 ~35ms settled; the remaining
~1.3s outlier tail is streaming-session switches (React work-loop, not
style/layout) — next target.
Captures medians of 5 runs for multitab and render-churn so tonight's
wins can't silently regress.
idle-cost is deliberately NOT gated. Its render attribution and idle
commit rate are trustworthy and are what the scenario exists for, but the
drag fps it reports (~0.6fps, p95 814ms) contradicts a direct
single-clock probe of the same gesture on the same build (57fps). I ruled
out sash selection, tile setup, render-counter residue, and a 20s soak,
and could not explain the gap — so the metric ships as a report, not a
gate. Gating CI on a number I can't defend would either fire on a phantom
or mask a real stall.
tier: 'report' is outside GATED ('ci','cold'), so the scenario still runs
and prints but neither compares nor writes a baseline.
withFrames ran its own requestAnimationFrame ticker while the gesture body
independently awaited rAF per step. Two rAF consumers, so the observer's
deltas counted the driver's frames as well as the app's — it reported
~3fps for a drag that a single-clock probe measures at ~23fps, and it
never moved no matter what got fixed underneath.
Timing now comes from the same callbacks the body drives (__MARK__).
This also fixes a silent false-negative on the typing pass: it paced on
setTimeout, so the independent ticker was mostly sampling idle waits
between keystrokes and reported a flat 61fps. On the driving clock the
same interaction reports ~30fps with 27 of 40 frames over 33ms — which
matches the 'typing feels slow' symptom I previously could not reproduce.
TYPE now records __TYPE_TARGET__ and the runner throws when no composer is
found, so a pass that measures nothing fails loudly instead of scoring a
perfect 0 deficit — same guard DRAG already had.
A CDP trace of one sash drag settled what the render counters could not.
I had assumed the remaining cost was layout/paint; it was not:
script 6770ms | style 1866ms | layout 71ms
Layout was never the problem. The top attributable callsite in our own
code was use-resize-observer.ts at 977ms.
Counting the callbacks named the mechanism exactly: 8,620 ResizeObserver
instances constructed, and during a 40-move drag 2,600 callbacks each
carrying exactly ONE entry — 65 separate callbacks per pointermove. Every
consumer owned a private observer, so N elements resizing under a common
ancestor meant N trips through the observer machinery instead of one
batched delivery. With five mounted tiles that is ~100 user bubbles, each
with its own observer, all woken by a width change.
One shared observer with a WeakMap of target -> handlers. Callers keep
their exact contract: a handler observing several elements is still
invoked once with all of its entries, and unobserve happens when the last
handler for an element goes away.
Verified by trace, before -> after:
use-resize-observer 977ms -> 42.5ms (-96%)
style recalc 1866ms -> 1145ms (-39%)
total script 6770ms -> 3929ms (-42%)
Callback count 2,600 -> 43: one delivery per frame instead of 65.
Adds the two probes that found it. diag-drag-trace.mjs takes a real
timeline trace and prints the style/layout/script split plus the top
script callsites — that split is what disproved the layout theory.
diag-ro-storm.mjs counts RO callbacks vs entries, which is what
distinguished 'a few expensive calls' from 'very many cheap ones'.
TreeGroup called useStore($layoutTree) to build its right-click menu's
move/split directions. That subscribes every zone — and therefore every
mounted pane and its entire transcript — to the whole layout tree. A sash
drag rewrites the tree once per frame, so dragging the sidebar re-rendered
all five tiles' message lists on every pointermove, for a context menu
nobody had open.
The directions are only read when the menu renders, so read the tree there
with .get() instead. Same lazy shape the neighbouring `closable` prop
already uses.
Measured over one 60px sash drag with five busy tiles:
commits 83 -> 12
ChatView 150 -> 10 (4465ms -> 353ms)
AuiProvider 9450 -> 630 (9868ms -> 774ms)
TreeGroup 180 -> 12
TreeSplit 90 -> 6
Also fixes an observer effect in the harness: idle-cost recorded render
attribution *during* the timed gesture, and the counter walks the fiber
tree on every commit. That was large enough to hide this 15x reduction
behind an unchanged fps, so timing and attribution are separate passes now
and `record` defaults off.
Adds scripts/diag-drag-churn.mjs — the probe that found this. It reports
the transcript chain (who above the messages re-rendered) plus every atom
that notified, which is what named TreeGroup instead of leaving it to be
guessed at. Notably the atom list came back EMPTY: this was never store
churn, so the render-attribution path was the only thing that could have
found it.
The synthetic gesture oscillated +/-3px, which nets to zero displacement
and can clamp to a no-op — so it reported a confident fps number for a
drag that never moved the sash. Sweeps monotonically now, dispatches
pointer events React's synthetic system accepts (isPrimary/button/buttons),
and records dragTarget + dragMoved so a drag that silently did nothing is
visible in the output rather than passing as a measurement.
Verified: dragMoved now reports 60px where it previously reported 0.
Adds an `idle-cost` scenario for the symptom Brooklyn reported: with a
thread spinning, resizing the sidebar feels slow. It holds N tiles busy,
pushes NO tokens, and measures the renderer's self-inflicted commit rate
plus fps while dragging the splitter and while typing.
It reproduces immediately. Five busy tiles, nothing streaming:
idle commits 17.7/sec (should be 0 — nothing is happening)
drag 1.4 fps p95 812ms, worst frame 1.9s
typing 61 fps (fine — this is specific to resize)
Attributing the drag window showed 105,385 TooltipProvider renders and
~15s of component time across a 60-frame gesture. Cause: `Tip` mounts a
full Radix provider + Tooltip per call site, and there are ~107 of them.
Radix's Tooltip holds real state and Popper subscribes to layout, so an
unrelated interaction re-rendered all of them.
Mounts the machinery lazily instead, on first hover/focus. Tooltip churn
drops ~4x (105k -> 26k) and drag doubles to 3fps.
Note `defaultOpen` on the armed Tooltip is load-bearing: the pointerenter
that armed it has already fired, so Radix never sees it and the tip mounts
silently closed. A test caught exactly that, and now guards it.
3fps is still bad — the remaining cost is the whole transcript
re-rendering per resize frame (MessagePrimitive.Parts 12,600 renders /
10.5s, Block/Ct 24,300 each, all 100% wasted). Separate fix.
Two problems, both found by distrusting the harness's own numbers.
1. The scenario slept a fixed 1s after mounting tabs, then recorded. Boot
and session hydration are not reliably done by then, so a variable
amount of unrelated work landed inside the measurement window. Three
back-to-back runs on identical code spread 2.2x on total_renders and
3.8x on wasted_renders — wide enough that a single-run before/after
delta could be mostly noise. Replaced with a quiesce gate that waits
for commits to hold still before recording, and reports 'quiet:N' or
'timeout:...' so a contaminated run is visible instead of silent.
2. The counter attributed a context-driven re-render as 'wasted', which
pointed at memo() as the fix when memo cannot block context at all.
Adds contextChanged via the fiber's context dependency list, and
excludes it from wasted.
The gate also turned up a finding worth more than the fix: with five busy
tiles and NO driver running, the renderer still commits ~18x/sec. The
report now names the cascade roots (own state changed, props did not)
rather than leaving them to be guessed at — Streamdown re-renders itself
105 times while idle, which is what drives Block/Ct.
Drives the same synthetic multi-tab streaming workload as `multitab`
(publishSessionState per session per flush, no backend, no credits) but
reports render attribution instead of frame pacing — sidebar_renders,
wasted_renders, and wasted_notifies.
Answers 'does the sidebar re-render while an agent is typing' directly.
The existing `stream` scenario measures streaming into an empty thread, which
is exactly the case where per-delta transcript work does not show up. This adds
`--historyTurns`, which mounts a settled transcript and lets it drain BEFORE
the recorders start, so the measurement window contains streaming work only —
and `stream-history`, a report-only preset that turns it on.
Report-only (tier: manual) rather than gated: how much history a host can mount
varies, so the absolute number is not comparable across machines. It is meant
for same-machine before/after runs.
Co-authored-by: Jakub Wolniewicz <frizikk@users.noreply.github.com>
probe-model-picker.mjs times pill-click → menu painted over N rounds;
profile-model-picker.mjs wraps one open in a CPU profile with a
top-self-time table. Both attach to the perf:serve instance.
N session tiles stacked as tabs, all mounted (keep-alive) and all
streaming concurrently through the real publishSessionState path — the
"several PR reviews at once" workload. Frame pacing + longtask metrics,
no backend or credits needed; the workload that exposed both fixes above
and the regression gate that keeps them fixed.
The desktop self-update chain (Desktop -> hermes-setup --update ->
hermes update -> hermes desktop --build-only -> relaunch) rebuilds
Hermes.exe on the user's machine and declared success on bare file
EXISTENCE. A truncated PE (corrupt cached Electron zip / interrupted
extraction or rcedit rewrite / full disk) or a wrong-architecture
unpacked tree therefore shipped as the 'updated' app, which Windows
refuses to load with 'This app can't run on your computer'
(此应用无法在你的电脑上运行) — and the previous working build had
already been wiped by before-pack.mjs, leaving nothing to fall back to.
Fix, in three parts:
- hermes_cli/main.py: post-build integrity gate on Windows
(_ensure_desktop_exe_launchable). Parses the PE header of the freshly
built Hermes.exe — MZ/PE magic, section-table completeness vs file
size (catches truncation), and COFF machine vs the host arch (catches
arm64/x64 mixups). On failure it purges the (likely corrupt) cached
Electron zip, invalidates the content-hash build stamp so the
updater's retry-once genuinely re-downloads and rebuilds, restores
the previous build from the .bak tree when one exists (keeping the
corrupt tree as .corrupt for diagnostics), tells the user the update
was aborted and their old version kept, and exits nonzero.
_desktop_packaged_executable also now prefers a host-loadable PE over
pure newest-mtime when multiple win-*-unpacked trees coexist.
- apps/desktop/scripts/before-pack.mjs: on win32, the previous unpacked
tree is preserved as <appOutDir>.bak (only when it holds the product
exe — partial/corrupt trees still get the plain wipe) instead of
being destroyed, providing the rollback material for the gate above.
Non-Windows behavior is unchanged.
- Behavior-contract tests: tests/hermes_cli/test_desktop_exe_integrity.py
(23 tests — synthetic PE fixtures for truncation/non-PE/arch-mismatch,
rollback semantics, and the build-only exit contract) and 6 new vitest
cases in before-pack.test.mjs for the .bak preservation rules.
Progresses #69179
Adds a full desktop Playwright E2E suite that launches the Electron app
against a mock inference server, exercising the full boot chain:
electron -> hermes serve -> mock provider -> renderer
Includes:
- Mock OpenAI-compatible inference server (mock-server.ts)
- Shared fixtures with sandbox isolation (credentials, HERMES_HOME,
userData, fixed window-state.json for reproducible screenshots)
- Test specs: boot, boot-failure, onboarding, mock-backend-setup, chat,
and packaged-app launch
- Visual regression: expectVisualSnapshot() wraps toHaveScreenshot in
try/catch so diffs are reported without failing the test suite
- CI workflow: xvfb at 1280x1024, baseline cache from main
(--update-snapshots on main, compare on PRs), step summary table with
diff/actual/expected image links, dedicated visual-diffs artifact
- dev:mock script for local fake-provider development
- test:e2e:visual + test:e2e:update-snapshots scripts using cage
- .gitignore: *-snapshots/ (baselines cached in CI, not committed)
* fix(desktop): allow write-build-stamp from non-git checkouts
Stop hard-failing npm pack when neither GITHUB_SHA nor git HEAD is
available (ZIP installs / broken .git). Emit an explicit fallback stamp
instead so local Windows desktop builds can finish (#50823).
* fix(desktop): treat fallback stamps as unpinned; harden Windows install
Keep all-zero fallback commits out of -Commit/--commit pins and fetch
install.ps1 by branch instead. After bootstrap, pin the marker to the
checkout HEAD so isBootstrapComplete accepts it. On Windows, force ZIP
checkout, seed GITHUB_SHA (ASCII-only install.ps1), and avoid the pack
stamp failure.
* fix(install): pin core.autocrlf=false before ZIP-path checkout (#50823 review)
The ZIP-fallback path added in #67643 runs `git checkout -f FETCH_HEAD`
before core.autocrlf gets pinned (which only happened later, on the
shared clone-path config). On Git for Windows -- where core.autocrlf
defaults to true -- that renormalizes the repo's LF text files to CRLF in
the working tree during checkout, leaving the freshly-created managed
checkout dirty versus HEAD and aborting the next `hermes update`. That is
the exact "dirty tree the user never touched" failure the surrounding
code already guards against (install.ps1:1461-1469, 1750-1753).
Move the `config core.autocrlf false` pin to run immediately after
`git init`, before the fetch/checkout. The later idempotent pin on the
shared clone path is retained so git-clone installs are unaffected.
Addresses teknium1's review on #67643 and supersedes it, preserving the
original author's two commits.
Co-authored-by: HexLab98 <8422520+HexLab98@users.noreply.github.com>
* chore(contributors): map austinpickett commit email for attribution
The check-attribution CI gate flagged austinpickett@users.noreply.github.com
as an unmapped commit-author email (introduced by the autocrlf fix commit
on this PR). Add the per-email mapping file as the gate instructs (the
legacy AUTHOR_MAP in scripts/release.py is frozen).
---------
Co-authored-by: HexLab98 <liruixinch@outlook.com>
Co-authored-by: austinpickett <austinpickett@users.noreply.github.com>
Co-authored-by: HexLab98 <8422520+HexLab98@users.noreply.github.com>
Profiling the boot answered "is there a real cold-start win?": no wasteful
hotspot — the renderer does only ~tens of ms of work at mount, no heavy library
(shiki/mermaid/katex/d3/motion) initializes at startup; the rest is Electron
runtime + waiting, near the Electron floor.
It also exposed that the cold-start number was pessimistic: a fresh
--user-data-dir per run means a COLD V8 code cache and worst-case bundle
recompile every launch. Real users reuse their profile. Measured delta:
fresh (cold cache): spawn→interactive ~1.48s
reused (warm cache): ~1.0s
So representative launch is ~1.0s; only first-launch-after-install pays ~+400ms.
- coldStartSamples() reuses one profile (run 0 warms the cache, discarded;
runs 1..N are warm samples), stepping ports + pausing so the single-instance
lock releases. `--cold-fresh` measures the first-launch worst case.
- Re-baselined cold-start with the representative warm numbers.
Net: nothing high-ROI left to optimize. The only lever is shipping a pre-warmed
V8 code cache to make first launch match warm (~400ms, once per update) — real
packaging complexity for a marginal win, deliberately not pursued.
* bench(desktop): measure the full picture — prod build, cold-start, first-token
Stop drip-feeding scenarios: extend the harness to cover the latencies that
actually dominate perceived speed, and measure them on a REAL production build.
- --prod: build a production renderer with the probe included (VITE_PERF_PROBE=1,
off in normal builds) and launch it from dist/. Measures minified React, so
numbers are representative shipped figures instead of ~3x-inflated dev ones.
- cold-start scenario (tier "cold"): launch → CDP → driver → first paint, via a
fresh isolated spawn per run. Captures spawn_to_cdp_ms, spawn_to_driver_ms, fcp_ms.
- first-token scenario (backend tier): Enter → first assistant token painted —
the TTFT latency an agent app is uniquely judged on.
- run.mjs gained --prod (build once), cold-start fresh-spawn loop, and gates
ci+cold tiers against the baseline.
Baseline re-captured on a PRODUCTION build (median of 5), darwin-arm64 — all
green. Representative numbers:
cold-start spawn→interactive ~1.6s, FCP ~0.5s
stream frame p95 22ms, 1 longtask
keystroke p50 2ms, p95 8.7ms
transcript mount 145ms, 82ms longtask (400-msg open)
The prod build also settled the open question from the dev numbers: the
transcript-mount "lead" (221ms longtask in dev) is only ~72-82ms in prod — not
actionable. Measurement did its job.
* bench(desktop): trustworthy cold-start measurement (code-splitting is NOT the lever)
Investigated code-splitting the ~22MB renderer bundle to cut cold start. It is
the wrong fix on both counts:
1. Intentional design: vite.config disables codeSplitting because Shiki emits
thousands of dynamic chunks and electron-builder OOMs scanning them — a
packaging/installer constraint, not an oversight.
2. The data says it wouldn't help. Fixing the cold-start measurement to be
trustworthy and reading the boot composition (prod build):
spawn → interactive ~1.5s
renderer nav → DOMInteractive ~0.8s, → DOMContentLoaded ~1.06s
so the whole 22MB bundle EVAL is only ~0.27s (DCL − DOMInteractive) of the
~1.5s. The dominant costs are Electron/window startup and React app mount —
neither touched by splitting.
The measurement fixes (the real content of this PR — no app change, since the
optimization was rejected):
- Drop HERMES_DESKTOP_BOOT_FAKE from spawned instances — it injected artificial
per-phase boot-overlay sleeps that inflated cold-start (and slowed every run).
- Unique debug/dev port per cold-start run — a just-killed instance can hold
:9222 briefly, so reusing it made CDP attach to the DYING instance and report
garbage (spawn_to_cdp of ~4ms). Stepping the port per run fixes the race.
- Richer boot marks (dom_interactive, dom_content_loaded, main-script size) so
cold-start composition is visible, not just a single number.
- Forward all numeric boot marks from the cold-start loop.
- Re-baseline cold-start with the clean numbers.
A real cold-start win would target Electron startup / app-mount (e.g. V8 code
cache, deferred non-critical mount) — a future pass, now that it's measurable.
Stop drip-feeding scenarios: extend the harness to cover the latencies that
actually dominate perceived speed, and measure them on a REAL production build.
- --prod: build a production renderer with the probe included (VITE_PERF_PROBE=1,
off in normal builds) and launch it from dist/. Measures minified React, so
numbers are representative shipped figures instead of ~3x-inflated dev ones.
- cold-start scenario (tier "cold"): launch → CDP → driver → first paint, via a
fresh isolated spawn per run. Captures spawn_to_cdp_ms, spawn_to_driver_ms, fcp_ms.
- first-token scenario (backend tier): Enter → first assistant token painted —
the TTFT latency an agent app is uniquely judged on.
- run.mjs gained --prod (build once), cold-start fresh-spawn loop, and gates
ci+cold tiers against the baseline.
Baseline re-captured on a PRODUCTION build (median of 5), darwin-arm64 — all
green. Representative numbers:
cold-start spawn→interactive ~1.6s, FCP ~0.5s
stream frame p95 22ms, 1 longtask
keystroke p50 2ms, p95 8.7ms
transcript mount 145ms, 82ms longtask (400-msg open)
The prod build also settled the open question from the dev numbers: the
transcript-mount "lead" (221ms longtask in dev) is only ~72-82ms in prod — not
actionable. Measurement did its job.
