fix(desktop): latch an expired remote OAuth session on the first confirmed 401 instead of flickering the Sign in overlay (#95701)
Cold-launching against a gated remote gateway whose stored session has been
invalidated server-side alternated between the connecting state and the
recovery overlay; the Sign in button was only intermittently clickable.
Root cause: fetchJson() built a bare Error("401: ...") for HTTP failures on
the native-bearer path, dropping res.statusCode. Every downstream classifier
is shape-based (isGatewayAuthRejection, isServerSideHttpError, the
ensureNativeAccessToken 401 check), so the confirmed rejection looked like a
transport blip: withTransientRetries hammered it, gatewayTicketFailure used
the transport copy, startHermes tagged the boot retryable, and the renderer's
bounded boot-retry loop re-emitted running:true over the overlay on every
attempt. Separately, gatewayTicketFailure only ever set needsOauthLogin,
which isReauthRequiredError ignores, so even a structured 401 from the cookie
path never latched.
- api-transport: httpStatusError() is the one HTTP-error shape; fetchJson and
fetchPublicJson use it, matching fetchJsonViaOauthSession.
- mintGatewayWsTicket: the gateway never rotates a native bearer server-side
(dashboard_auth/middleware.py), so a bearer 401 gets ONE forced
/auth/native/refresh; a live refresh token retries the mint once, a dead one
drops the stored tokens and the rejection is confirmed.
- gatewayTicketFailure: a confirmed 401/403 is tagged isReauthRequired so
startHermes latches it and marks the boot non-retryable.
- startHermes: latches are set before the first await in the failure path;
updateBootProgress holds every update that is not a re-emit of the latched
failure until a recovery path clears the latch.
Tests: composition + main.ts source pins (remote-reauth-latch.test.ts), unit
coverage for the new helpers, and a Playwright e2e that boots the real app
against a fake gateway with a dead session and proves the overlay latches
once (one mint, one refresh, retryable:false, Sign in stays clickable).
This commit is contained in:
285
apps/desktop/e2e/remote-reauth-latch.spec.ts
Normal file
285
apps/desktop/e2e/remote-reauth-latch.spec.ts
Normal file
@@ -0,0 +1,285 @@
|
||||
/**
|
||||
* E2E regression for issue #95701: a remote OAuth gateway whose stored
|
||||
* session has been invalidated server-side must boot into ONE stable
|
||||
* recovery overlay with a clickable Sign in action — not alternate between
|
||||
* the connecting state and the overlay while the renderer's transient-boot
|
||||
* retry loop re-drives a rejection that can never self-heal.
|
||||
*
|
||||
* The shape under test is the RFC 8252 native-bearer flow: the desktop holds
|
||||
* a locally-unexpired access token, but the gateway rejects it (restarted
|
||||
* with new signing state), and the refresh token is dead too. The gateway
|
||||
* never rotates a native bearer server-side (see
|
||||
* hermes_cli/dashboard_auth/middleware.py), so the desktop must try ONE
|
||||
* rotation via /auth/native/refresh and then treat the rejection as
|
||||
* confirmed: latch reauth, mark the boot non-retryable, hold the overlay.
|
||||
*
|
||||
* Contract asserted here, end to end through the real Electron main process
|
||||
* and renderer:
|
||||
*
|
||||
* 1. The recovery overlay appears and then stays continuously visible —
|
||||
* no sample over a window longer than the renderer's boot-retry backoff
|
||||
* sees it hidden.
|
||||
* 2. The gateway saw exactly one ticket mint and exactly one refresh
|
||||
* attempt: the rejection was confirmed once, not hammered.
|
||||
* 3. The boot-progress snapshot the renderer reads is the non-retryable,
|
||||
* structured-401 verdict the reauth latch depends on.
|
||||
* 4. The overlay offers the remote sign-in action (dead tokens were dropped,
|
||||
* so Settings no longer reports the session as connected).
|
||||
*
|
||||
* Prerequisite: `npm run build` must have been run so dist/ exists.
|
||||
*/
|
||||
|
||||
import * as fs from 'node:fs'
|
||||
import * as http from 'node:http'
|
||||
import type { AddressInfo } from 'node:net'
|
||||
import * as path from 'node:path'
|
||||
|
||||
import { buildAppEnv, createSandbox, launchDesktop, type Sandbox, waitForBootFailure } from './fixtures'
|
||||
import { allowErrorBanners, type ElectronApplication, expect, type Page, test } from './test'
|
||||
|
||||
// Longer than the renderer's first boot-retry delay (2s base, full jitter,
|
||||
// #82679) by a wide margin, so a retry loop that hides the overlay cannot
|
||||
// slip between two samples.
|
||||
const STABILITY_WINDOW_MS = 15_000
|
||||
const SAMPLE_INTERVAL_MS = 100
|
||||
|
||||
interface RecordedRequest {
|
||||
method: string
|
||||
path: string
|
||||
bearer: boolean
|
||||
}
|
||||
|
||||
interface ExpiredSessionGateway {
|
||||
url: string
|
||||
requests: RecordedRequest[]
|
||||
count: (method: string, pathname: string) => number
|
||||
close: () => Promise<void>
|
||||
}
|
||||
|
||||
/**
|
||||
* A gated gateway whose every authenticated session is dead. Public
|
||||
* discovery works (status advertises the native flow, providers are
|
||||
* OAuth-style so the desktop keeps the strict guard), but the ticket mint
|
||||
* and the native refresh both answer the middleware's structured
|
||||
* `session_expired` 401. Every request is recorded so the test can prove the
|
||||
* rejection was confirmed exactly once.
|
||||
*/
|
||||
async function startExpiredSessionGateway(): Promise<ExpiredSessionGateway> {
|
||||
const requests: RecordedRequest[] = []
|
||||
|
||||
const server = http.createServer((req, res) => {
|
||||
const url = new URL(req.url ?? '/', 'http://127.0.0.1')
|
||||
const authorization = req.headers.authorization
|
||||
|
||||
requests.push({
|
||||
method: req.method ?? 'GET',
|
||||
path: url.pathname,
|
||||
bearer: typeof authorization === 'string' && authorization.startsWith('Bearer '),
|
||||
})
|
||||
|
||||
const json = (status: number, body: unknown): void => {
|
||||
res.writeHead(status, { 'Content-Type': 'application/json' })
|
||||
res.end(JSON.stringify(body))
|
||||
}
|
||||
|
||||
// Drain the body before answering so a POST never sees an early close.
|
||||
req.resume()
|
||||
req.on('end', () => {
|
||||
if (url.pathname === '/api/status') {
|
||||
json(200, { auth_required: true, auth_flows: ['native_pkce'], ok: true, version: '0.0.0-e2e-fake' })
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
if (url.pathname === '/api/auth/providers') {
|
||||
json(200, { providers: [{ name: 'portal', supports_password: false }] })
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
if (url.pathname === '/api/auth/ws-ticket' || url.pathname === '/auth/native/refresh') {
|
||||
json(401, {
|
||||
error: 'session_expired',
|
||||
detail: 'Unauthorized',
|
||||
reason: 'invalid_or_expired_session',
|
||||
login_url: '/login',
|
||||
})
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
if (url.pathname.startsWith('/api/')) {
|
||||
json(401, { error: 'unauthenticated', detail: 'Unauthorized', reason: 'no_cookie', login_url: '/login' })
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
json(404, { detail: 'not found' })
|
||||
})
|
||||
})
|
||||
|
||||
// No WebSocket can ever be authenticated here; refuse the upgrade fast.
|
||||
server.on('upgrade', (_req, socket) => {
|
||||
socket.destroy()
|
||||
})
|
||||
|
||||
await new Promise<void>(resolve => server.listen(0, '127.0.0.1', resolve))
|
||||
|
||||
const { port } = server.address() as AddressInfo
|
||||
|
||||
return {
|
||||
close: () =>
|
||||
new Promise<void>(resolve => {
|
||||
server.closeAllConnections?.()
|
||||
server.close(() => resolve())
|
||||
}),
|
||||
count: (method, pathname) => requests.filter(r => r.method === method && r.path === pathname).length,
|
||||
requests,
|
||||
url: `http://127.0.0.1:${port}`,
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Seed the sandbox the way a previously signed-in desktop leaves it: a saved
|
||||
* remote OAuth connection and a stored native token set whose access token
|
||||
* has NOT expired locally (so the desktop presents it as-is). The blob uses
|
||||
* the plain encoding the desktop's secret reader accepts verbatim — an OS
|
||||
* keyring is not available on the CI runner and is irrelevant to the contract.
|
||||
*/
|
||||
function seedExpiredNativeSession(sandbox: Sandbox, gatewayUrl: string): void {
|
||||
fs.writeFileSync(
|
||||
path.join(sandbox.userDataDir, 'connection.json'),
|
||||
JSON.stringify({ mode: 'remote', remote: { url: gatewayUrl, authMode: 'oauth' }, profiles: {} }, null, 2),
|
||||
{ encoding: 'utf8', mode: 0o600 },
|
||||
)
|
||||
|
||||
const tokens = {
|
||||
accessToken: 'e2e-stale-access-token',
|
||||
refreshToken: 'e2e-stale-refresh-token',
|
||||
expiresAt: Math.floor(Date.now() / 1000) + 3600,
|
||||
provider: 'portal',
|
||||
userId: 'e2e-user',
|
||||
}
|
||||
|
||||
fs.writeFileSync(
|
||||
path.join(sandbox.userDataDir, 'native-oauth-tokens.json'),
|
||||
JSON.stringify({ [gatewayUrl]: { encoding: 'plain', value: JSON.stringify(tokens) } }, null, 2),
|
||||
{ encoding: 'utf8', mode: 0o600 },
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Sample the recovery overlay inside the page at a fixed cadence and report
|
||||
* every sample in which it was NOT showing. The overlay is identified by its
|
||||
* recovery actions, the same signal waitForBootFailure keys on; while the
|
||||
* renderer re-drives a boot the overlay is unmounted (`running: true`) and
|
||||
* the connecting surface paints instead.
|
||||
*/
|
||||
async function sampleOverlayStability(
|
||||
page: Page,
|
||||
windowMs: number,
|
||||
intervalMs: number,
|
||||
): Promise<{ samples: number; hidden: number; flips: number }> {
|
||||
return page.evaluate(
|
||||
async ({ windowMs: w, intervalMs: i }) => {
|
||||
const overlayShowing = (): boolean => {
|
||||
const text = document.body.textContent ?? ''
|
||||
|
||||
return (
|
||||
text.includes('Sign out & sign in') ||
|
||||
text.includes('Gateway settings') ||
|
||||
text.includes('Use local gateway') ||
|
||||
text.includes('Repair install')
|
||||
)
|
||||
}
|
||||
|
||||
const started = Date.now()
|
||||
let samples = 0
|
||||
let hidden = 0
|
||||
let flips = 0
|
||||
let last = overlayShowing()
|
||||
|
||||
while (Date.now() - started < w) {
|
||||
await new Promise(resolve => setTimeout(resolve, i))
|
||||
const showing = overlayShowing()
|
||||
samples += 1
|
||||
|
||||
if (!showing) {
|
||||
hidden += 1
|
||||
}
|
||||
|
||||
if (showing !== last) {
|
||||
flips += 1
|
||||
last = showing
|
||||
}
|
||||
}
|
||||
|
||||
return { samples, hidden, flips }
|
||||
},
|
||||
{ windowMs, intervalMs },
|
||||
)
|
||||
}
|
||||
|
||||
let gateway: ExpiredSessionGateway | null = null
|
||||
let sandbox: Sandbox | null = null
|
||||
let app: ElectronApplication | null = null
|
||||
|
||||
test.afterAll(async () => {
|
||||
await app?.close().catch(() => undefined)
|
||||
await gateway?.close()
|
||||
sandbox?.cleanup()
|
||||
app = null
|
||||
gateway = null
|
||||
sandbox = null
|
||||
})
|
||||
|
||||
test.describe('remote OAuth session rejected at cold boot (#95701)', () => {
|
||||
test.beforeEach(() => {
|
||||
// The boot deliberately fails; the "Desktop boot failed" toast is expected.
|
||||
allowErrorBanners()
|
||||
})
|
||||
|
||||
test('the recovery overlay latches once and its Sign in action stays put', async () => {
|
||||
gateway = await startExpiredSessionGateway()
|
||||
sandbox = createSandbox('reauth-latch')
|
||||
seedExpiredNativeSession(sandbox, gateway.url)
|
||||
|
||||
const launched = await launchDesktop(buildAppEnv(sandbox))
|
||||
app = launched.app
|
||||
const page = launched.page
|
||||
|
||||
await waitForBootFailure(page, 60_000)
|
||||
|
||||
// 1. Stability: once the overlay is up it never blinks out again. A
|
||||
// transient-boot retry (`running: true` re-emitted over the failure)
|
||||
// unmounts it, which is exactly the flicker the issue describes.
|
||||
const stability = await sampleOverlayStability(page, STABILITY_WINDOW_MS, SAMPLE_INTERVAL_MS)
|
||||
expect(stability.samples).toBeGreaterThan(50)
|
||||
expect(stability, 'recovery overlay must stay visible for the whole window').toMatchObject({ hidden: 0, flips: 0 })
|
||||
|
||||
// 2. The rejection was confirmed exactly once: one bearer ticket mint,
|
||||
// one forced rotation attempt, and no further boot re-entry hammering
|
||||
// the dead session for the rest of the window.
|
||||
const ticketMints = gateway.requests.filter(r => r.method === 'POST' && r.path === '/api/auth/ws-ticket')
|
||||
expect(ticketMints, 'exactly one ws-ticket mint').toHaveLength(1)
|
||||
expect(ticketMints[0]?.bearer, 'the mint presented the stored native bearer').toBe(true)
|
||||
expect(gateway.count('POST', '/auth/native/refresh'), 'exactly one native refresh attempt').toBe(1)
|
||||
|
||||
// 3. The renderer-visible verdict is the structured, non-retryable one.
|
||||
const snapshot = await page.evaluate(() =>
|
||||
(window as unknown as { hermesDesktop: { getBootProgress: () => Promise<Record<string, unknown>> } })
|
||||
.hermesDesktop.getBootProgress(),
|
||||
)
|
||||
expect(snapshot).toMatchObject({ running: false, retryable: false, statusCode: 401 })
|
||||
// A session that DID exist reads as expired, not as never signed in — the
|
||||
// dead tokens are dropped on the way out, so this copy must be chosen
|
||||
// from the pre-mint state.
|
||||
expect(String(snapshot.error)).toMatch(/session has expired/i)
|
||||
|
||||
// 4. The overlay offers remote sign-in — the dead tokens were dropped, so
|
||||
// the connection no longer reads as "connected" and the reauth branch
|
||||
// renders instead of the local Retry/Repair buttons.
|
||||
await expect(page.getByRole('button', { name: /sign out & sign in/i })).toBeVisible()
|
||||
await expect(page.getByRole('button', { name: /sign out & sign in/i })).toBeEnabled()
|
||||
})
|
||||
})
|
||||
@@ -19,6 +19,7 @@ import { afterAll, describe, expect, it } from 'vitest'
|
||||
import {
|
||||
destroyKeepaliveAgents,
|
||||
downloadAgentFor,
|
||||
httpStatusError,
|
||||
isIdempotentMethod,
|
||||
isTransientTransportError,
|
||||
jsonAgentFor,
|
||||
@@ -381,3 +382,25 @@ describe('live: POST reset after server-side processing', () => {
|
||||
}
|
||||
}, 20_000)
|
||||
})
|
||||
|
||||
describe('httpStatusError', () => {
|
||||
it('carries the integer statusCode downstream classifiers read, plus the legacy "<status>: <body>" message', () => {
|
||||
const err = httpStatusError(401, '{"error":"session_expired"}', 'Unauthorized')
|
||||
|
||||
expect(err).toBeInstanceOf(Error)
|
||||
expect(err.statusCode).toBe(401)
|
||||
expect(err.message).toBe('401: {"error":"session_expired"}')
|
||||
})
|
||||
|
||||
it('falls back to the status message, then to an empty detail, when the body is empty', () => {
|
||||
expect(httpStatusError(503, '', 'Service Unavailable').message).toBe('503: Service Unavailable')
|
||||
expect(httpStatusError(403, '', undefined).message).toBe('403: ')
|
||||
expect(httpStatusError(403, '', undefined).statusCode).toBe(403)
|
||||
})
|
||||
|
||||
it('normalizes a missing/invalid status to 500 exactly like the `res.statusCode || 500` guard', () => {
|
||||
expect(httpStatusError(undefined, 'boom').statusCode).toBe(500)
|
||||
expect(httpStatusError(undefined, 'boom').message).toBe('500: boom')
|
||||
expect(httpStatusError(0, 'boom').statusCode).toBe(500)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -167,9 +167,32 @@ async function withRetry(makeAttempt, options: any = {}) {
|
||||
throw lastError
|
||||
}
|
||||
|
||||
/**
|
||||
* The one error shape the REST helpers throw for an HTTP >= 400 response.
|
||||
*
|
||||
* `statusCode` is the structured contract every downstream classifier reads —
|
||||
* isGatewayAuthRejection (401/403 → reauth, never retried), isServerSideHttpError
|
||||
* (502/503/504 → Cloud-down), ensureNativeAccessToken's dead-refresh-token
|
||||
* check — and the "<status>: <body>" message keeps the legacy prefix readers
|
||||
* working. fetchJson used to build a bare Error here, so a native-bearer 401
|
||||
* reached the boot path as an anonymous transport failure: it was retried,
|
||||
* then classified as transient, and the renderer's boot-retry loop flickered
|
||||
* the Sign in overlay away (#95701). Shared by fetchJson, fetchPublicJson and
|
||||
* the OAuth-session fetch so the three paths cannot drift apart again.
|
||||
*/
|
||||
function httpStatusError(statusCode, text, statusMessage?) {
|
||||
const status = Number.isInteger(statusCode) && statusCode > 0 ? statusCode : 500
|
||||
const detail = String(text || statusMessage || '')
|
||||
const error: any = new Error(`${status}: ${detail}`)
|
||||
error.statusCode = status
|
||||
|
||||
return error
|
||||
}
|
||||
|
||||
export {
|
||||
destroyKeepaliveAgents,
|
||||
downloadAgentFor,
|
||||
httpStatusError,
|
||||
isIdempotentMethod,
|
||||
isTransientTransportError,
|
||||
jsonAgentFor,
|
||||
|
||||
@@ -220,10 +220,13 @@ test('a credentialed 401 fails fast for reauth instead of reporting a dead sessi
|
||||
assert.deepEqual(calls, [['probe', 'https://gateway.example/api/health']])
|
||||
})
|
||||
|
||||
test('unsigned OAuth is a terminal reauth failure; needsOauthLogin alone is not', () => {
|
||||
test('unsigned OAuth is a terminal reauth failure; a bare needsOauthLogin hint is not', () => {
|
||||
// The unsigned-in throw must set isReauthRequired so startHermes latches.
|
||||
// needsOauthLogin alone (ticket 401/403) stays a Sign-in hint, not a latch —
|
||||
// a lapsed AT cookie can still rotate from a live RT on the next mint.
|
||||
// A bare needsOauthLogin (the IPC-shaped hint) stays Sign-in copy, not a
|
||||
// latch. A CONFIRMED ticket 401/403 is different: the gateway has already
|
||||
// tried the AT/RT rotation (cookie) or the desktop has forced one (native)
|
||||
// before that rejection reaches gatewayTicketFailure, which tags it
|
||||
// isReauthRequired itself (#95701).
|
||||
const unsigned = makeUnsignedOauthError() as any
|
||||
|
||||
assert.equal(unsigned.needsOauthLogin, true)
|
||||
|
||||
@@ -206,8 +206,10 @@ export function makeReauthRequiredError(detail?: string): Error {
|
||||
|
||||
/**
|
||||
* No native token and no live cookie: boot cannot self-heal. Must carry
|
||||
* `isReauthRequired` so startHermes latches; `needsOauthLogin` alone only
|
||||
* drives Sign in copy and would retry after #88070, hiding the overlay.
|
||||
* `isReauthRequired` so startHermes latches; a bare `needsOauthLogin` (the
|
||||
* IPC-shaped hint) only drives Sign in copy and would retry after #88070,
|
||||
* hiding the overlay. A confirmed ticket-mint 401/403 carries the same tag
|
||||
* (see gatewayTicketFailure, #95701).
|
||||
*/
|
||||
export function makeUnsignedOauthError(): Error {
|
||||
const error = new Error(REMOTE_UNSIGNED_OAUTH_MESSAGE) as any
|
||||
|
||||
@@ -6,6 +6,7 @@ import { isReauthRequiredError, makeUnsignedOauthError } from './backend-health'
|
||||
import {
|
||||
isHostKeyChangedBootFailure,
|
||||
isRetryableRemoteBootFailure,
|
||||
shouldHoldBootProgressForReauth,
|
||||
shouldLatchBackendStartFailure,
|
||||
shouldLatchHostKeyChangedFailure,
|
||||
shouldLatchRemoteReauthFailure
|
||||
@@ -70,8 +71,11 @@ test('a CONFIRMED reauth rejection is never auto-retried (missing capability, no
|
||||
assert.equal(isRetryableRemoteBootFailure({ attemptedRemote: true, isReauth: true }), false)
|
||||
})
|
||||
|
||||
test('unsigned OAuth latches and is never auto-retried; needsOauthLogin alone still retries', () => {
|
||||
test('unsigned OAuth latches and is never auto-retried; a bare needsOauthLogin hint still retries', () => {
|
||||
// Production composition in startHermes: isReauth = isReauthRequiredError(error).
|
||||
// A bare `{ needsOauthLogin: true }` is the IPC-shaped hint, not a confirmed
|
||||
// rejection; gatewayTicketFailure tags a confirmed 401/403 with
|
||||
// isReauthRequired itself (#95701, see remote-reauth-latch.test.ts).
|
||||
const unsigned = isReauthRequiredError(makeUnsignedOauthError())
|
||||
const ticketHint = isReauthRequiredError({ needsOauthLogin: true })
|
||||
|
||||
@@ -147,3 +151,25 @@ test('every remote failure picks exactly one path: retry, reauth latch, or host-
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
test('FIX #95701: while a reauth rejection is latched, only re-emits of that failure reach the renderer', () => {
|
||||
const latched = 'Your remote gateway session has expired. Sign in again.'
|
||||
|
||||
// The latched failure's own (re-)emit passes — it carries retryable:false.
|
||||
assert.equal(shouldHoldBootProgressForReauth(latched, { error: latched }), false)
|
||||
|
||||
// Anything that would lift the overlay is held: a running phase from an
|
||||
// attempt already in flight when the latch closed, a cleared error, or a
|
||||
// sibling failure that would flip retryable back on.
|
||||
assert.equal(shouldHoldBootProgressForReauth(latched, { error: null }), true)
|
||||
assert.equal(shouldHoldBootProgressForReauth(latched, {}), true)
|
||||
assert.equal(shouldHoldBootProgressForReauth(latched, { error: 'Could not reach the remote Hermes gateway' }), true)
|
||||
})
|
||||
|
||||
test('FIX #95701: with no reauth latch every boot-progress update flows as before', () => {
|
||||
for (const latch of [null, undefined, '']) {
|
||||
assert.equal(shouldHoldBootProgressForReauth(latch, { error: null }), false)
|
||||
assert.equal(shouldHoldBootProgressForReauth(latch, {}), false)
|
||||
assert.equal(shouldHoldBootProgressForReauth(latch, { error: 'Desktop boot failed: spawn ENOENT' }), false)
|
||||
}
|
||||
})
|
||||
|
||||
@@ -138,3 +138,35 @@ export function shouldLatchHostKeyChangedFailure(context: RemoteBootRetryContext
|
||||
export function isRetryableRemoteBootFailure(context: RemoteBootRetryContext): boolean {
|
||||
return context.attemptedRemote && !context.isReauth && context.isHostKeyChanged !== true
|
||||
}
|
||||
|
||||
export interface BootProgressUpdateLike {
|
||||
/** The failure text an error update carries; absent/null on progress updates. */
|
||||
error?: unknown
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a boot-progress update must be dropped because a CONFIRMED remote
|
||||
* reauth rejection is latched.
|
||||
*
|
||||
* `latchedMessage` is the message of the latched reauth failure (null when
|
||||
* nothing is latched). While it is set, the recovery overlay with its Sign in
|
||||
* button is the only correct surface until the user signs in, and any update
|
||||
* that is not a re-emit of that same failure would lift it: a `running: true`
|
||||
* phase or a cleared error from an attempt that was already in flight when the
|
||||
* latch closed, or an unrelated failure from a sibling caller (which would
|
||||
* also swap the retryable verdict back to true and re-arm the renderer's
|
||||
* boot-retry loop — the flicker of #95701). Re-emits of the latched failure
|
||||
* pass so the non-retryable verdict is never lost. Every recovery path
|
||||
* (reset, repair, apply-config, confirmed sign-in) clears the latch BEFORE it
|
||||
* re-drives boot, so this never holds a legitimate restart.
|
||||
*/
|
||||
export function shouldHoldBootProgressForReauth(
|
||||
latchedMessage: string | null | undefined,
|
||||
update: BootProgressUpdateLike
|
||||
): boolean {
|
||||
if (!latchedMessage) {
|
||||
return false
|
||||
}
|
||||
|
||||
return update.error !== latchedMessage
|
||||
}
|
||||
|
||||
@@ -1342,3 +1342,34 @@ test('OAuth ticket-mint 401 stays on the reauth path (never Cloud-down)', () =>
|
||||
assert.equal((wrapped as any).needsOauthLogin, true)
|
||||
assert.equal((wrapped as any).statusCode, 401)
|
||||
})
|
||||
|
||||
test('FIX #95701: a confirmed 401/403 ticket rejection is tagged isReauthRequired so startHermes latches it', () => {
|
||||
for (const statusCode of [401, 403]) {
|
||||
const source = Object.assign(new Error(`${statusCode}: rejected`), { statusCode })
|
||||
const wrapped = gatewayTicketFailure(source, 'auth copy', 'transport copy') as any
|
||||
|
||||
assert.equal(wrapped.message, 'auth copy')
|
||||
assert.equal(wrapped.needsOauthLogin, true)
|
||||
assert.equal(wrapped.isReauthRequired, true, `a ${statusCode} mint rejection cannot self-heal`)
|
||||
assert.equal(wrapped.statusCode, statusCode)
|
||||
}
|
||||
|
||||
// A pre-tagged rejection (needsOauthLogin from an upstream classifier) is
|
||||
// confirmed the same way.
|
||||
const tagged = gatewayTicketFailure({ needsOauthLogin: true }, 'auth copy', 'transport copy') as any
|
||||
assert.equal(tagged.isReauthRequired, true)
|
||||
})
|
||||
|
||||
test('FIX #95701: transport and server failures at the ticket mint stay retryable — never reauth', () => {
|
||||
for (const source of [
|
||||
Object.assign(new Error('503: unavailable'), { statusCode: 503 }),
|
||||
new Error('Timed out connecting to Hermes backend after 8000ms'),
|
||||
Object.assign(new Error('read ECONNRESET'), { code: 'ECONNRESET' })
|
||||
]) {
|
||||
const wrapped = gatewayTicketFailure(source, 'auth copy', 'transport copy') as any
|
||||
|
||||
assert.equal(wrapped.message, 'transport copy')
|
||||
assert.equal(wrapped.needsOauthLogin, undefined)
|
||||
assert.equal(wrapped.isReauthRequired, undefined)
|
||||
}
|
||||
})
|
||||
|
||||
@@ -132,6 +132,15 @@ function gatewayTicketFailure(error, authMessage, transportMessage) {
|
||||
|
||||
if (needsOauthLogin) {
|
||||
;(err as any).needsOauthLogin = true
|
||||
// A rejected ticket mint is a CONFIRMED reauth failure, not a hint. The
|
||||
// cookie path only sees a 401/403 after the gateway's transparent AT/RT
|
||||
// rotation has already failed, and the native-bearer path only after
|
||||
// mintGatewayWsTicket's forced /auth/native/refresh has. Nothing will
|
||||
// change until the user signs in, so tag it the way startHermes latches
|
||||
// (isReauthRequiredError): the boot is marked non-retryable and the
|
||||
// overlay's Sign in button stops flickering away under the renderer's
|
||||
// transient-boot retry loop (#95701).
|
||||
;(err as any).isReauthRequired = true
|
||||
}
|
||||
|
||||
// Preserve structured HTTP context when the source error carried an integer
|
||||
|
||||
@@ -30,7 +30,7 @@ import {
|
||||
} from 'electron'
|
||||
|
||||
import { classifyActiveRuntime } from './active-runtime-state'
|
||||
import { destroyKeepaliveAgents, downloadAgentFor, jsonAgentFor, withRetry } from './api-transport'
|
||||
import { destroyKeepaliveAgents, downloadAgentFor, httpStatusError, jsonAgentFor, withRetry } from './api-transport'
|
||||
import { appIconCandidates, resolveAppIcon } from './app-icon'
|
||||
import { stopBackendChild as stopBackendChildImpl, stopBackendTreesForUpdate } from './backend-child'
|
||||
import {
|
||||
@@ -68,6 +68,7 @@ import { isPidAliveWindows, waitForBackendRelease } from './backend-release-gate
|
||||
import {
|
||||
isHostKeyChangedBootFailure,
|
||||
isRetryableRemoteBootFailure,
|
||||
shouldHoldBootProgressForReauth,
|
||||
shouldLatchBackendStartFailure,
|
||||
shouldLatchHostKeyChangedFailure,
|
||||
shouldLatchRemoteReauthFailure
|
||||
@@ -259,7 +260,8 @@ import {
|
||||
resolveGatedDownloadAuth,
|
||||
resolveJsonBody,
|
||||
resolveOauthRestAuth,
|
||||
resolveReadinessProbeAuth
|
||||
resolveReadinessProbeAuth,
|
||||
shouldRotateNativeTokenAfterRejection
|
||||
} from './native-auth-decisions'
|
||||
import {
|
||||
nativeRefreshUrl,
|
||||
@@ -2237,7 +2239,28 @@ function abandonFirstRunSetupChoiceForRemoteApply() {
|
||||
return resumedGatedConnection
|
||||
}
|
||||
|
||||
// The latched reauth failure whose hold has already been logged, so a burst of
|
||||
// dropped updates from one in-flight sibling attempt logs once, not per event.
|
||||
let bootProgressHeldFor = null
|
||||
|
||||
function updateBootProgress(update, options: { allowDecrease?: boolean } = {}) {
|
||||
// A latched CONFIRMED reauth rejection owns the boot surface until a
|
||||
// recovery path clears it. Updates that are not a re-emit of that failure —
|
||||
// a running:true phase or cleared error from an attempt already in flight
|
||||
// when the latch closed, or an unrelated sibling failure that would flip
|
||||
// retryable back on — must not reach the renderer, or the overlay's Sign in
|
||||
// button flickers away again (#95701).
|
||||
if (shouldHoldBootProgressForReauth(remoteReauthFailure ? remoteReauthFailure.message : null, update)) {
|
||||
if (bootProgressHeldFor !== remoteReauthFailure) {
|
||||
bootProgressHeldFor = remoteReauthFailure
|
||||
rememberLog('[boot] remote reauth latched: holding the recovery overlay against a stale boot-progress update')
|
||||
}
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
bootProgressHeldFor = null
|
||||
|
||||
const nextProgressRaw =
|
||||
typeof update.progress === 'number' ? clampBootProgress(update.progress) : bootProgressState.progress
|
||||
|
||||
@@ -5374,7 +5397,7 @@ function fetchJson(url, token, options: any = {}) {
|
||||
const text = Buffer.concat(chunks).toString('utf8')
|
||||
|
||||
if ((res.statusCode || 500) >= 400) {
|
||||
reject(new Error(`${res.statusCode}: ${text || res.statusMessage}`))
|
||||
reject(httpStatusError(res.statusCode, text, res.statusMessage))
|
||||
|
||||
return
|
||||
}
|
||||
@@ -5540,7 +5563,7 @@ function fetchPublicJson(url, options: any = {}) {
|
||||
const text = Buffer.concat(chunks).toString('utf8')
|
||||
|
||||
if ((res.statusCode || 500) >= 400) {
|
||||
reject(new Error(`${res.statusCode}: ${text || res.statusMessage}`))
|
||||
reject(httpStatusError(res.statusCode, text, res.statusMessage))
|
||||
|
||||
return
|
||||
}
|
||||
@@ -7949,14 +7972,20 @@ function postJsonNoAuth(url: string, body: unknown, opts: any = {}) {
|
||||
// Return a valid native access token for baseUrl, refreshing via
|
||||
// /auth/native/refresh if the stored one is at/near expiry. Returns null when
|
||||
// there are no tokens or the refresh is terminally rejected (caller re-logins).
|
||||
async function ensureNativeAccessToken(baseUrl: string): Promise<string | null> {
|
||||
// `forceRefresh` rotates even a locally-unexpired access token: the gateway
|
||||
// never rotates a native bearer server-side, so after the gate rejects one
|
||||
// the desktop must run the refresh itself before the rejection is confirmed.
|
||||
async function ensureNativeAccessToken(
|
||||
baseUrl: string,
|
||||
options: { forceRefresh?: boolean } = {}
|
||||
): Promise<string | null> {
|
||||
const tokens = _loadNativeTokens(baseUrl)
|
||||
|
||||
if (!tokens) {
|
||||
return null
|
||||
}
|
||||
|
||||
if (!tokenNeedsRefresh(tokens, Math.floor(Date.now() / 1000))) {
|
||||
if (!options.forceRefresh && !tokenNeedsRefresh(tokens, Math.floor(Date.now() / 1000))) {
|
||||
return tokens.accessToken
|
||||
}
|
||||
|
||||
@@ -8286,20 +8315,29 @@ async function mintGatewayWsTicket(baseUrl, headers = {}) {
|
||||
const nativeAt = await ensureNativeAccessToken(baseUrl).catch(() => null)
|
||||
|
||||
if (nativeAt) {
|
||||
const body = (await fetchJson(`${baseUrl}/api/auth/ws-ticket`, null, {
|
||||
method: 'POST',
|
||||
timeoutMs: 8_000,
|
||||
bearer: nativeAt,
|
||||
headers
|
||||
})) as any
|
||||
try {
|
||||
return await mintGatewayWsTicketWithBearer(baseUrl, nativeAt, headers)
|
||||
} catch (error) {
|
||||
// The gate rejected a bearer the desktop still considered valid. It
|
||||
// never rotates a native access token server-side (only cookie
|
||||
// sessions get the transparent refresh; the native flow is told to
|
||||
// call /auth/native/refresh itself), so run ONE forced rotation here:
|
||||
// a live refresh token yields a fresh bearer and the mint is retried
|
||||
// once; a dead one drops the stored set, and the original 401 stands
|
||||
// as a CONFIRMED rejection that latches into the Sign in overlay
|
||||
// instead of being replayed by every boot retry (#95701).
|
||||
if (!shouldRotateNativeTokenAfterRejection(error)) {
|
||||
throw error
|
||||
}
|
||||
|
||||
const ticket = body?.ticket
|
||||
const rotatedAt = await ensureNativeAccessToken(baseUrl, { forceRefresh: true }).catch(() => null)
|
||||
|
||||
if (!ticket || typeof ticket !== 'string') {
|
||||
throw new Error('Gateway did not return a WS ticket.')
|
||||
if (!rotatedAt || rotatedAt === nativeAt) {
|
||||
throw error
|
||||
}
|
||||
|
||||
return await mintGatewayWsTicketWithBearer(baseUrl, rotatedAt, headers)
|
||||
}
|
||||
|
||||
return ticket
|
||||
}
|
||||
|
||||
const body = (await fetchJsonViaOauthSession(`${baseUrl}/api/auth/ws-ticket`, {
|
||||
@@ -8318,6 +8356,26 @@ async function mintGatewayWsTicket(baseUrl, headers = {}) {
|
||||
})
|
||||
}
|
||||
|
||||
// One bearer-authenticated ticket mint. Kept separate from the rotation
|
||||
// decision in mintGatewayWsTicket so the retry-after-refresh leg presents the
|
||||
// rotated bearer through exactly the same request shape as the first attempt.
|
||||
async function mintGatewayWsTicketWithBearer(baseUrl, bearer, headers = {}) {
|
||||
const body = (await fetchJson(`${baseUrl}/api/auth/ws-ticket`, null, {
|
||||
method: 'POST',
|
||||
timeoutMs: 8_000,
|
||||
bearer,
|
||||
headers
|
||||
})) as any
|
||||
|
||||
const ticket = body?.ticket
|
||||
|
||||
if (!ticket || typeof ticket !== 'string') {
|
||||
throw new Error('Gateway did not return a WS ticket.')
|
||||
}
|
||||
|
||||
return ticket
|
||||
}
|
||||
|
||||
// Build a fresh WS URL for the *current* connection. Critical for reconnects:
|
||||
// OAuth WS tickets are single-use with a ~30s TTL, so the ticket baked into
|
||||
// the cached connection's wsUrl is stale on the second connect. The renderer
|
||||
@@ -10086,6 +10144,12 @@ async function buildRemoteConnection(
|
||||
throw makeUnsignedOauthError()
|
||||
}
|
||||
|
||||
// Snapshot BEFORE the mint: a confirmed native rejection drops the dead
|
||||
// token set on its way out (mintGatewayWsTicket's forced rotation), and
|
||||
// the failure copy must still say "session expired" for a session that
|
||||
// did exist — "not signed in" is for a jar that never held one.
|
||||
const hadNativeSession = hasNativeSession(baseUrl)
|
||||
|
||||
let ticket
|
||||
|
||||
try {
|
||||
@@ -10106,7 +10170,7 @@ async function buildRemoteConnection(
|
||||
|
||||
throw gatewayTicketFailure(
|
||||
error,
|
||||
oauthTicketFailureAuthMessage(hasNativeSession(baseUrl)),
|
||||
oauthTicketFailureAuthMessage(hadNativeSession),
|
||||
'Could not reach the remote Hermes gateway while refreshing its WebSocket ticket. Try reconnecting.'
|
||||
)
|
||||
}
|
||||
@@ -13287,14 +13351,15 @@ async function startHermes() {
|
||||
|
||||
const failedProcess = backendConnectionState.invalidate()
|
||||
stopBackendChild(failedProcess)
|
||||
await waitForBackendExit(failedProcess)
|
||||
|
||||
if (error instanceof FirstRunSetupResetError) {
|
||||
await waitForBackendExit(failedProcess)
|
||||
throw error
|
||||
}
|
||||
|
||||
const message = error instanceof Error ? error.message : String(error)
|
||||
const hostKeyChanged = isHostKeyChangedBootFailure(error)
|
||||
const isReauth = isReauthRequiredError(error)
|
||||
|
||||
// Carry structured Cloud-down metadata through the boot-progress / IPC
|
||||
// boundary when present, so the renderer overlay can key on it rather than
|
||||
@@ -13329,10 +13394,19 @@ async function startHermes() {
|
||||
|
||||
// A confirmed reauth rejection latches separately: it can't self-heal, and
|
||||
// leaving it unlatched hides the overlay's "Sign in" button on every retry.
|
||||
if (shouldLatchRemoteReauthFailure({ attemptedRemote, isReauth: isReauthRequiredError(error) })) {
|
||||
if (shouldLatchRemoteReauthFailure({ attemptedRemote, isReauth })) {
|
||||
remoteReauthFailure = error instanceof Error ? error : new Error(message)
|
||||
}
|
||||
|
||||
// Every latch above is set BEFORE this first yield back to the event loop.
|
||||
// invalidate() already dropped the shared attempt promise, so a concurrent
|
||||
// getConnection()/startHermes() caller arriving during the exit wait would
|
||||
// otherwise start a brand-new attempt, re-emit running:true over the
|
||||
// failure and re-drive the identical rejection. With the latch in place it
|
||||
// short-circuits on the cached failure instead: the first confirmed
|
||||
// rejection owns the transition into recovery (#95701).
|
||||
await waitForBackendExit(failedProcess)
|
||||
|
||||
updateBootProgress(
|
||||
{
|
||||
error: message,
|
||||
@@ -13347,7 +13421,7 @@ async function startHermes() {
|
||||
// sign-in affordance.
|
||||
retryable: isRetryableRemoteBootFailure({
|
||||
attemptedRemote,
|
||||
isReauth: isReauthRequiredError(error),
|
||||
isReauth,
|
||||
isHostKeyChanged: hostKeyChanged
|
||||
}),
|
||||
running: false,
|
||||
|
||||
@@ -18,7 +18,8 @@ import {
|
||||
resolveGatedDownloadAuth,
|
||||
resolveJsonBody,
|
||||
resolveOauthRestAuth,
|
||||
resolveReadinessProbeAuth
|
||||
resolveReadinessProbeAuth,
|
||||
shouldRotateNativeTokenAfterRejection
|
||||
} from './native-auth-decisions'
|
||||
|
||||
// --- 1. body encoding (guards the double-JSON.stringify 422) ---
|
||||
@@ -171,3 +172,33 @@ test('resolveGatedDownloadAuth uses the session token for token and local modes'
|
||||
assert.deepEqual(resolveGatedDownloadAuth('local', null, 'sess'), { kind: 'token', token: 'sess' })
|
||||
assert.deepEqual(resolveGatedDownloadAuth(undefined, null, null), { kind: 'token', token: null })
|
||||
})
|
||||
|
||||
// --- 7. forced native rotation after a bearer rejection (#95701) ---
|
||||
|
||||
test('shouldRotateNativeTokenAfterRejection: only a structured 401 earns the one forced refresh', () => {
|
||||
// The gate never rotates a native bearer server-side, so a 401 on a
|
||||
// locally-unexpired access token is ambiguous until /auth/native/refresh
|
||||
// has run once.
|
||||
assert.equal(
|
||||
shouldRotateNativeTokenAfterRejection(Object.assign(new Error('401: expired'), { statusCode: 401 })),
|
||||
true
|
||||
)
|
||||
assert.equal(shouldRotateNativeTokenAfterRejection({ statusCode: 401 }), true)
|
||||
})
|
||||
|
||||
test('shouldRotateNativeTokenAfterRejection: 403, 5xx, transport, and anonymous errors never rotate', () => {
|
||||
// 403 is a policy refusal for an identity the gate recognized — a fresh
|
||||
// bearer for the same identity cannot change it.
|
||||
assert.equal(
|
||||
shouldRotateNativeTokenAfterRejection(Object.assign(new Error('403: forbidden'), { statusCode: 403 })),
|
||||
false
|
||||
)
|
||||
assert.equal(shouldRotateNativeTokenAfterRejection(Object.assign(new Error('503: down'), { statusCode: 503 })), false)
|
||||
assert.equal(shouldRotateNativeTokenAfterRejection(Object.assign(new Error('reset'), { code: 'ECONNRESET' })), false)
|
||||
// The pre-fix fetchJson shape: a "401: ..." message with no statusCode says
|
||||
// nothing structured about the credential and must not trigger rotation.
|
||||
assert.equal(shouldRotateNativeTokenAfterRejection(new Error('401: {"error":"session_expired"}')), false)
|
||||
assert.equal(shouldRotateNativeTokenAfterRejection(null), false)
|
||||
assert.equal(shouldRotateNativeTokenAfterRejection(undefined), false)
|
||||
assert.equal(shouldRotateNativeTokenAfterRejection('401'), false)
|
||||
})
|
||||
|
||||
@@ -233,3 +233,24 @@ export function oauthGuardMayHardFail(providers: unknown): boolean {
|
||||
|
||||
return !named.every(provider => provider.supportsPassword)
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a rejected native-bearer request should try ONE forced token
|
||||
* rotation before the rejection counts as confirmed.
|
||||
*
|
||||
* The gateway gate never rotates a native access token server-side — only
|
||||
* cookie sessions get the transparent AT-from-RT refresh; the native flow is
|
||||
* told to call `/auth/native/refresh` itself (dashboard_auth/middleware.py).
|
||||
* So a 401 on a bearer the desktop still considers unexpired is ambiguous
|
||||
* until that refresh has had its say: a live refresh token yields a fresh
|
||||
* bearer, a dead one confirms the session is gone (#95701). Only a structured
|
||||
* 401 qualifies. A 403 is a policy refusal for an identity the gate DID
|
||||
* recognize, and a rotated token for the same identity cannot change it; a
|
||||
* transport failure or an anonymous message-only error says nothing about
|
||||
* the credential and must keep its transient classification.
|
||||
*/
|
||||
export function shouldRotateNativeTokenAfterRejection(error: unknown): boolean {
|
||||
const statusCode = Number(error && typeof error === 'object' ? (error as { statusCode?: unknown }).statusCode : NaN)
|
||||
|
||||
return statusCode === 401
|
||||
}
|
||||
|
||||
234
apps/desktop/electron/remote-reauth-latch.test.ts
Normal file
234
apps/desktop/electron/remote-reauth-latch.test.ts
Normal file
@@ -0,0 +1,234 @@
|
||||
/**
|
||||
* Regression suite for issue #95701: an expired remote OAuth session must
|
||||
* boot into ONE latched recovery overlay, not flicker between the connecting
|
||||
* state and the overlay while the renderer re-drives a rejection that can
|
||||
* never self-heal.
|
||||
*
|
||||
* The chain that broke:
|
||||
*
|
||||
* fetchJson (native bearer) → bare Error("401: ...") — no statusCode
|
||||
* withTransientRetries → not an auth rejection → hammered 3x
|
||||
* gatewayTicketFailure → transport copy, no needsOauthLogin
|
||||
* startHermes → isReauth=false → NOT latched, retryable:true
|
||||
* renderer boot-retry loop → running:true hides the overlay, repeat
|
||||
*
|
||||
* The first half of this file composes the REAL modules exactly the way
|
||||
* main.ts does, so the contract is proven on the code that ships rather than
|
||||
* on a mock of it. The second half pins the main.ts wiring with the repo's
|
||||
* source-assertion pattern (main.ts has no exports; see hardening.test.ts).
|
||||
*/
|
||||
|
||||
import assert from 'node:assert/strict'
|
||||
import fs from 'node:fs'
|
||||
import path from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
|
||||
import { test } from 'vitest'
|
||||
|
||||
import { httpStatusError } from './api-transport'
|
||||
import { isReauthRequiredError } from './backend-health'
|
||||
import {
|
||||
isRetryableRemoteBootFailure,
|
||||
shouldHoldBootProgressForReauth,
|
||||
shouldLatchRemoteReauthFailure
|
||||
} from './backend-start-failure'
|
||||
import { gatewayTicketFailure, isGatewayAuthRejection, withTransientRetries } from './connection-config'
|
||||
import { shouldRotateNativeTokenAfterRejection } from './native-auth-decisions'
|
||||
|
||||
// --- composition: the real modules, in production order ------------------
|
||||
|
||||
test('FIX #95701: a native-bearer 401 is a confirmed, non-retryable reauth rejection end to end', async () => {
|
||||
// What fetchJson now throws for the gate's structured session_expired 401.
|
||||
const bearerRejection = httpStatusError(401, '{"error":"session_expired","reason":"invalid_or_expired_session"}')
|
||||
|
||||
assert.equal(isGatewayAuthRejection(bearerRejection), true)
|
||||
|
||||
// mintGatewayWsTicket's transient-retry wrapper fails immediately: a dead
|
||||
// session is never hammered.
|
||||
let attempts = 0
|
||||
|
||||
const mintError = await withTransientRetries(
|
||||
async () => {
|
||||
attempts += 1
|
||||
throw bearerRejection
|
||||
},
|
||||
{ sleep: async () => {} }
|
||||
).then(
|
||||
() => null,
|
||||
(error: unknown) => error
|
||||
)
|
||||
|
||||
assert.equal(attempts, 1)
|
||||
assert.equal(mintError, bearerRejection)
|
||||
|
||||
// buildRemoteConnection wraps the rejection for the boot path.
|
||||
const wrapped = gatewayTicketFailure(mintError, 'session expired — sign in', 'could not reach gateway') as any
|
||||
|
||||
assert.equal(wrapped.message, 'session expired — sign in')
|
||||
assert.equal(wrapped.needsOauthLogin, true)
|
||||
assert.equal(wrapped.statusCode, 401)
|
||||
|
||||
// startHermes's own composition: isReauth = isReauthRequiredError(error).
|
||||
const isReauth = isReauthRequiredError(wrapped)
|
||||
|
||||
assert.equal(isReauth, true)
|
||||
assert.equal(shouldLatchRemoteReauthFailure({ attemptedRemote: true, isReauth }), true)
|
||||
assert.equal(
|
||||
isRetryableRemoteBootFailure({ attemptedRemote: true, isReauth }),
|
||||
false,
|
||||
'the boot progress must be non-retryable so the renderer never re-drives the boot'
|
||||
)
|
||||
})
|
||||
|
||||
test('FIX #95701: the pre-fix error shape is exactly what made the rejection look transient', async () => {
|
||||
// A "401: ..." message with no structured statusCode — what fetchJson used
|
||||
// to throw. Every classifier below is shape-based on purpose, so this is
|
||||
// the regression the structured error prevents.
|
||||
const anonymous = new Error('401: {"error":"session_expired"}')
|
||||
|
||||
assert.equal(isGatewayAuthRejection(anonymous), false)
|
||||
assert.equal(shouldRotateNativeTokenAfterRejection(anonymous), false)
|
||||
|
||||
let attempts = 0
|
||||
|
||||
await withTransientRetries(
|
||||
async () => {
|
||||
attempts += 1
|
||||
throw anonymous
|
||||
},
|
||||
{ attempts: 3, sleep: async () => {} }
|
||||
).catch(() => undefined)
|
||||
|
||||
assert.equal(attempts, 3, 'an anonymous 401 was retried like a transport blip')
|
||||
|
||||
const wrapped = gatewayTicketFailure(anonymous, 'auth copy', 'transport copy')
|
||||
|
||||
assert.equal(wrapped.message, 'transport copy')
|
||||
assert.equal(isReauthRequiredError(wrapped), false)
|
||||
assert.equal(isRetryableRemoteBootFailure({ attemptedRemote: true, isReauth: isReauthRequiredError(wrapped) }), true)
|
||||
})
|
||||
|
||||
test('FIX #95701: a bearer 401 earns exactly one forced rotation; a 403 or a 5xx does not', () => {
|
||||
assert.equal(shouldRotateNativeTokenAfterRejection(httpStatusError(401, 'expired')), true)
|
||||
assert.equal(shouldRotateNativeTokenAfterRejection(httpStatusError(403, 'forbidden')), false)
|
||||
assert.equal(shouldRotateNativeTokenAfterRejection(httpStatusError(503, 'down')), false)
|
||||
|
||||
// A 403 is still a confirmed rejection for the boot path — it just skips
|
||||
// the pointless rotation on the way there.
|
||||
const forbidden = gatewayTicketFailure(httpStatusError(403, 'forbidden'), 'auth copy', 'transport copy')
|
||||
|
||||
assert.equal(isReauthRequiredError(forbidden), true)
|
||||
assert.equal(isRetryableRemoteBootFailure({ attemptedRemote: true, isReauth: true }), false)
|
||||
})
|
||||
|
||||
test('FIX #95701: once latched, the boot surface ignores everything but the latched failure', () => {
|
||||
const latched = gatewayTicketFailure(httpStatusError(401, 'expired'), 'session expired — sign in', 'transport')
|
||||
|
||||
// The latching emit itself (startHermes: updateBootProgress({ error: message, retryable: false })).
|
||||
assert.equal(shouldHoldBootProgressForReauth(latched.message, { error: latched.message }), false)
|
||||
// A sibling attempt's progress phase (advanceBootProgress → error: null, running: true).
|
||||
assert.equal(shouldHoldBootProgressForReauth(latched.message, { error: null }), true)
|
||||
// A sibling attempt's unrelated transport failure (would flip retryable back on).
|
||||
assert.equal(shouldHoldBootProgressForReauth(latched.message, { error: 'Desktop boot failed: transport' }), true)
|
||||
})
|
||||
|
||||
// --- main.ts wiring ------------------------------------------------------
|
||||
|
||||
const here = path.dirname(fileURLToPath(import.meta.url))
|
||||
const mainSource = fs.readFileSync(path.join(here, 'main.ts'), 'utf8').replace(/\r\n/g, '\n')
|
||||
|
||||
/** Source of one top-level function of main.ts, from its header to the next top-level function. */
|
||||
function mainFunction(header: string): string {
|
||||
const start = mainSource.indexOf(header)
|
||||
|
||||
assert.notEqual(start, -1, `${header} must exist in main.ts`)
|
||||
|
||||
const candidates = ['\nfunction ', '\nasync function ']
|
||||
.map(marker => mainSource.indexOf(marker, start + header.length))
|
||||
.filter(index => index !== -1)
|
||||
|
||||
const end = candidates.length > 0 ? Math.min(...candidates) : mainSource.length
|
||||
|
||||
return mainSource.slice(start, end)
|
||||
}
|
||||
|
||||
test('main.ts: fetchJson and fetchPublicJson throw the structured httpStatusError, never a bare Error', () => {
|
||||
for (const header of ['function fetchJson(', 'function fetchPublicJson(']) {
|
||||
const body = mainFunction(header)
|
||||
|
||||
assert.match(body, /reject\(httpStatusError\(res\.statusCode, text, res\.statusMessage\)\)/, header)
|
||||
assert.doesNotMatch(
|
||||
body,
|
||||
/new Error\(`\$\{res\.statusCode\}:/,
|
||||
`${header} must not rebuild the anonymous status error`
|
||||
)
|
||||
}
|
||||
})
|
||||
|
||||
test('main.ts: the native ticket mint forces one refresh before the rejection is confirmed', () => {
|
||||
const mint = mainFunction('async function mintGatewayWsTicket(')
|
||||
|
||||
assert.match(mint, /if \(!shouldRotateNativeTokenAfterRejection\(error\)\) \{\s*throw error/)
|
||||
assert.match(mint, /ensureNativeAccessToken\(baseUrl, \{ forceRefresh: true \}\)/)
|
||||
assert.match(mint, /return await mintGatewayWsTicketWithBearer\(baseUrl, rotatedAt, headers\)/)
|
||||
|
||||
const ensure = mainFunction('async function ensureNativeAccessToken(')
|
||||
|
||||
assert.match(ensure, /if \(!options\.forceRefresh && !tokenNeedsRefresh\(/)
|
||||
// A dead refresh token drops the stored set (the overlay then reads "not
|
||||
// connected" and offers Sign in) — this branch only works because the
|
||||
// refresh POST now carries a structured statusCode.
|
||||
assert.match(ensure, /error\.statusCode === 401\) \{\s*_clearNativeTokens\(baseUrl\)/)
|
||||
})
|
||||
|
||||
test('main.ts: startHermes latches the failure before its first yield back to the event loop', () => {
|
||||
const setPromise = mainSource.indexOf('backendConnectionState.setPromise(connectionAttempt, connectionPromise)')
|
||||
|
||||
assert.notEqual(setPromise, -1)
|
||||
|
||||
const catchStart = mainSource.lastIndexOf('.catch(async error => {', setPromise)
|
||||
|
||||
assert.notEqual(catchStart, -1)
|
||||
|
||||
const failurePath = mainSource.slice(catchStart, setPromise)
|
||||
const invalidate = failurePath.indexOf('backendConnectionState.invalidate()')
|
||||
const resetGuard = failurePath.indexOf('error instanceof FirstRunSetupResetError')
|
||||
const reauthLatch = failurePath.indexOf('remoteReauthFailure = error instanceof Error ? error : new Error(message)')
|
||||
const localLatch = failurePath.indexOf('backendStartFailure = error instanceof Error ? error : new Error(message)')
|
||||
const exitWait = failurePath.lastIndexOf('await waitForBackendExit(failedProcess)')
|
||||
const emit = failurePath.indexOf('updateBootProgress(')
|
||||
|
||||
for (const [label, index] of Object.entries({ invalidate, resetGuard, reauthLatch, localLatch, exitWait, emit })) {
|
||||
assert.notEqual(index, -1, `${label} must exist in the startHermes failure path`)
|
||||
}
|
||||
|
||||
// No await between invalidate() (which drops the shared attempt promise)
|
||||
// and the latch assignments — a concurrent caller must hit the latch, not
|
||||
// start a fresh attempt that re-emits running:true over the overlay.
|
||||
const beforeLatch = failurePath.slice(invalidate, Math.max(reauthLatch, localLatch))
|
||||
const awaitsBeforeLatch = beforeLatch.match(/\bawait\b/g) ?? []
|
||||
|
||||
assert.equal(
|
||||
awaitsBeforeLatch.length,
|
||||
1,
|
||||
'the only await before the latches is the FirstRunSetupResetError early-exit branch'
|
||||
)
|
||||
assert.ok(resetGuard < reauthLatch, 'a first-run reset is never latched as a failure')
|
||||
assert.ok(reauthLatch < exitWait && localLatch < exitWait, 'latches are set before the exit wait yields')
|
||||
assert.ok(exitWait < emit, 'the failure is emitted after the child has exited, as before')
|
||||
})
|
||||
|
||||
test('main.ts: updateBootProgress holds every non-latched update while the reauth latch is set', () => {
|
||||
const body = mainFunction('function updateBootProgress(')
|
||||
|
||||
const hold = body.indexOf(
|
||||
'shouldHoldBootProgressForReauth(remoteReauthFailure ? remoteReauthFailure.message : null, update)'
|
||||
)
|
||||
|
||||
const assign = body.indexOf('bootProgressState = {')
|
||||
|
||||
assert.notEqual(hold, -1, 'the hold must key on the latched failure message')
|
||||
assert.notEqual(assign, -1)
|
||||
assert.ok(hold < assign, 'the hold runs before the state is touched or broadcast')
|
||||
assert.match(body.slice(hold, assign), /return\s*\}/, 'a held update returns without broadcasting')
|
||||
})
|
||||
Reference in New Issue
Block a user