fix(sessions): transcript exports keep compacted turns, so --delete-after-verified no longer deletes them unseen
In-place compaction is the default. It soft-archives every earlier row of a session under the same id (active = 0, compacted = 1), and Desktop and the dashboard still show those turns. The md/qmd export read the session through export_session -> get_messages with the default live-only clause, so it wrote only the compaction summary and the carried tail. verify_export_file then compared the file with that same dict, and delete_session removed every row of the session, including the archived turns that never reached the file. The md/qmd export now reads the display history (include_compacted), for a single session and for --lineage logical. export_session and export_session_lineage take include_compacted, off by default: import_sessions inserts every message as live context, so the JSON export and stranded-session adoption keep reading live rows only. The other transcripts people read had the same hole without the delete: `/save md` and `/save html` (CLI and gateway), `sessions export --format html` (one session or all of them) and `--only user-prompts` each held 1 of 6 answers on a six-turn session after one compaction. They now read the display history too (SAVE_TRANSCRIPT_FORMATS; export_all gains include_compacted and reads per session then, since the display read dedupes per session). `/save json`, JSONL and the dashboard's JSON export stay live-only for the import reason above. Before deleting, the verify step also re-counts the store's display rows for every session the file covers and refuses on a mismatch. A message that lands while the files are written, or a later export change that reads a narrower view, now refuses the delete instead of being removed unseen. Like the adoption retire loop, the re-count runs just before delete_session, not inside its transaction. Rewind rows (undone turns, the superseded originals of a carried tail) are still deleted without being exported, as `hermes sessions delete` does: they are not part of the history the session shows. Measured through the real CLI on a session with 6 turns and one default in-place compaction (15 rows, 13 shown): before, 3 messages were exported and all 15 rows deleted, with answers 1-5 missing from the file; after, 13 messages are exported in display order, then deleted. (cherry picked from commit adeaff1e33f1ae2b8a066ef2374bb29ade50b3b8)
This commit is contained in:
committed by
kshitij
parent
55138d85b2
commit
fe8b643db6
@@ -332,11 +332,15 @@ def _cmd_export(db, args):
|
||||
from hermes_cli.session_export_md import redact_session_data
|
||||
return redact_session_data(data)
|
||||
|
||||
# HTML and --only are read by people, so they carry the turns in-place compaction archived; JSONL stays the
|
||||
# live rows import_sessions restores.
|
||||
shown = args.format == "html" or bool(getattr(args, "only", None))
|
||||
|
||||
def _collect_sessions():
|
||||
"""--session-id / filters / bare export -> redacted session dicts, or None after printing an error."""
|
||||
if args.session_id:
|
||||
resolved = db.resolve_session_id(args.session_id)
|
||||
data = _redact(db.export_session(resolved)) if resolved else None
|
||||
data = _redact(db.export_session(resolved, include_compacted=shown)) if resolved else None
|
||||
if not data:
|
||||
_not_found(args.session_id)
|
||||
return None
|
||||
@@ -345,10 +349,10 @@ def _cmd_export(db, args):
|
||||
candidates = db.list_prune_candidates(**filters)
|
||||
if args.dry_run:
|
||||
return _print_dry_run_preview(candidates, filters)
|
||||
return [s for s in (_redact(db.export_session(row["id"])) for row in candidates) if s]
|
||||
return [s for s in (_redact(db.export_session(row["id"], include_compacted=shown)) for row in candidates) if s]
|
||||
if args.dry_run:
|
||||
return print("--dry-run requires at least one filter.")
|
||||
return [_redact(s) for s in db.export_all(source=None)]
|
||||
return [_redact(s) for s in db.export_all(source=None, include_compacted=shown)]
|
||||
if getattr(args, "only", None):
|
||||
return _export_flat("only", args, _collect_sessions)
|
||||
if args.format == "trace":
|
||||
@@ -472,7 +476,10 @@ def _export_markdown(db, args, filters, redact):
|
||||
output_dir = _export_dir(args.output)
|
||||
|
||||
def _export_one(session_id: str, *, include_lineage: bool = False):
|
||||
data = db.export_session_lineage(session_id) if include_lineage else db.export_session(session_id)
|
||||
# The history the user sees, not only the live rows: in-place compaction archives earlier turns under
|
||||
# the same id, and --delete-after-verified removes every row of it.
|
||||
export = db.export_session_lineage if include_lineage else db.export_session
|
||||
data = export(session_id, include_compacted=True)
|
||||
if not data:
|
||||
return None, None
|
||||
data = redact(data)
|
||||
@@ -538,6 +545,14 @@ def _export_markdown_single(db, args, export_one, output_dir, lineage_is_logical
|
||||
return
|
||||
for data, exported_path in exported_items:
|
||||
ok, reason = verify_export_file(exported_path, data)
|
||||
# The file only proves it matches the dict it was written from; the delete removes what the store holds
|
||||
# now, so re-count the store just before it (like the adoption retire loop, outside its transaction).
|
||||
exported = len(data.get("messages") or [])
|
||||
shown = sum(len(db.get_messages(sid, include_compacted=True))
|
||||
for sid in data.get("lineage_session_ids") or [data["id"]])
|
||||
if ok and shown != exported:
|
||||
ok, reason = False, (f"the session changed after it was exported ({shown} messages now, {exported} in "
|
||||
"the file); run the export again")
|
||||
if not ok:
|
||||
print(f"Export verification failed; not deleting session '{data.get('id')}': {reason}")
|
||||
return
|
||||
|
||||
Reference in New Issue
Block a user