monitor: publish signed+sealed state reports for monitoring - #9788
Open
ThomasWaldmann wants to merge 7 commits into
Open
monitor: publish signed+sealed state reports for monitoring#9788ThomasWaldmann wants to merge 7 commits into
ThomasWaldmann wants to merge 7 commits into
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #9788 +/- ##
==========================================
- Coverage 85.83% 83.34% -2.49%
==========================================
Files 95 98 +3
Lines 17034 17335 +301
Branches 2607 2652 +45
==========================================
- Hits 14621 14448 -173
- Misses 1673 2163 +490
+ Partials 740 724 -16 ☔ View full report in Codecov by Harness. |
ThomasWaldmann
force-pushed
the
monitoring
branch
from
June 18, 2026 00:26
4006d72 to
41e69cc
Compare
ThomasWaldmann
force-pushed
the
monitoring
branch
from
June 18, 2026 10:48
41e69cc to
7a6876a
Compare
ThomasWaldmann
force-pushed
the
monitoring
branch
2 times, most recently
from
June 18, 2026 21:50
2b1f6a1 to
5d10236
Compare
ThomasWaldmann
force-pushed
the
monitoring
branch
2 times, most recently
from
July 29, 2026 09:55
d8337d2 to
73ef255
Compare
ThomasWaldmann
marked this pull request as ready for review
July 30, 2026 14:07
ThomasWaldmann
force-pushed
the
monitoring
branch
from
July 30, 2026 14:28
73ef255 to
0536679
Compare
After create/prune, the borg client now publishes a small state report into the repository's new `monitoring/` namespace. The report is signed (Ed25519) and sealed (HPKE, RFC 9180) using key material derived from the existing borg key, so the untrusted repository server can neither forge nor read it - it can only relay it. The new `borg monitor` command reads, verifies and decrypts the reports from from the repository using the monitoring key, applies a freshness window (--max-age) and exits non-zero if the report is missing, stale, unsigned or unsuccessful. `borg monitor --key` (which needs the unlocked borg key) derives and prints BORG_MONITORING_KEY for the monitoring host; that value only allows verifying and decrypting the state reports, not creating them. It also does not give access to other data or metadata in the repository. Unencrypted (`--encryption none`) repos have no key to derive the monitoring key from, so their reports are published as unsigned, unencrypted plaintext and these reports are flagged untrusted on read. Requires OpenSSL >= 3.2 (for the built-in HPKE API). borg monitor --json outputs report(s) in json format. borg monitor --keep=N keeps the latest N status reports (default: 500), older reports are deleted. --keep=0 disables this. borg monitor reads all reports and prints, per archive series (and per maintenance command), the latest status and freshness; --name / --command restrict the output. The exit code is the worst across all units. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reports now include the hostname and username of the backup (for create, the archive's own host/user; for repo-wide commands like prune, the local host/user). borg monitor groups reports by (host, user, command, archive series) and reports each distinct backup job independently, so several hosts backing up the same archive series name to one repository no longer mask each other. New --host and --user options restrict the output (alongside --name and --command); the JSON entries expose hostname/username/command/archive. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
borg delete, borg undelete and borg transfer now publish a monitoring report as the last action while the store is open (skipped on --dry-run), with host/user defaulting to the local machine like prune. Stats record the number of archives deleted / undeleted / transferred (plus considered/skipped and transferred bytes). These appear as their own units in borg monitor (command=delete / undelete / transfer). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Object names are plaintext and thus under the untrusted repository server's control, but the reader treated their order as truth: iter_reports() yielded objects in name order and the last one per job won. So the server could copy an old, validly signed "success" report to a new, later-sorting name and shadow a newer failure report for the whole --max-age window. Two defenses: - borg monitor now picks the latest report per job by the signed timestamp inside the report instead of by arrival order. This is the only defense available for unencrypted repos, whose reports are not sealed at all. - The object name is bound into the HPKE aad, so a sealed report only opens under the name it was published as. Re-serving it under another name now fails loudly instead of silently winning. Since every report's timestamp is parsed now (not just the winner's), a report with a missing or unparsable time is skipped with a warning rather than crashing the reader - the contents of an unsigned report are attacker controlled. Skipping can only drop an entry, never mask a newer one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Tampering with a sealed report already fails the run, but the server could avoid that entirely: delete the sealed report it can not forge and put an unsigned "success" of its own in its place. That came back as trusted=False, which is only a warning - so a compromised server could downgrade a failing backup into something an operator alerting on errors alone would never see. A monitoring key only exists for an encrypted repository, and such a repository's client always seals its reports, so an unsigned report there can not be genuine. Reject it like any other verification failure instead of reporting it as merely untrusted. Unencrypted repos (no key configured) keep the old behaviour: their plaintext reports stay readable, untrusted and warning. Also fix the text output hint, which claimed an untrusted report meant the repo is unencrypted - it can equally mean no BORG_MONITORING_KEY was set. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The body of an unsigned report is whatever the untrusted server chooses to serve, but the reader took its shape on faith. A body that is not a JSON object crashed borg monitor with an AttributeError traceback, a list where a string belongs (e.g. "archive") crashed it with an unhashable-type TypeError when grouping, and a report without a status was neither an error nor a warning - so it silently counted as a success. deserialize() now shape-checks every report: the fields the reader relies on (command, time, status, rc) must be present and of the right type, the optional ones (hostname, username, archive, archive_id, stats) must match their type if present, and status must be one of success/warning/error. Anything else is refused with a clean error. Reports from a real client always pass, so this only affects bodies nobody signed. An unparsable *value* in the time field is still tolerated (the report is skipped with a warning, see the previous commit); this is about the shape. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ThomasWaldmann
force-pushed
the
monitoring
branch
from
July 30, 2026 16:13
0536679 to
421d796
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
After create/prune, the borg client now publishes a small state report into the repository's new
monitoring/namespace. The report is signed (Ed25519) and sealed (HPKE, RFC 9180) using key material derived from the existing borg key, so the untrusted repository server can neither forge nor read it - it can only relay it.The new
borg monitorcommand reads, verifies and decrypts the reports from from the repository using the monitoring key, applies a freshness window (--max-age) and exits non-zero if the report is missing, stale, unsigned or unsuccessful.borg monitor --key(which needs the unlocked borg key) derives and prints BORG_MONITORING_KEY for the monitoring host; that value only allows verifying and decrypting the state reports, not creating them. It also does not give access to other data or metadata in the repository.Unencrypted (
--encryption none) repos have no key to derive the monitoring key from, so their reports are published as unsigned, unencrypted plaintext and these reports are flagged untrusted on read.borg monitor --json outputs report(s) in json format.
borg monitor --keep=N keeps the latest N status reports (default: 500), older reports are deleted. --keep=0 disables this.
borg monitor reads all reports and prints, per archive series (and per maintenance command), the latest status and freshness; --name / --command restrict the output. The exit code is the worst across all units.
Requires OpenSSL >= 3.2 (for the built-in HPKE API).
Also: reports record host/user and are grouped per backup job (host, user, command, archive series), so several hosts backing up the same series name to one repository do not mask each other;
--host/--userrestrict the output. delete, undelete and transfer publish reports as well.Hardening (review follow-up)
The last three commits fix issues found while reviewing this PR - all of them ways the untrusted repository server could still influence what the monitor concludes:
--max-agewindow. The latest report per job is now chosen by the signed timestamp inside the report, and the object name is bound into the HPKE aad, so a sealed report only opens under the name it was published as.borg monitorwith a traceback, a list where a string belongs crashed it while grouping, and a report without astatuscounted as a success. Reports are now shape-checked before the reader uses them.Known residual limitation: the server can still delete the newest report, which reverts the view to the previous one until it goes stale (
--max-age). Detecting deletion requires state on the monitoring side and is out of scope here.