# Validation and freeze-point evidence

## Reported production fencing and recovery test

Evidence source: operator-supplied production handoff received during the
2026-09-12 documentation review. The operator reports a successful real
three-node Pacemaker/Corosync test with SBD fencing. Codex did not execute or
independently witness this production test. Its exact start/end timestamps and
raw cluster logs were not supplied; no timestamps or measurements are invented.

1. david-eve was primary and was fenced successfully through SBD.
2. david-prodesk promoted automatically; the PostgreSQL VIP moved with it.
3. david-macbookpro remained/reconnected as a streaming standby.
4. david-eve rebooted with former-primary PGDATA. Ordinary resource start
   correctly refused it because standby.signal was absent.
5. The operator manually started its maintenance container with PostgreSQL stopped.
6. The private rejoin action used the PostgreSQL VIP as rewind source.
   pg_rewind detected divergence, found the common checkpoint and completed.
7. david-eve started as a streaming standby. Only after recovery, Pacemaker
   cleanup allowed the cluster to adopt it as Unpromoted.

Reported final state:

| Node | Role | Score |
| --- | --- | ---: |
| david-prodesk | Promoted; PostgreSQL VIP local | 1000 |
| david-macbookpro | Unpromoted; asynchronous streaming | 100 |
| david-eve | Unpromoted; asynchronous streaming | 100 |

No PostgreSQL failed actions remained. The successful run demonstrates the
reported fencing/failover/VIP/rejoin sequence, not every possible partition,
latency, concurrent-write, or recovery failure. The intermediate disconnected
score and lost-transaction count were not supplied. The quoting/authentication
incident and required fix are documented in [operations](operations.md).

## Validated production snapshot — exact deployed hashes pending

The production outcome is operator-reported validated; **the byte-level production
snapshot is not yet established**. Exact hashes from david-prodesk,
david-macbookpro and david-eve have not been supplied or collected. Repository
hashes below must not be represented as deployed hashes. A later capture also
requires operator confirmation that those bytes are the ones used during the test.

| Artifact on each production node | Snapshot status |
| --- | --- |
| /usr/lib/ocf/resource.d/loxthelion/managed-container | Pending production SHA-256 |
| managed-container.d/*.sh (all nine modules) | Pending production SHA-256 |
| managed-container.d/promotion_state.py | Pending production SHA-256 |
| Running container /usr/local/sbin/postgres-ha-bootstrap | Pending production SHA-256 |

Run the following read-only commands on **each production node**, using the
rootless container owner for podman. They hash code, not environment or secrets:

```sh
hostname
date -u '+%Y-%m-%dT%H:%M:%SZ'
sha256sum /usr/lib/ocf/resource.d/loxthelion/managed-container \
  /usr/lib/ocf/resource.d/loxthelion/managed-container.d/*.sh \
  /usr/lib/ocf/resource.d/loxthelion/managed-container.d/promotion_state.py
podman exec postgres sha256sum /usr/local/sbin/postgres-ha-bootstrap
podman inspect --format '{{.Image}}' postgres
```

If permissions require it, run only the host sha256sum command with sudo. Do not
run rootful podman instead of the actual container owner. If the installed
bootstrap path/container name differs, record the actual path/name. Capture
node identity, UTC capture time, all filenames/digests, immutable image ID, and
whether any code changed since the test. Hash the running container's bootstrap;
a host snapshot or image tag alone is not equivalent. Do not dump environment,
secret files, or pgpass. Compare every node individually, preserving mismatches.

## Repository freeze snapshot

The exact repository code hashes appear below and in the reviewed checksum
manifest. These identify the source freeze only, pending production comparison.
The bootstrap is a saved runtime-context artifact, not part of the RA RPM and
not yet confirmed identical to the production bootstrap.

| Repository file | SHA-256 |
| --- | --- |
| `refactored/managed-container` | `a4b775a0ddcabdb0558da9b34c7fe2a883c07d0ef308c1c7224526abf53f63a0` |
| `refactored/managed-container.d/auth.sh` | `93191771d3cb8aefadb2a16d2f89d64a8b6c584071a473439be818237bdf7f17` |
| `refactored/managed-container.d/common.sh` | `97e93d9164e854e5f3688cf9d3cc656b001eb30ef3c2073c907450ca43588f3d` |
| `refactored/managed-container.d/container.sh` | `ac6bc363a2301f1491d162ef609103dbfbceb3b099a3556614c6165952309374` |
| `refactored/managed-container.d/demote.sh` | `0af3bb362a303017372c631354ba94ef9e723bf14e64d2eff91b6662e54a5be0` |
| `refactored/managed-container.d/notify.sh` | `6a98fcdda82cea9365db95b4ee779be5c567b3d846f48b90fbc8c254af697670` |
| `refactored/managed-container.d/postgres.sh` | `d2779ce6e7d1f426ffca2e41bbb8de77503005dc38d658d69f184020365796eb` |
| `refactored/managed-container.d/promote.sh` | `3085ec9b4410905ab9d4370786ed776eac3470a0415aa1b4f608b083945aeb58` |
| `refactored/managed-container.d/rejoin.sh` | `21beea72bcb7f24a51c17ea48666ac87df9436702ad77b166b45f4647e28e8ad` |
| `refactored/managed-container.d/replication.sh` | `6b2021887e3b4cd6a12490e7a8dff98320b2133b472ab25a22a0eead46137547` |
| `refactored/managed-container.d/promotion_state.py` | `5166fc3370f49ab14cd975ded4b03a74b9bd7dbff5762b41a15552c42f934ea7` |
| `runtime-context/postgres-ha-bootstrap` | `3f1a15f7054fb69b9aa92204de30306fd6053c63560310f4231f314b148e2dcf` |


## Non-production checks

The source retains all 29 original lifecycle function bodies and original
parameter defaults. Added scoring changes public dispatch and metadata; it is
not a claim of identical total behavior to the monolithic reference.

Run from the repository root:

```sh
python3 tests/verify_refactor.py
python3 tests/test_promotion_scores.py
python3 tests/test_disconnected_state.py
python3 tests/test_promotion_scheduler.py
python3 tests/test_promotion_cleanup.py
sha256sum -c notes/REFACTOR-SHA256SUMS
git diff --check
```

The baseline suite covers 22 differential lifecycle cases, 25 scoring tests,
16 state/monitor tests, 11 real scheduler simulations and 4 planner tests.
Scheduler failure simulations use synthetic CIBs, enable fencing, and assert
fencing prerequisites for promotion. They do not operate a live cluster.

`tests/verify_promotion_sql.py` explicitly creates/removes a disposable PostgreSQL
18 container and checks nine strict/observation SQL scenarios. Previous candidate
RPM build/install and root-helper CLI verification passed in disposable Fedora
containers. They are separate from the operator's production validation.
Current freeze review results are recorded in [review.md](review.md).