PostgreSQL HA resource-agent modular refactor

The refactored agent is in refactored/managed-container, with nine sourced modules beside it. Both reference copies remain unchanged at SHA256 5eae1f3dba0586dc06105e4dd7a9b8809a5c9961452abe7654dcfce77c3c5ce2.

Final file tree

refactored/
  managed-container
  managed-container.d/
    common.sh
    container.sh
    postgres.sh
    auth.sh
    replication.sh
    notify.sh
    promote.sh
    demote.sh
    rejoin.sh
notes/
  REFACTOR-PLAN.md
  REFACTOR-REPORT.md
  REFACTOR-SHA256SUMS
tests/
  verify_refactor.py

The SHA256 manifest lists every delivered code, test, and documentation file (excluding the manifest itself). Existing context files are unchanged.

Function assignments

Module	Functions
common.sh	meta_data, validate_all; original parameter defaults and runtime globals
container.sh	service_uid, service_home, as_service_user, service_running, container_running
postgres.sh	pg_command, pgdata_path, pg_running, pg_ready, pg_role, pg_wal_receiver_status, managed_monitor, managed_start, managed_stop
auth.sh	pg_replication_exec, ensure_rewind_privileges
replication.sh	pg_remote_query, ensure_local_ha_slots, ensure_remote_slot, candidate_caught_up, candidate_is_standby
notify.sh	demote_target, managed_notify
promote.sh	managed_promote
demote.sh	node_to_addr, managed_demote
rejoin.sh	managed_rejoin

The entrypoint retains OCF initialization and the original dispatcher. See REFACTOR-PLAN.md for parameters, globals, function dependencies, and source order.

Validation results

Run from the workspace root:

python3 tests/verify_refactor.py
sha256sum -c notes/REFACTOR-SHA256SUMS

bash -n: entrypoint and all nine modules pass.
All 29 original function definitions are byte-identical. Original defaults/global initialization and dispatcher are unchanged. This also preserves function/variable references inside original code.
Real OCF-library metadata execution succeeds from /tmp; output is byte-identical to the reference and parses as XML with the same 12 parameters and eight advertised actions.
All modules load in the installed OCF-library environment. The loader uses command dirname because this OCF library defines a dirname shell function that does not support --.
Missing module directory produces OCF_ERR_CONFIGURED (6).
22 isolated differential scenarios pass, comparing return codes, stdout, and stderr: eight monitor states; three start cases; two demote cases; rejoin while PostgreSQL is running; existing-primary promotion reconciliation; already-stopped stop; and six validate-all paths.
Direct validate-all on both original and refactored agents returns 6 with User service 'postgres.service' cannot be found for david. The execution environment cannot validate the target service. Stubbed validation success and missing user/command/runtime/service/Podman failure paths all match.
No .pgpass, PGPASSFILE, or host environment-file sourcing was introduced into the RA.
Both frozen and working reference hashes still match the handoff.

No live cluster actions or destructive HA tests were executed. Offline comparisons establish extraction equivalence; they do not replace controlled live validation of a multi-file installation.

Behavior differences and limits

No intended action behavior differences. Function bodies, SQL, quoting, log messages, OCF return paths, authentication, and recovery flow are preserved exactly. The new packaging requires all nine modules beside the executable. Missing/unloadable modules now fail explicitly with configured error. The only added persistent shell global is MC_MODULE_DIR; the loader unsets its loop variable MC_MODULE.

Original function names and duplicated demote/rejoin recovery configuration remain in this first modular extraction. Namespacing and deduplication were preferences in the handoff; retaining them makes exact body equivalence checkable. The existing unused service_home helper remains too. No global strict-mode options were added. The executable should be installed as a regular file at the public OCF path, with modules in its sibling managed-container.d directory; a symlink elsewhere does not automatically locate modules at its target.

The bootstrap script's existing pgpass handling and the Quadlet host environment-file injection are outside this RA refactor and were not changed. The metadata continues to omit the dispatched rejoin action and status alias, matching the reference.

Installation commands for the second lab node

Target the handoff's standby, DAVID-ELITEDESK (192.168.1.6). These commands are provided for operator execution and have not been run. Run the first block from the workspace root on the machine holding these files. The staging path must be unused; it is created with mkdir so an existing staging directory stops the sequence.

set -e
ssh david@192.168.1.6 'mkdir /tmp/managed-container-refactor'
scp -r refactored david@192.168.1.6:/tmp/managed-container-refactor/

Then log into DAVID-ELITEDESK and run the following block. Confirm the saved lab roles still hold before applying. Leave maintenance mode enabled throughout.

ssh david@192.168.1.6

On DAVID-ELITEDESK:

set -e
sudo pcs property config maintenance-mode
# Confirm maintenance-mode=true in the output before continuing.
sudo bash -n /tmp/managed-container-refactor/refactored/managed-container
for module in /tmp/managed-container-refactor/refactored/managed-container.d/*.sh; do
    sudo bash -n "$module"
done

mc_backup_dir="/var/backups/managed-container-$(date -u +%Y%m%d-%H%M%S)"
sudo mkdir -m 0700 "$mc_backup_dir"
sudo cp -a /usr/lib/ocf/resource.d/loxthelion/managed-container "$mc_backup_dir/"
if sudo test -d /usr/lib/ocf/resource.d/loxthelion/managed-container.d; then
    sudo cp -a /usr/lib/ocf/resource.d/loxthelion/managed-container.d "$mc_backup_dir/"
fi
sudo install -d -o root -g root -m 0755 /usr/lib/ocf/resource.d/loxthelion/managed-container.d
sudo install -o root -g root -m 0644 \
    /tmp/managed-container-refactor/refactored/managed-container.d/*.sh \
    /usr/lib/ocf/resource.d/loxthelion/managed-container.d/
sudo install -o root -g root -m 0755 \
    /tmp/managed-container-refactor/refactored/managed-container \
    /usr/lib/ocf/resource.d/loxthelion/managed-container
sudo restorecon -RF /usr/lib/ocf/resource.d/loxthelion/managed-container \
    /usr/lib/ocf/resource.d/loxthelion/managed-container.d
sudo /usr/lib/ocf/resource.d/loxthelion/managed-container meta-data
sudo /usr/lib/ocf/resource.d/loxthelion/managed-container validate-all
printf 'Original installation saved at %s\n' "$mc_backup_dir"

Module files must be root-owned and not writable by the service user because the RA sources them as root. Install modules before the entrypoint, with resource management still disabled. No service restart is required just to replace the agent files. No commands here disable maintenance mode.

Recommended behavioral comparison sequence

Perform in the isolated lab with maintenance mode retained and a recoverable data baseline. Compare original and refactored agents from equivalent initial states, recording OCF return codes, logs, service/container state, PostgreSQL role, receiver status, slot state and data-directory names. Do not invoke both implementations sequentially against already-mutated state and call that an equivalence test.

Validate metadata/parameters and validate-all on the target node. Run monitor on healthy primary and standby (8 and 0); verify stopped cases return 7.
Verify standby start succeeds, primary start is refused, and missing standby.signal prevents startup of stopped PostgreSQL.
Verify full stop shuts down PostgreSQL and the maintenance service/container, then restore the standby setup.
Promote a healthy standby and repeat promotion on the primary; check configured physical slots and all four rewind grants without deleting inactive slots.
Reproduce controlled switchover using the proven lab orchestration: persist a fresh matching pre-demote notification, resolve the successor through Corosync ring0, verify standby/catch-up checks, and coordinate successor promotion so streaming can complete. Repeat in the opposite direction. Keep the maintenance container alive throughout demote.
Rejoin a stopped divergent node using successful pg_rewind; verify normalized recovery settings, permissions, recovery=true and streaming. Verify rejoin refuses running PostgreSQL.
With a recoverable lab baseline, force rewind failure with fallback disabled; verify failure and no basebackup replacement.
Enable fallback for the explicit lab test; force rewind failure and successful basebackup. Verify old PGDATA preservation and streaming standby recovery.
Force basebackup failure; verify partial-copy quarantine, restoration of canonical old PGDATA, PostgreSQL stopped, and failure returned. Also exercise source-role/slot/container/free-space preflight failures.
Restore the intended DAVID-DV6 primary / DAVID-ELITEDESK standby topology and verify maintenance mode remains true. Decide separately when to deploy beyond this lab.

9.1 KiB Raw Blame History