david/ocf-managed-container

Fork 0

Files

T

Lyncoln 4a358ae0f6 Add PostgreSQL HA OCF agent modular baseline

2026-09-12 01:28:33 -04:00

7.8 KiB

Raw Blame History

PostgreSQL HA OCF Resource Agent — Codex Refactor Handoff

Objective

Perform a behavior-preserving modular refactor of the proven PostgreSQL HA OCF resource agent contained in:

reference/managed-container

Known-good SHA256:

5eae1f3dba0586dc06105e4dd7a9b8809a5c9961452abe7654dcfce77c3c5ce2

A frozen identical reference copy is:

reference/managed-container.pre-refactor-5eae1f3

DO NOT MODIFY THE FROZEN REFERENCE COPY.

The objective is maintainability and modularity, not redesign.

Architecture

Control path:

Pacemaker
  -> OCF resource agent
  -> runuser david
  -> rootless systemd / Podman
  -> persistent PostgreSQL maintenance container
  -> PostgreSQL

The maintenance container PID 1 is intentionally sleep infinity.

PostgreSQL itself is independently started/stopped with pg_ctl.

The maintenance container must remain alive during demote/rejoin operations so the RA can perform:

SQL checks
pg_rewind
pg_basebackup
replication slot management
recovery configuration

PostgreSQL storage is NOT DRBD-backed.

Quadlet owns container construction and environment injection.

The RA may consume environment variables from the already-running container, but must not gain a dependency on parsing a host-side env file.

Public OCF entrypoint

The public executable must remain:

/usr/lib/ocf/resource.d/loxthelion/managed-container

Preferred modular layout:

managed-container
managed-container.d/
    common.sh
    container.sh
    postgres.sh
    auth.sh
    replication.sh
    promote.sh
    demote.sh
    rejoin.sh
    notify.sh

Helper files should normally be sourced shell modules rather than separate executables.

Namespaced helper functions such as these are preferred:

mc_container_*
mc_pg_*
mc_auth_*
mc_replication_*
mc_promote_*
mc_demote_*
mc_rejoin_*

Preserve these actions

start
stop
monitor
promote
demote
notify
rejoin
meta-data
validate-all

Do not change OCF return codes or action semantics.

start()

start must NOT promote PostgreSQL.

It should:

succeed for an already healthy standby
refuse an already-running primary
start/ensure the maintenance container where appropriate
require standby.signal for stopped PostgreSQL
start PostgreSQL with pg_ctl
succeed only as Unpromoted

Promotion belongs exclusively to promote().

stop()

stop performs a full resource shutdown:

stop PostgreSQL cleanly
stop the rootless postgres.service
verify service/container disappear

Demote and rejoin intentionally behave differently and preserve the maintenance container.

promote()

Preserve current behavior.

Promotion must:

promote a standby
verify primary role
ensure configured local physical HA slots
ensure pg_rewind privileges

POSTGRES_HA_SLOTS is obtained from the running container environment.

Do not automatically delete inactive replication slots.

pg_rewind privileges

Preserve grants for:

pg_catalog.pg_ls_dir(text, boolean, boolean)
pg_catalog.pg_stat_file(text, boolean)
pg_catalog.pg_read_binary_file(text)
pg_catalog.pg_read_binary_file(text, bigint,bigint,boolean)

demote()

The proven controlled demotion logic must remain behaviorally identical.

It:

determines the current role
retrieves the persisted pre-demote target from Pacemaker notify state
resolves that Corosync node to its ring0 address
determines the local physical replication slot
verifies the target is a standby
verifies the target is caught up
ensures this node's slot on the destination
stops the old primary
keeps the maintenance container alive
rewrites recovery settings
creates standby.signal
starts PostgreSQL
verifies recovery mode
verifies WAL receiver is streaming

Do not replace Corosync ring0 addressing with normal hostname resolution.

notify()

Pacemaker notification state is persisted because notification environment variables may no longer exist by demote time.

Preserve this mechanism.

Replication authentication

Runtime replication authentication comes from the running container environment.

POSTGRES_REPLICATION_PASSWORD is exposed inside the running container.

The RA sets PGPASSWORD only inside the container process executing libpq tools.

Preserve this design.

Do NOT:

introduce .pgpass
place passwords in connection strings
parse a host environment file for runtime credentials

rejoin()

Preconditions include:

maintenance service alive
maintenance container alive
PostgreSQL stopped
source reachable
source confirmed primary
physical replication slot ensured on source

Repair order:

pg_rewind
optional pg_basebackup fallback

pg_basebackup fallback

Parameter:

rejoin_allow_basebackup

Default:

false

If rewind fails and fallback is disabled, fail safely.

If enabled:

revalidate source primary
revalidate PostgreSQL stopped
revalidate maintenance container alive
reensure replication slot
calculate approximate source size
check conservative free-space requirement
rename old canonical PGDATA to:
```
$PGDATA.pre-basebackup-TIMESTAMP
```
create new canonical PGDATA
perform pg_basebackup

If basebackup fails:

quarantine partial copy as:
```
$PGDATA.failed-basebackup-TIMESTAMP
```
restore the old PGDATA to its canonical pathname
leave PostgreSQL stopped
return failure

Never blindly:

rm -rf "$PGDATA"

Recovery normalization

After successful rewind or basebackup:

normalize primary_conninfo
normalize primary_slot_name
create standby.signal
apply expected permissions
start PostgreSQL
verify pg_is_in_recovery() = true
verify WAL receiver state = streaming

Prefer extracting common recovery-configuration logic rather than duplicating it.

Behavior already proven

The reference build has successfully passed:

standby start
promotion
controlled demotion
bidirectional switchover
slot reconciliation after promotion
rewind privilege reconciliation
successful pg_rewind rejoin
pg_rewind failure with fallback=false
pg_rewind failure followed by successful pg_basebackup
pg_rewind failure followed by failed pg_basebackup and successful rollback
refusal to run rejoin while PostgreSQL is running

Do not change these semantics.

Current lab

DAVID-DV6:

primary

DAVID-ELITEDESK:

standby
streaming from 192.168.1.2
slot ha_david_elitedesk

Pacemaker is intentionally in:

maintenance-mode=true

Do NOT disable maintenance mode.

Do NOT touch production systems.

Refactor procedure

Before editing:

read the complete reference implementation
inventory global variables
inventory OCF parameters
inventory action entrypoints
map helper/function dependencies
produce a proposed module dependency map

Then refactor incrementally.

Avoid changing shell semantics unless required to modularize the code.

Preserve quoting, SQL, command invocation, OCF error handling and logging.

Validation

At minimum:

bash -n main entrypoint
bash -n every module
meta-data action works
validate-all works
all modules resolve correctly in OCF execution environment
no missing functions
no missing variables
same metadata parameters
same OCF actions
no .pgpass usage
no host env-file parsing dependency
frozen reference remains unchanged

Do NOT automatically perform destructive HA tests.

When finished, report:

final file tree
SHA256 of every file
functions assigned to each module
syntax validation results
metadata validation results
validate-all results
any behavior differences, ideally none
exact commands to install the refactored version on the second lab node
recommended behavioral comparison test sequence

The frozen source implementation is authoritative whenever this document and the implementation differ.

7.8 KiB Raw Blame History