Backup and Restore Strategy

When this applies

This note covers backing up each stateful component in the stack to S3-compatible object storage, using engine-native tooling per component rather than a general volume-snapshot pipeline.

That backup is worth having in a EFCX-High-Availability-Within-a-Site deployment regardless of DR: it protects a single site against data loss, corruption, or a bad change, independent of whether a second site exists to fail over to. The same backups are also the foundation for DR: EFCX-High-Availability-Across-Two-Sites’s active/passive option assumes platform-native, storage/hypervisor-layer asynchronous replication between sites, and when that capability isn’t available, or isn’t confirmed, the DR site is instead rebuilt from these backups when a failover is triggered. The rest of this note covers that fallback path: what to back up, with what, and what it costs in recovery time compared to the storage-replication approach.

Backup approach

Each database in this stack backs up with its own engine-native tool, built to produce a transaction-consistent, restorable copy and write it directly to S3-compatible storage. Stateless configuration, Kubernetes manifests and Helm values, is not part of this backup scope: it already lives in version control and gets redeployed from there (see backup-recommendations). The specific tool and what it delivers per component is in the table below.

Options by component

Component	Suggested approach	What it gives you
PostgreSQL (License Manager, Keycloak data)	An engine-native backup tool (for example pgBackRest) writing full/differential backups plus continuous WAL archiving directly to S3-compatible storage	Point-in-time restore, not just restore-to-last-backup. RPO is bounded by how often WAL segments ship, typically minutes, not by the backup schedule itself
MongoDB (conversation and customer data)	An engine-native backup tool (for example Percona Backup for MongoDB) writing full/incremental backups plus oplog capture directly to S3-compatible storage	Same point-in-time shape as PostgreSQL: restore to a specific moment, not just the last full backup
MinIO / S3-compatible object storage (attachments, recordings)	Bucket-level replication to a bucket at the DR site, a capability most S3-compatible platforms provide natively	Not gated on the block/VM storage-replication question at all, since it operates at the object-storage layer directly. Confirm the specific platform in use supports it and at what interval.
Redis (session and cache state)	Generally not worth backing up for DR	Session and cache state is disposable by design, the application already tolerates starting from empty. Restoring it from a stale backup would bring back data almost certainly invalid by the time a DR event happens. If retention is wanted anyway, a periodic snapshot copied to S3-compatible storage works, but as a point-in-time image, not continuous.
ActiveMQ / Artemis (message bus)	Generally not worth backing up for DR either	Messages in flight at the moment of a site loss are at risk regardless of backup strategy, and durable conversation state already lives in MongoDB, not in the bus. Treat in-flight message loss as an accepted gap of this design rather than something this backup strategy needs to close.
Kubernetes / application configuration	Redeploy from version control (Helm values, manifests), not restore from a backup image	Matches the “rebuild beats restore” principle in backup-recommendations
Vault / secrets	Its own periodic snapshot or export, stored alongside the database backups	Scope still open, confirm before finalizing this row

Recovery sequence at the DR site

Stand up, or confirm already standing, the cluster and namespace at the DR site.
Redeploy application and configuration from version control.
Restore each database to the latest available point: full backup plus replayed WAL or oplog up to the most recent archived segment.
Confirm object storage at the DR site already has current replicated bucket contents.
Start the stateful services, then the application tier, then switch traffic.

This is a rebuild, not a promotion: expect it to take materially longer than a storage-replication handover, since every component is being restored from its own backup rather than one volume simply becoming writable.

RTO/RPO shape of this approach

RPO is set per component by its own backup or archive interval, rather than one platform-wide number. Continuous WAL/oplog archiving can get PostgreSQL and MongoDB down to a few minutes even without synchronous storage replication. Object-storage bucket replication has its own separate interval to confirm.
RTO includes provisioning time if the DR site isn’t already standing, restore time for each database (which scales with data volume), and the startup/traffic-switch sequence above. Expect this to be measured in hours rather than the minutes quoted for storage-replication failover, until actually tested.

Trade-offs versus storage/hypervisor-layer replication

Doesn’t depend on the infrastructure platform offering cross-site block or VM replication: works with any S3-compatible object storage at the DR site.
Each backup mechanism is purpose-built and independently simpler to operate and verify than a general volume-snapshot pipeline, at the cost of running and monitoring several separate backup jobs instead of one platform-level mechanism.
Slower recovery: a rebuild from backups instead of a storage promotion.
The general backup principles in backup-recommendations still apply on top of this: immutability, credential separation, retention, since a backup used for DR carries the same corruption/ransomware exposure as any other backup.

What needs to be confirmed

Backup and archive intervals actually achievable per component, to set a real RPO rather than an assumed one.
Whether the DR site’s object storage target is genuinely offsite: a separate bucket or site, reachable independently of the primary site.
Restore performance at realistic data volumes, since RTO scales with the amount of data being restored, not just the amount being backed up.
A tested, timed restore drill at the DR site, not a design review, before this is relied on.