High Availability with Redundant Deployment

A redundant deployment for high availability via Docker compose is supported. The solution is designed to provide complete failover between 2 systems (with an Arbiter) in case site-A goes down, site-B takes over. 

Manual intervention is however required. 

In an HA environment, following nodes are setup for high availability.

1. Master Node ( site-a )
2. Slave Node   ( site-b )
3. Arbiter Node  ( neutral location )
4. Load Balancer

Deployment Architecture

For HA, the solution works in a primary standby mode. When the primary site is down, the secondary takes over. 

1. An External Load Balancer (ELB) receives all requests and switches between primary and secondary site. 
2. Each site has an Application Gateway (NGINX) that routes the traffic to internal solution components.

HA Deployment Architecture

Solution Components

Components	Details
Application Gateway	This is the border element on each site that receives internal/external requests and performs service discovery of internal Expertflow software components.
Hybrid Chat Components	All the Hybrid Chat components are installed in a redundant fashion on Site-A and Site-B. warning There might be a downtime of a few seconds while restarting the container. This implies that any requests received on the solution component during its downtime (or while the container is restarting) will either be rejected or queued.
Load Balancer	A Load Balancer based on HA Proxy is shipped with the solution. You may either opt for this Load Balancer or may use your own layer-4 Load Balancer to route all traffic to either Site-A or Site-B. DNS Load Balancing may also be used keeping in view the down-side of it that you'll have to manually redirect to backup Site in case failover is needed. HA Proxy shipped with the solution is a single point of failure.
Arbiter node	This node is used only for electing between primary and secondary nodes. This node may be setup separately or co-located with the Load Balancer.
Other 3rd party components	Active MQ (stateful) - Deployed in a failover cluster configuration. Each AMQ client should point to the failover cluster IP.  Mongo DB (stateful) - Deployed in a Three member replica set with one being Arbiter. SQL Reporting DB (stateful) - SQL Server is used for external reporting purpose via Cisco CUIC or similar. The customer needs to provide SQL Server cluster for SQL Server failover support. Other SQL redundancy options such as Log shipping may also be used. MySQL (stateful) - Some components may require MySQL or SQL Server for configurations and transactional storage. MySQL master-slave replication should mostly serve the purpose.

After syncing the users from UMM ( on both Primary and Secondary Systems )

MongDB Priority Setup

Please follow steps mentioned at  MongoDB ReplicaSet Configuration, otherwise required state of databases will not be prioritized and system will not function properly.