High Availability with DNS

The purpose of this document is to describe steps to deploy an RKE2 Kubernetes distribution in high availability with DNS.

Preparing for Deployment

All control-plane nodes must be ready as per the environment preparation mentioned in RKE2 Control-plane Deployment | Environment-Preparation.

Installation and Configuration Steps

1. Setup DNS Configurations

For DNS based load balancing you need to setup a virtual FQDN that can point to all control plane nodes. Contact your network administrator to do that.

The DNS server should perform health checks on the Control-Plane nodes availability on ports 6443, 9345, 80 and 443. Otherwise routing to control-plane nodes will have to be managed manually.

Step 2. Create first Control Plane node

Follow RKE2 Control-plane Deployment to create the first control-plane node.

Get the server node token from the first control plane. This is required for adding remaining control plane and worker nodes.

BASH

cat /var/lib/rancher/rke2/server/node-token

Step 3. Adding Remaining Control Plane Nodes

Before proceeding, make sure your control plane environment is ready following RKE2 Control-plane Deployment | Environment-Preparation

Create the directories as listed below in the control plane nodes to be added.
BASH
```
mkdir -p /etc/rancher/rke2/
mkdir -p  /var/lib/rancher/rke2/server/manifests/
```

Create a deployment manifest called config.yaml and replace <FQDN> with the FQDN/IP of the first conrtol plane.

BASH

cat<<EOF|tee /etc/rancher/rke2/config.yaml
server: https://<FQDN>:9345
token: [token from /var/lib/rancher/rke2/server/node-token on server node 1]
write-kubeconfig-mode: "0644" 
tls-san:
  - <FQDN>e
write-kubeconfig-mode: "0644"
etcd-expose-metrics: true
cni:
  - canal

EOF

Ingress-Nginx config for RKE2 - By default RKE-2 based ingress controller does not allow additional snippet information in ingress manifests, create this config before starting the deployment of RKE2.

BASH

cat<<EOF| tee /var/lib/rancher/rke2/server/manifests/rke2-ingress-nginx-config.yaml
---
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: rke2-ingress-nginx
  namespace: kube-system
spec:
  valuesContent: |-
    controller:
      metrics:
        service:
          annotations:
            prometheus.io/scrape: "true"
            prometheus.io/port: "10254"
      config:
        use-forwarded-headers: "true"
      allowSnippetAnnotations: "true"
EOF

Step 4. Install RKE2 HA with DNS

Begin the RKE2 Deployment. Starting the Service will take approx. 10-15 minutes based on the network connection
BASH
```
curl -sfL https://get.rke2.io |INSTALL_RKE2_TYPE=server  sh - 
```
Start the RKE2 service
BASH
```
systemctl start rke2-server
```
Enable the RKE2 Service
BASH
```
systemctl enable rke2-server
```
By default, RKE2 deploys all the binaries in /var/lib/rancher/rke2/bin path, add this path to system's default PATH for kubectl utility to work appropriately
BASH
```
export PATH=$PATH:/var/lib/rancher/rke2/bin
export KUBECONFIG=/etc/rancher/rke2/rke2.yaml
```

Append these lines into current user's .bashrc file.

BASH

echo "export PATH=$PATH:/var/lib/rancher/rke2/bin" >> $HOME/.bashrc
echo "export KUBECONFIG=/etc/rancher/rke2/rke2.yaml"  >> $HOME/.bashrc

Step 5. Deploy Worker Nodes

.

CX Worker HA Deployment

On each worker node,

Run the following command to install RKE2 agent on the worker.
BASH
```
curl -sfL https://get.rke2.io | INSTALL_RKE2_TYPE="agent" sh -
```
Enable the rke2-agent service by using the following command.
BASH
```
systemctl enable rke2-agent.service
```
Create a directory by running the following commands.
BASH
```
mkdir -p /etc/rancher/rke2/
```
Add/edit /etc/rancher/rke2/config.yaml and update the following fields.
1. <Control-Plane-IP> This is the IP for the control-plane node.
2. <Control-Plane-TOKEN> This is the token which can be extracted from first control-plane by running cat /var/lib/rancher/rke2/server/node-token
  BASH
```
server: https://<Control-Plane-IP>:9345
token: <Control-Plane-TOKEN>
tls-san:
  - <FQDN>
write-kubeconfig-mode: \"0644\"
etcd-expose-metrics: true
```
Start the service by using follow command.
BASH
```
systemctl start rke2-agent.service
```

Next Steps

Choose storage - See Storage Solution - Getting Started
CX-Core deployment on Kubernetes

.