RKE2 Multi-Node Installation ( Without HA )

Purpose

The purpose of this document is to describe the additional system requirements and steps to deploy the Multi-Node RKE2 Kubernetes Distribution.

Pre-requisites for Multi-Node Installation

The prerequisites and pre-deployment phases are describe in the RKE2 Pre-Deployment & Installation Guide (Single & Multi Node). Please complete the steps before proceeding with Multi-Node Installation.

Quick Links

Installation Steps

Customize the RKE2 Deployment for your Environment

Click here to see customization steps.....

Below given options can also be used for customized environment setup:

Option	Switch	Default	Description
Default Deployment Directory of RKE2 (Not recommended)	`--data-dir value, -d` value	`/var/lib/rancher/rke2` or `${HOME}/.rancher/rke2` if not root	Important Note: Moving the default destination folder to another location is not recommended. However, if there is need for storing the containers in different partition, it is recommended to deploy the containerd separately and change its destination to the partition where you have space available using `--root` flag in containerd.server manifest, and subsequently adding `#container-runtime-endpoint: "/path/to/containerd.sock"` switch in RKE2 config.yaml file.
Default POD IP Assignment Range	--cluster-cidr value	"10.42.0.0/16"	IPv4/IPv6 network CIDRs to use for pod IPs
Default Service IP Assignment Range	`--service-cidr` value	"10.43.0.0/16"	IPv4/IPv6 network CIDRs to use for service IPs

If any of the above option is required, add it in the next step.

cluster-cidr and service-cidr are independently evaluated. Decide wisely well before the the cluster deployment. This option is not configurable once the cluster is deployed and workload is running.

Step 1: Enable Customization for Ingress-Nginx

This step is required for the Nginx Ingress Controller to allow customized configurations:

1. Create the destination folder

BASH

mkdir -p  /var/lib/rancher/rke2/server/manifests/

2. Generate the ingress-nginx controller config file so that the RKE2 server bootstraps it accordingly.

BASH

cat<<EOF| tee /var/lib/rancher/rke2/server/manifests/rke2-ingress-nginx-config.yaml
---
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: rke2-ingress-nginx
  namespace: kube-system
spec:
  valuesContent: |-
    controller:
      metrics:
        service:
          annotations:
            prometheus.io/scrape: "true"
            prometheus.io/port: "10254"
      config:
        use-forwarded-headers: "true"
      allowSnippetAnnotations: "true"
EOF

Step 2: Download the RKE2 binaries and start the Installation

Step 3: For Master Node

1. Run the below command on the master node.

BASH

curl -sfL https://get.rke2.io |INSTALL_RKE2_TYPE=server  sh -

RKE2 will be installed on the master node.

2. Enable the rke2-server service

BASH

systemctl enable rke2-server.service

3. Start the service

BASH

systemctl start rke2-server.service

RKE2 server requires 10-15 minutes (at least) to bootstrap completely You can check the status of the RKE2 Server using systemctl status rke2-server. Once it reports as running, please proceed with the rest of the steps as given below.

4. By default RKE2 deploys all the binaries in /var/lib/rancher/rke2/bin path. Add this path to system's default PATH for kubectl utility to work appropriately

BASH

export PATH=$PATH:/var/lib/rancher/rke2/bin
export KUBECONFIG=/etc/rancher/rke2/rke2.yaml

5. Also append these lines into current user's .bashrc file

BASH

echo "export PATH=$PATH:/var/lib/rancher/rke2/bin" >> $HOME/.bashrc
echo "export KUBECONFIG=/etc/rancher/rke2/rke2.yaml"  >> $HOME/.bashrc

6. and source your ~/.bashrc

BASH

source ~/.bashrc

Step 4: For Worker Node

1. If you want to add a worker node, run the following command on the master node and copy the content of the token. We will need this when we deploy RKE2 on a worker node.

BASH

cat /var/lib/rancher/rke2/server/node-token

2. It will display the node-token as something like this:

BASH

K10e2bfc647bbf0839a7997cdcbee8754b3cd841e85e4250686161893f2b139c7d8::server:a342ef5189711287fb48f05c05346b89

3. Run the installer command on the worker node.

BASH

curl -sfL https://get.rke2.io | INSTALL_RKE2_TYPE="agent" sh -

4. Enable the rke2-agent service by using the following command on the worker node.

BASH

systemctl enable rke2-agent.service

5. Once the installation is finished, create a directory by running the following commands. (The following command is to be run on the worker node(s)).

BASH

mkdir -p /etc/rancher/rke2/

6. To edit the config.yaml uses the following command. (The following command is to be run on the worker node(s).

BASH

vi /etc/rancher/rke2/config.yaml

7. Copy the content of the config file. Please make sure to update the following fields:

<MASTER-IP>
<MASTER-NODE-TOKEN>

BASH

server: https://<IP-OF-MASTER-NODE>:9345
token: K10e2bfc647bbf0839a7997cdcbee8754b3cd841e85e4250686161893f2b139c7d8::server:a342ef5189711287fb48f05c05346b89

8. Start the service by using follow command.

BASH

systemctl start rke2-agent.service

Step 5: Bash Completion for kubectl

1. Install bash-completion package:

BASH

yum install bash-completion -y

2. Set up autocomplete in bash into the current shell, bash-completion . The package should be installed first.

BASH

source <(kubectl completion bash) 
echo "source <(kubectl completion bash)" >> ~/.bashrc

3. Also, add the alias for the short notation of kubectl:

BASH

echo "alias k=kubectl"  >> ~/.bashrc 
echo "complete -o default -F __start_kubectl k"  >> ~/.bashrc

4. Source your ~/.bashrc

BASH

source ~/.bashrc

Step 6: Install helm

1. Add this command in ~/.bashrc file.

BASH

echo "export KUBECONFIG=/etc/rancher/rke2/rke2.yaml" >> ~/.bashrc

2. Run this in the command prompt.

BASH

source ~/.bashrc

3. Helm is a super tool to deploy external components. To install helm on cluster, execute the following command:

BASH

curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3|bash

BASH

cd /usr/local/bin
mkdir -p $HOME/bin && cp ./helm $HOME/bin/helm && export PATH=$HOME/bin:$PATH

Step 7: Enable bash completion for the helm

1. Generate the scripts for helm bash-completion:

BASH

helm completion bash > /etc/bash_completion.d/helm

2. Either re-login or run this command to enable the helm bash completion instantly.

BASH

source <(helm completion bash)

Step 8: Clone the CIM Repo

Use the following command for CIM Repo and replace the branch name with the actual release.

BASH

git clone -b <branch-name>  https://efcx: RecRpsuH34yqp56YRFUb@gitlab.expertflow.com/cim/cim-solution.git

Step 9: Storage for RKE2 Multi-Node Installation

The recommended storage option for RKE2 Multi-Node Installation is to use OpenEBS. The details of deployment of OpenEBS can be found in this document.

Step 10: Expertflow CX Deployment on Kubernetes

Please follow the steps in the document, Expertflow CX Deployment on Kubernetes to deploy Expertflow CX Solution.