Breadcrumbs

K3s Pre-Deployment & Installation Guide

Purpose

The purpose of this document is to describe what is K3s, its pre-requisites and the system requirements to deploy the Kubernetes distribution K3s built by Rancher Labs. K3s - Lightweight Kubernetes which is a container orchestration tool. There are multiple modes for deployment of K3s Kubernetes Distribution such as Single-Node (Without HA), Multi-Node (Without HA) and High Availability installation.

Each mode has been described separately for the intended audience. However, this is a generic document which outlines common requirements that need to be fulfilled while deploying the solution in any of the modes. 

Intended Audience

This document is intended for IT operations personnel and system administrators who want to deploy Expertflow CX using the K3s distribution platform. Familiarity with computer network component networking and storage is preferable.

About K3s

K3s is a fully compliant Kubernetes distribution with the following enhancements:

  • Packaged as a single binary.

  • Lightweight storage backend based on sqlite3 as the default storage mechanism. etcd3, MySQL, and Postgres are also still available.

  • Wrapped in a simple launcher that handles a lot of the complexity of TLS and options.

  • Secure by default with reasonable defaults for lightweight environments.

  • Simple but powerful “batteries-included” features have been added, such as  local storage provider, a service load balancer, a Helm controller, and the Traefik ingress controller.

  • The operation of all Kubernetes control plane components is encapsulated in a single binary process. This allows K3s to automate and manage complex cluster operations like distributing certificates.

  • External dependencies have been minimized (just a modern kernel and cgroup mounts are needed). K3s packages required dependencies, including:containerdFlannelCoreDNSCNIHost utilities (iptables, socat, etc)Ingress controller (Traefik)Embedded service LoadBalancerEmbedded network policy controller



On this Page


Quick Links





    Preparing for Deployment

    The three K3s modes for deployment should be decided based on what level of availability is required. For this, nodes should be prepared according to the structure of the solution as given below:

    The steps in this guide need to be completed before moving on to any of these modes:

    • K3S Single-Node Installation ( Without HA )

    • K3S Multi-Node Installation ( Without HA )

    • K3s Deployment in High Availability


    CIM Prerequisites 

    Kubernetes distributions and their corresponding requirements are:

    Type

    RAM (GB) 

    CPU

    DISK

    Scalability 

    Network  Ports

    Minimum Nodes

    Single-Node

    Multi & High

    Single-Node

    Multi-Node

    High Availability 

    Single-Node

    Multi-Node

    High Availability 

    K3s1

    16

    8

    250 GiB 

    • 150-250 GiB control-plane

    • 150-250 GiB each worker

    No

    High

    High

    • 6443/TCP to be accessible by all nodes

    • 8472/UDP for CNI

    • 10250/TCP for metrics-server

    • 2379-2380/TCP for Cluster HA 

    1

    2+

    3+2



    RASA-X Prerequisites


    In a single node Deployment, Rasa-X must be deployed separately from the CIM solution.

    Type

    RAM (GB) 

    CPU

    DISK

    Scalability 

    Network  Ports

    Minimum Nodes

    Single-Node

    Multi & High

    Single-Node

    Multi-Node

    High Availability 

    Single-Node

    Multi-Node

    High Availability 

    K3s1

    8

    8

    250 GiB 

    • 150-250 GiB control-plane

    • 150-250 GiB each worker

    No

    High

    High

    • 6443/TCP to be accessible by all nodes

    • 8472/UDP for CNI

    • 10250/TCP for metrics-server

    • 2379-2380/TCP for Cluster HA 

    1

    2+

    3+2



    SuperSet Prerequisites

    For BI Reporting, SuperSet must be deployed separately from the main CIM Solution.


    Type

    RAM (GB) 

    CPU

    DISK

    Scalability 

    Network  Ports

    Minimum Nodes

    Single-Node

    Multi & High

    Single-Node

    Multi-Node

    High Availability 

    Single-Node

    Multi-Node

    High Availability 

    K3s1

    8

    8

    250 GiB 

    • 150-250 GiB control-plane

    • 150-250 GiB each worker

    No

    High

    High

    • 6443/TCP to be accessible by all nodes

    • 8472/UDP for CNI

    • 10250/TCP for metrics-server

    • 2379-2380/TCP for Cluster HA 

    1

    2+

    3+2


    Minimal Requirements for K3s

    1 K3s detailed requirements are available in this guide.

    2 Kubernetes ports and protocols are available in this guide.


    Run Iptables tables​

    If you are running iptables in nftables mode instead of legacy you might encounter issues. We recommend utilizing newer iptables (such as 1.6.1+) to avoid issues.

    Additionally, versions 1.8.0-1.8.4 have known issues that can cause K3s to fail. See Additional OS Preparations for workarounds.

    Environment Preparation

    Before starting with K3s installation, following are the optional steps and the checklist to make sure that environment is prepared for the installation:

    Linux-Based OS Instructions


    Debian/Ubuntu Commands

    Following command must be run for Ubuntu:

    Disable firewall on all nodes

    Bash
    systemctl disable firewalld --now
reboot



    RHEL Commands

    We must run the following commands for RHEL OS before starting the installation of K3S.

    Step1: Disable firewall and nm-cloud-setup service on RHEL

    Bash
    systemctl disable firewalld --now
systemctl disable nm-cloud-setup.service nm-cloud-setup.timer
reboot

    Step 2: Lock the RedHat Release to version 8.4 only

    To lock the release of RHEL to 8.4, which is the latest supported release by longhorn, please execute these commands:

    Bash
    subscription-manager release --set=8.4 ;
yum clean all;
subscription-manager release --show;
rm -rf /var/cache/dnf

    Step 3: Update the RHEL package for 8.4 release


    Bash
    yum update -y




    Checklist

    Before proceeding with the deployment of Single/Multi-Node HA cluster for K3s, go through the checklist:

    Object

    Required
    • Internet Access is available for all the nodes. 1

    Internet access will be needed for all the nodes to fetch and run K3s
    • Minimum Number of Nodes

    3 Control-Plane Nodes + 2 Worker Nodes( For High Availability  ) 
    • All Nodes running verified OS Release

    RHEL-8.4 or Ubuntu-20.04
    • Firewall Service on all nodes is disabled

    Firewall and nm-cloud-setup must be disabled
    • In case of RHEL, Release is fixed to 8.4

    RHEL-8.4 is only supported
    • Virtual IP obtained

    IP from the same range of CP nodes is needed for VIP fail-over ( For HA Only)
    • If Longhorn is to be deployed, iscsid.service is enabled and started

    ( check with systemctl status iscsid.service and confirm its enabled ). On All Nodes.2

    iscsid.service must be running before deploying longhorn Storage Manager
    • If NTP is available ( preferred ) 

    NTP should be enabled for all nodes 
    • POD + services IP range decided

    POD + Services IP Range must not co-exist with already existing IP Range
    • All nodes in cluster have same identical network interface names  Required for High Availability Only  

    Kube-VIP needs consistent interface names across all the control-plane nodes to fail-over. (  ip addr | grep -E ':\s.*?:' | cut -d ":" -f 2 | tr -d " "  ) can be used to list interfaces

    1. Air-Gapped deployment is also possible, check K3s web-site for more details at Air-Gapped install of K3s

    2.  if any of the nodes is not running iscsid.service, the stateful workload will fail and may result in data loss .

    Corporate HTTP/S Proxy Requirement

    If the environment has strict HTTP or HTTPS proxy set, we must exclude the environment from the proxy controls. 

    The NO_PROXY variable must include your  cluster pod and service IP ranges.

    Bash
    HTTP_PROXY=http://your-proxy.example.com:8888
HTTPS_PROXY=http://your-proxy.example.com:8888
NO_PROXY=127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,.svc,.cluster.local

    If you want to configure the proxy settings for containerd without affecting K3s and the Kubelet, you can prefix the variables with CONTAINERD_:

    Bash
    CONTAINERD_HTTP_PROXY=http://your-proxy.example.com:8888
CONTAINERD_HTTPS_PROXY=http://your-proxy.example.com:8888
CONTAINERD_NO_PROXY=127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,.svc,.cluster.local


    Choose an Installation

    Once the pre-requisites are complete, you can choose to select a mode of installation as per your requirement. The steps are explained in each of these guides: