Node Affinity and Node Selector

This is a user guide on how to apply node affinity and node selector while using Kubernetes. Node Affinity and Node Selector restrict a pod to a specific node type. We use this in case we want the pod to run only on SSD-based nodes, or if it is a Stateful state with local storage to stay attached to one node. Similarly, if a pod requires higher memory consumption or CPU usage you can separate it from other pods.

Add a label to a node

To begin you first need to label your nodes based on your requirements, i.e if it is to separate nodes with faster storage, more memory, or high CPU capacity. This label can be anything easier for you to differentiate between the nodes.

List the node and their labels

CODE

kubectl get nodes --show-labels

Output will be similar to this

NAME       STATUS   ROLES                       AGE   VERSION        LABELS
vm3     Ready    control-plane,etcd,master   37d   v1.24.7+k3s1   ..node-role.kubernetes.io/control-plane=true,node-role.kubernetes.io...
vm05   Ready    <none>                      37d   v1.24.7+k3s1   ..egress.k3s.io/cluster=true,env=cti,kubernetes.io/arch=amd64..
vm1    Ready    control-plane,etcd,master   37d   v1.24.7+k3s1   ..egress.k3s.io/cluster=true,env=cim,kubernetes.io/arch=amd64..

By default Kubernetes applies labels to its nodes to define their roles and environments, you can use these to differentiate as well. If you are creating your own labels make sure to have unique label key

2. Select a node you want to add the label to.

To add a unique label to a node.

Note: Labels can be assigned to any node including the worker and control plane. kindly refer to the guide related to specific deployment types to check which node needs to be labeled.

CODE

kubectl label nodes <node name> <unique key>=<value>

Example:
kubectl label nodes fn-vm3 env=cim

Based on the name of the nodes in the above example we have added a new label called "env" and assigned it a value "cim"

3. Verify the label you created has been properly attached to the node. Once again you will run the previous command to get the nodes with their label again.

CODE

kubectl get nodes --show-labels

The new output will show all the labels attached to the node

NAME       STATUS   ROLES                       AGE   VERSION        LABELS
fn-vm3     Ready    control-plane,etcd,master   37d   v1.24.7+k3s1   ..node-role.kubernetes.io/control-plane=true,env=cim,...

As from the output above you can verify the label is now attached to the node and we can go ahead with node affinity or node selection based on the new key label above.

Node Affinity

We will be using node affinity to restrict scheduling pods on labeled nodes.

In the following example, we will use a specific term called Required During Scheduling, which will restrict the pod to only schedule/initialize if that node is present.

run the following commands to change the directory to the configuration directory.

CODE

cd cim-solution/kubernetes/cim/Deployments

Select each pod spec yaml file one by one and add Node Affinity, as shown below.

CODE

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    ef.service: ef-360-connector
    ef: expertflow
  name: ef-360-connector
  namespace: expertflow

spec:
  replicas: 1
  selector:
    matchLabels:
      ef.service: ef-360-connector
  template:
    metadata:
      labels:
        ef.service: ef-360-connector
        ef: expertflow
    spec:
      imagePullSecrets:
      - name: expertflow-reg-cred
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                - key: env
                  operator: In
                  values:
                  - cim

On the above example you only need to focus on adding the affinity lines in the pod spec section and if any other values are present make sure to keep in proper indentations. Make sure in the key parameter you pass in the unique key you assigned during the node label and assign the appropriate value you gave to that variable while label.

Since I added "env=cim", and I want this pod to only run on the "cim" labeled pods, I will be adding those values in the node affinity bloc. Change the key and value as per your label

CODE

    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                - key: env
                  operator: In
                  values:
                  - cim

After the changes has been made apply the above yaml file using

CODE

kubectl apply -f (your-pod-file).yaml

Node Selector

If you want to assign node selector as well on a node it is simpler operation.

Open the pod deployment .yaml file i.e example below

CODE

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    ef.service: ef-360-connector
    ef: expertflow
  name: ef-360-connector
  namespace: expertflow

spec:
  replicas: 1
  selector:
    matchLabels:
      ef.service: ef-360-connector
  template:
    metadata:
      labels:
        ef.service: ef-360-connector
        ef: expertflow
    spec:
      imagePullSecrets:
      - name: expertflow-reg-cred
      nodeSelector:
        env: cim

As on the example above the labels I assigned to my node were env=cim. Hence based on your labels this value might be changed but will be added in the pod spec section.

CODE

nodeSelector:
  env: cim

Once changes are done, apply these.

CODE

kubectl apply -f (your-pod-deployment).yaml