Node Affinity and Node Selector

This is a user guide on how to apply node affinity and node selector while using Kubernetes. Node Affinity and Node Selector both restrict a pod to a specific node type. We use this incase we want the pod to run only on a SSD based nodes, or if it is a Stateful state with local storage to stay attached to one node. Similarly if a pod requires higher memory consumption or CPU usage you can separate it from other pods.

Add a label to a node

To begin you first need to label your nodes based on your requirements, i.e if it is to separate nodes with faster storage, more memory or high CPU capacity. This label can be anything that is easier for your to differentiate between the nodes.

List the node and their labels

CODE

kubectl get nodes --show-labels

Output will be similar to this

NAME       STATUS   ROLES                       AGE   VERSION        LABELS
vm3     Ready    control-plane,etcd,master   37d   v1.24.7+k3s1   ..node-role.kubernetes.io/control-plane=true,node-role.kubernetes.io...
vm05   Ready    <none>                      37d   v1.24.7+k3s1   ..egress.k3s.io/cluster=true,env=cti,kubernetes.io/arch=amd64..
vm1    Ready    control-plane,etcd,master   37d   v1.24.7+k3s1   ..egress.k3s.io/cluster=true,env=cim,kubernetes.io/arch=amd64..

By default Kubernetes applies labels to it's nodes to define their roles and environments, you can use these to differentiate as well. If you are creating your own labels make sure to have unique label key

2. Select a node you want to add the label to.

To add a unique label to a node

CODE

kubectl label nodes <node name> <unique key>=<value>

Example:
kubectl label nodes fn-vm3 env=cim

Based on the name of the nodes on the above example we have added a new label called "env" and assigned it a value "cim"

3. Verify the label you created has been properly attached to the node

Once again you will run the previous command to get the nodes with their label again.

CODE

kubectl get nodes --show-label

The new output will show all the labels attached to the node

NAME       STATUS   ROLES                       AGE   VERSION        LABELS
fn-vm3     Ready    control-plane,etcd,master   37d   v1.24.7+k3s1   ..node-role.kubernetes.io/control-plane=true,env=cim,...

As from the output above you can verify the label is now attached to the node and we can go ahead with node affinity or node selection based on the new key label above.

Node Affinity

We will be using node affinity to restrict scheduling pods on labeled nodes.

In the following example we will be using a specific term called Required During Scheduling, which will restrict the pod to only schedule/initialize if that node is present.

Select the pod spec yaml file usually found in our Deployment folder and add Node Affinity, as shown below.

CODE

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    ef.service: ef-360-connector
    ef: expertflow
  name: ef-360-connector
  namespace: expertflow

spec:
  replicas: 1
  selector:
    matchLabels:
      ef.service: ef-360-connector
  template:
    metadata:
      labels:
        ef.service: ef-360-connector
        ef: expertflow
    spec:
      imagePullSecrets:
      - name: expertflow-reg-cred
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                - key: env
                  operator: In
                  values:
                  - cim

On the above example you only need to focus on adding the affinity lines in the pod spec section and if any other values are present make sure to keep in proper indentations. Make sure in the key parameter you pass in the unique key you assigned during the node label and assign the appropriate value you gave to that variable while label.

Since I added "env=cim", and I want this pod to only run on the "cim" labeled pods, I will be adding those values in the node affinity bloc. Change the key and value as per your label

CODE

    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                - key: env
                  operator: In
                  values:
                  - cim

After the changes has been made apply the above yaml file using

CODE

kubectl apply -f (your-pod-file).yaml

Node Selector

If you want to assign node selector as well on a node it is simpler operation.

Open the pod deployment .yaml file i.e example below

CODE

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    ef.service: ef-360-connector
    ef: expertflow
  name: ef-360-connector
  namespace: expertflow

spec:
  replicas: 1
  selector:
    matchLabels:
      ef.service: ef-360-connector
  template:
    metadata:
      labels:
        ef.service: ef-360-connector
        ef: expertflow
    spec:
      imagePullSecrets:
      - name: expertflow-reg-cred
      nodeSelector:
        env: cim

As on the example above the labels I assigned to my node were env=cim. Hence based on your labels this value might be changed but will be added in the pod spec section.

CODE

nodeSelector:
  env: cim

One changes are done, apply these.

CODE

kubectl apply -f (your-pod-deployment).yaml