ETCD Server Administration in K3s

This document gives a brief intro into administration of ETCD server in K3s env.

Run this command to identify the ETCD Version running locally to interact with the K3s ETCD server, as the default k3s deployment doesn't contain any utility to interact with the ETCD server.

CODE

curl -L --cacert /var/lib/rancher/k3s/server/tls/etcd/server-ca.crt --cert /var/lib/rancher/k3s/server/tls/etcd/server-client.crt --key /var/lib/rancher/k3s/server/tls/etcd/server-client.key https://127.0.0.1:2379/version

Note the version and change the ETCD_VER in below

CODE

ETCD_VER=v3.5.2

# choose either URL
GOOGLE_URL=https://storage.googleapis.com/etcd
GITHUB_URL=https://github.com/etcd-io/etcd/releases/download
DOWNLOAD_URL=${GOOGLE_URL}

rm -f /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz
rm -rf /tmp/etcd-download-test && mkdir -p /tmp/etcd-download-test

curl -L ${DOWNLOAD_URL}/${ETCD_VER}/etcd-${ETCD_VER}-linux-amd64.tar.gz -o /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz
tar xzvf /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz -C /tmp/etcd-download-test --strip-components=1
rm -f /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz

/tmp/etcd-download-test/etcd --version
/tmp/etcd-download-test/etcdctl version
/tmp/etcd-download-test/etcdutl version

 cp /tmp/etcd-download-test/etcdutl/{etcd,etcdctl,etcdutl} /usr/local/bin/

Export all the required variables for ETCD

CODE

export ETCDCTL_ENDPOINTS='https://127.0.0.1:2379'
export ETCDCTL_CACERT='/var/lib/rancher/k3s/server/tls/etcd/server-ca.crt'
export ETCDCTL_CERT='/var/lib/rancher/k3s/server/tls/etcd/server-client.crt'
export ETCDCTL_KEY='/var/lib/rancher/k3s/server/tls/etcd/server-client.key'
export ETCDCTL_API=3

Check ETCD Performance: ( don't run on a busy production )

CODE

etcdctl check perf

Check Endpoint Status

CODE

etcdctl endpoint status --cluster --write-out=table

Check ETCD Health Status

CODE

etcdctl endpoint health --cluster --write-out=table

Alarm list for ETCD

CODE

etcdctl alarm list

ETCD compact

CODE

rev=$(etcdctl endpoint status --write-out fields | grep Revision | cut -d: -f2)
etcdctl compact $rev

etcd defrag ( dont run in degraded cluster mode )

CODE

etcdctl defrag --cluster

etcd get cluster keys ( don't run on a busy production env )

CODE

etcdctl get / --prefix --keys-only