CentOS Stream 10: Add Control Plane Node
Add additional Control Plane nodes for high availability in a multi-node Kubernetes cluster on CentOS Stream 10.
Add new Control Plane Nodes to existing Kubernetes Cluster.
This example is based on the cluster environment like follows.
It adds [dlp-1.srv.world (10.0.0.31)] as Control Plane Node to this cluster.
*Note
When etcd is started on the Control Plane, the fault tolerance of etcd is 0 for 1-2 units, so in a configuration with 2 Control Planes,
if one of them goes down, it will no longer be possible to connect to etcd and the cluster will not be able to be used normally.
+----------------------+ +----------------------+
| [ ctrl.srv.world ] | | [ dlp.srv.world ] |
| Manager Node | | Control Plane |
+-----------+----------+ +-----------+----------+
eth0|10.0.0.25 eth0|10.0.0.30
| |
------------+--------------------------+-----------
| |
eth0|10.0.0.51 eth0|10.0.0.52
+-----------+----------+ +-----------+----------+
| [ node01.srv.world ] | | [ node02.srv.world ] |
| Worker Node#1 | | Worker Node#2 |
+----------------------+ +----------------------+
Step 1
On a new Node, Configure common settings to join in Cluster.
Step 2
Add proxy setting for new Control Plane on Manager Node.
[root@ctrl ~]# vi /etc/nginx/nginx.conf
<span class="color2"># add new Control Plane</span>
stream {
upstream k8s-api {
server 10.0.0.30:6443;
<span class="color1">server 10.0.0.31:6443;</span>
}
server {
listen 6443;
proxy_pass k8s-api;
}
}
[root@ctrl ~]# systemctl reload nginx
Step 3
Confirm join command on existing Control Plane Node and also transfer certificate files to new Node with any user.
[root@dlp ~]# cd /etc/kubernetes/pki
[root@dlp pki]# tar czvf kube-certs.tar.gz sa.pub sa.key ca.crt ca.key front-proxy-ca.crt front-proxy-ca.key etcd/ca.crt etcd/ca.key
[root@dlp pki]# scp kube-certs.tar.gz centos@10.0.0.31:/tmp
[root@dlp pki]# kubeadm token create --print-join-command
kubeadm join 10.0.0.25:6443 --token m4wviz.1szeu4oamqneo8l5 --discovery-token-ca-cert-hash sha256:f7ed0f751e3370eb32367ee7b38ec48c5e48a997ca6935045b573c3b57df5b3e
Step 4
Run join command you confirmed on a new Node with [--control-plane] option.
copy certificates transferred from existing Control Plane
[root@dlp-1 ~]# mkdir /etc/kubernetes/pki
[root@dlp-1 ~]# tar zxvf /tmp/kube-certs.tar.gz -C /etc/kubernetes/pki
[root@dlp-1 ~]# kubeadm join 10.0.0.25:6443 --token m4wviz.1szeu4oamqneo8l5 \
--discovery-token-ca-cert-hash sha256:f7ed0f751e3370eb32367ee7b38ec48c5e48a997ca6935045b573c3b57df5b3e \
--control-plane
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[preflight] Running pre-flight checks before initializing the new control plane instance
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost www.srv.world] and IPs [10.0.0.31 127.0.0.1 ::1]
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost www.srv.world] and IPs [10.0.0.31 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local www.srv.world] and IPs [10.96.0.1 10.0.0.31 10.0.0.25]
.....
.....
This node has joined the cluster and a new control plane instance was created:
* Certificate signing request was sent to apiserver and approval was received.
* The Kubelet was informed of the new secure connection details.
* Control plane label and taint were applied to the new node.
* The Kubernetes control plane instances scaled up.
* A new etcd member was added to the local/stacked etcd cluster.
To start administering your cluster from this node, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Run 'kubectl get nodes' to see this node join the cluster.
Step 5
Verify settings on Manager Node. That's OK if the status of new Node turns to [STATUS = Ready].
[root@ctrl ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
dlp-1.srv.world Ready control-plane 69s v1.31.9
dlp.srv.world Ready control-plane 66m v1.31.9
node01.srv.world Ready <none> 61m v1.31.9
node02.srv.world Ready <none> 60m v1.31.9
[root@ctrl ~]# kubectl get pods -A -o wide | grep dlp-1
kube-system calico-node-m9cc9 1/1 Running 0 2m7s 10.0.0.31 dlp-1.srv.world <none> <none>
kube-system etcd-www.srv.world 1/1 Running 0 2m7s 10.0.0.31 dlp-1.srv.world <none> <none>
kube-system kube-apiserver-www.srv.world 1/1 Running 0 2m7s 10.0.0.31 dlp-1.srv.world <none> <none>
kube-system kube-controller-manager-www.srv.world 1/1 Running 0 2m7s 10.0.0.31 dlp-1.srv.world <none> <none>
kube-system kube-proxy-4r2xt 1/1 Running 0 2m7s 10.0.0.31 dlp-1.srv.world <none> <none>
kube-system kube-scheduler-www.srv.world 1/1 Running 0 2m7s 10.0.0.31 dlp-1.srv.world <none> <none>