一、注意事项

升级前最好备份所有组件及数据，例如etcd

不要跨两个大版本进行升级，可能会存在版本bug

1.19.4-->1.20.4 可以

1.19.4-->1.21.4 不可以

跨多个版本的可以逐个版本进行升级。

二、查看当前版本

[root@k8s-master1 ~]# kubectl get nodes
NAME          STATUS   ROLES                  AGE   VERSION
k8s-master1   Ready    control-plane,master   41d   v1.20.4
k8s-node1     Ready    jenkins,node           41d   v1.20.4
k8s-node2     Ready    gitlab,node            38d   v1.20.4

目前我的版本是v1.20.4，官网最新版本已经到1.25.0。升级到最新版本需要逐步升级：v1.20.4-->v1.21.4-->v1.22.4-->v1.23.4-->v1.24.4-->v1.25.0，共升级5次。

三、确定当前版本

先配置阿里云yum源，每台节点都需要配置

[root@k8s-master1 yum.repos.d]# pwd
/etc/yum.repos.d
[root@k8s-master1 yum.repos.d]# cat kubernetes.repo 
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg

配置后更新yum源，执行命令：

#yum clean all
#yum makecache

查询版本，会列出目前所有的正式版本，我们现在从1.20.4升级到1.21版本，所以需要找到v.1.21.4版本号

[root@k8s-master1 yum.repos.d]# yum list --showduplicates kubeadm --disableexcludes=kubernetes

四、开始升级---升级 master 节点

4.1 在 master 节点上执行如下命令，升级 kubeadm，对应的版本必须一致v1.24.4

如果是高可用集群，找其中一台master执行以下命令

[root@k8s-master1 ~]# yum install -y kubeadm-1.21.4-0 --disableexcludes=kubernetes

Upgrade  1 Package

Total download size: 9.1 M
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
61c56c520cec529ff02ca33f37f190d23253acff6e84bd695cc045cdd4f52b2e-kubeadm-1.21.4-0.x86_64.rpm                                                 | 9.1 MB  00:01:24     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : kubeadm-1.21.4-0.x86_64                                                                                                                          1/2 
  Cleanup    : kubeadm-1.20.4-0.x86_64                                                                                                                          2/2 
  Verifying  : kubeadm-1.21.4-0.x86_64                                                                                                                          1/2 
  Verifying  : kubeadm-1.20.4-0.x86_64                                                                                                                          2/2 

Updated:
  kubeadm.x86_64 0:1.21.4-0                                                                                                                                         

Complete!

4.2、在同一台master上继续继续以下命令，kubeadm更新计划

[root@k8s-master1 ~]# kubeadm upgrade plan

kubeadm更新计划会打印出目前能支持到的版本，显示我最高可以升级到v1.21.14，由于前面kubeadmin更新的是v1.21.4版本，后续也必须保持统一版本，如果这里apply了1.21.14会报错 在打印信息中可以看到，升级集群每个组件对应的当前版本和升级后的版本。而且升级的组件只包括kube-apiserver，kube-controller-manager，kube-scheduler，kube-proxy，CoreDNS，etcd 不包括kubectl,kubelet,docker和网络组件flannel等

4.3 根据上面输出提示，执行如下命令以升级：

[root@k8s-master1 ~]# kubeadm upgrade apply v1.21.4
....
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.21.4". Enjoy!

[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven't already done so.
#升级成功

如果是高可用还需要在其他 master 节点上执行命令:

kubeadm upgrade node

在所有的 master 节点上执行如下命令升级 kubelet 和 kubectl

[root@k8s-master1 ~]# yum install -y kubelet-1.21.4-0 kubectl-1.21.4-0 --disableexcludes=kubernetes

执行如下命令，以重启 kubelet

[root@k8s-master1 ~]# systemctl daemon-reload && systemctl restart kubelet

五、升级 node 节点

5.1 在所有的 node 节点上执行如下命令，升级 kubeadm

[root@k8s-node2 ~]# yum install -y kubeadm-1.21.4-0 --disableexcludes=kubernetes

5.2 升级 kubelet 的配置，在所有node节点上执行

[root@k8s-node2 ~]# kubeadm upgrade node
[upgrade] Reading configuration from the cluster...
[upgrade] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[preflight] Running pre-flight checks
[preflight] Skipping prepull. Not a control plane node.
[upgrade] Skipping phase. Not a control plane node.
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[upgrade] The configuration for this node was successfully updated!
[upgrade] Now you should go ahead and upgrade the kubelet package using your package manager.

5.3 升级 kubelet 和 kubectl

[root@k8s-node2 ~]# yum install -y kubelet-1.21.4-0 kubectl-1.21.4-0 --disableexcludes=kubernetes

执行如下命令，以重启 kubelet

[root@k8s-node2 ~]# systemctl daemon-reload && systemctl restart kubelet

六、检查集群的状态

版本已经从v1.20.4升级到v1.21.4

[root@k8s-master1 ~]# kubectl get nodes 
NAME          STATUS   ROLES                  AGE   VERSION
k8s-master1   Ready    control-plane,master   41d   v1.21.4
k8s-node1     Ready    jenkins,node           41d   v1.21.4
k8s-node2     Ready    gitlab,node            38d   v1.21.4