星伴同行

千里之行，始于脚下;会当凌绝顶，一览众山小。

K8s系列K8s安装

菜单

1.基础环境（三个服务器都配置）

1.1 修改hosts和修改hostname

###########修改hosts##############
shell> cat >> /etc/hosts << EOF
192.168.8.10 k8s-master
192.168.8.11 k8s-node1
192.168.8.12 k8s-node2
EOF

###########修改hostname#############
shell>hostnamectl set-hostname k8s-master
shell>vim /etc/hostname
shell>reboot -h

1.2 确认MAC和product_uuid的唯一

shell>ifconfig -a
shell>cat /sys/class/dmi/id/product_uuid

注: 如果你的centos7没有ifconfig命令, 可以执行yum install net-tools进行安装.

1.3 关闭防火墙firewall

shell> systemctl disable firewalld.service
shell> systemctl stop firewalld.service
shell> systemctl status firewalld.service

1.4 禁用SELinux

修改/etc/selinux/config, 设置SELINUX=disabled. 重启机器.

[root@k8s-master ~]$ sestatus	# 查看SELinux状态
SELinux status: disabled

1.5 禁用交换分区

编辑/etc/fstab, 将swap注释掉. 重启机器.

[root@k8s-master ~]$ vim /etc/fstab 
#/dev/mapper/cl-swap     swap                    swap    defaults        0 0

注意：swapoff –a  该命令只是临时禁用

K8s系列K8s安装插图
注释禁用swap

2.配置yum aliyun源（三个服务器都配置）

2.1 docker

shell> cd /etc/yum.repos.d/
shell> wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

Docker官方文档对安装步骤描述已经足够详细, 过程并不复杂, 本文便不再赘述.

  • Docker请使用18.09, k8s暂不支持Docker最新版19.x, 安装时请按照文档描述的方式明确指定版本号yum install docker-ce-18.09.9-3.el7 docker-ce-cli-18.09.9-3.el7 containerd.io.
  • 若网络不好, 可换用国内源, 阿里云、中科大等都可. 此处附上阿里云源docker安装文档地址: 容器镜像服务.
  • 安装完毕后, 建议将docker源替换为国内. 推荐阿里云镜像加速, 有阿里云账号即可免费使用.阿里云 -> 容器镜像服务 -> 镜像中心 -> 镜像加速

配置Docker

修改/etc/docker/daemon.json为如下内容:

{
  "registry-mirrors": ["https://9hriuiyg.mirror.aliyuncs.com"],
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}

2.2 kubernetes

shell> cat < /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

3.安装软件（三个服务器都配置）

3.1 安装kubelet kubeadm kubectl 

###查看可以下载的版本
shell> yum --showduplicates list kubelet
shell> yum install -y kebelet-1.26.0-0 kubeadm-1.26.0-0 kubectl-1.26.0-0
shell> yum install kubelet kubeadm kubectl
shell> systemctl enable kubelet && systemctl start kubelet

3.2 修改网络配置

cat <<EOF >  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system

注意: 至此, 以上的全部操作, 在Worker机器上也需要执行. 注意hostname等不要相同.

4.master节点，初始化Master

4.11 master节点上执行

[root@k8s-master ~]$ kubeadm config print init-defaults > kubeadm-init.yaml

该文件有两处需要修改:

  • 将advertiseAddress: 192.168.28.110修改为本机地址
  • 将imageRepository: k8s.gcr.io修改为imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
  • kubernetesVersion 更改此处的版本号和kubeadm version一致
apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 192.168.28.110
  bindPort: 6443
nodeRegistration:
  criSocket: unix:///var/run/containerd/containerd.sock
  imagePullPolicy: IfNotPresent
  name: node
  taints: null
---
apiServer:
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns: {}
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: v1.26.0
networking:
  dnsDomain: cluster.local
  serviceSubnet: 10.96.0.0/12
scheduler: {}

下载镜像

[root@k8s-master ~]$ kubeadm config images pull --config kubeadm-init.yaml

shell> kubeadm init --image-repository registry.aliyuncs.com/google_containers --pod-network-cidr=10.244.0.0/16 (--service-cidr=10.96.0.0/12 --token-ttl 0) #--token永不过期
shell> kubeadm init --image-repository registry.aliyuncs.com/google_containers --pod-network-cidr=10.244.0.0/16 --service-cidr=10.96.0.0/12 --token-ttl 0 #--token永不过期
shell> kubeadm init --image-repository=registry.cn-zhangjiakou.aliyuncs.com/k8sx --pod-network-cidr=10.244.0.0/16

4.2 状态为NotReady，coredns总是Pending，需要部署flannel。

shell> kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master NotReady master 94m v1.15.1
shell> kubectl get pods --all-namespaces
NAME READY STATUS RESTARTS AGE
coredns-bccdc95cf-8qwzc 0/1 Pending 0 95m
coredns-bccdc95cf-czb4f 0/1 Pending 0 95m
etcd-k8s-master 1/1 Running 1 94m
kube-apiserver-k8s-master 1/1 Running 1 94m
kube-controller-manager-k8s-master 1/1 Running 1 94m
kube-proxy-5955l 1/1 Running 0 91m
kube-proxy-gg49s 1/1 Running 0 82m
kube-proxy-vwq94 1/1 Running 1 95m
kube-scheduler-k8s-master 1/1 Running 1 94m

4.3 为了让Pods间可以相互通信，必须安装一个网络插件，并且必须在部署任何应用之前安装，CoreDNS也是在网络插件安装之后才会启动的。

shell> kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
net-conf.json: |
{
"Network": "10.244.0.0/16",
"Backend": {
"Type": "vxlan"
}
}

5、node节点

node> kubeadm join 192.168.8.10:6443 --token kw581b.sn628jbadzvodyzx \
--discovery-token-ca-cert-hash sha256:27d40de1f2bbcdb6efa6da49d8e2fe93951f2e266f135c284f795e16e1ac96ec

5.2 默认token的有效期为24小时

master> kubeadm token list
master> kubeadm token create -ttl 0 #--重新生成新的token
master> openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
86b90357027e8c42591ec964b84a81218e939ab5348a5ef12d5e5faa8a567e01

6.检查运行的pod

shell> kubectl get pods -n kube-system -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
coredns-bccdc95cf-8cfdf 1/1 Running 0 20m 10.244.0.3 k8s-master
coredns-bccdc95cf-lvt2d 1/1 Running 0 20m 10.244.0.2 k8s-master
etcd-k8s-master 1/1 Running 0 19m 192.168.8.10 k8s-master
kube-apiserver-k8s-master 1/1 Running 0 19m 192.168.8.10 k8s-master
kube-controller-manager-k8s-master 1/1 Running 0 19m 192.168.8.10 k8s-master
kube-flannel-ds-amd64-dmhxg 1/1 Running 0 70s 192.168.8.12 k8s-node2
kube-flannel-ds-amd64-kkgdz 1/1 Running 0 70s 192.168.8.11 k8s-node1
kube-flannel-ds-amd64-scq54 1/1 Running 0 70s 192.168.8.10 k8s-master
kube-proxy-nx922 1/1 Running 0 20m 192.168.8.10 k8s-master
kube-proxy-v2xls 1/1 Running 0 11m 192.168.8.11 k8s-node1
kube-proxy-wmtqq 1/1 Running 0 9m1s 192.168.8.12 k8s-node2
kube-scheduler-k8s-master 1/1 Running 0 19m 192.168.8.10 k8s-master

7.卸载集群

7.1 想要撤销kubeadm执行的操作，首先要排除节点，并确保该节点为空，然后再将其关闭。

7.2 在master节点上运行：

shell> kubectl drain k8s-node2 --delete-local-data --force --ignore-daemonsets
shell> kubectl delete nodes k8s-node2

7.3 在master上面删除node并不会清理node上面运行的容器，需要在删除节点上面手动运行清理命令重置所有kubeadm安装状态。

7.4 按y重置状态，重置后docker ps检查容器消失。

k8s-node2> kubeadm reset

7.5 重新配置集群，使用新的参数重新运行kubeadm init或者kubeadm join即可。

8.Master隔离

8.1 默认情况下，由于安全原因，集群并不会将Pods部署在Master节点上。

8.2 只有一个Master节点，可以使用下面的命令来解除限制。

shell> kubectl taint nodes --all node-role.kubernetes.io/master-

9.issue

9.1 [ERROR NumCPU]: the number of available CPUs 1 is less than the required 2

(1).所有节点运行
shell> 2 CPUs or more

9.2 [WARNING Hostname]: hostname “centos” could not be reached

—-[WARNING Hostname]: hostname “centos”: lookup centos on 223.5.5.5:53: no such host

shell> cat >> /etc/hosts << EOF
192.168.8.3 k8s-master
EOF

9.3 [ERROR Swap]: running with swap on is not supported. Please disable swap

(1).所有节点运行
shell> swapoff -a
shell> vim /etc/fstab

/dev/mapper/centos-swap swap swap defaults 0 0

9.4 [ERROR Service-Docker]: docker service is not active, please run ‘systemctl start docker.service’

(1).所有节点运行
shell> systemctl enable docker.service
shell> systemctl start docker.service

9.5 [ERROR FileContent–proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables contents are not set to 1

(1).所有节点运行
shell> cat > /etc/sysctl.d/99-kubernetes-cri.conf <<EOF
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF
shell> sysctl --system

9.6 [WARNING Service-Kubelet]: kubelet service is not enabled, please run ‘systemctl enable kubelet.service’

(1).所有节点运行
shell> systemctl enable kubelet.service

9.7 [WARNING IsDockerSystemdCheck]: detected “cgroupfs” as the Docker cgroup driver. The recommended driver is “systemd”. Please follow the guide at https://kubernetes.io/docs/setup/cri/

(1).所有节点运行
shell> cat > /etc/docker/daemon.json <<EOF
{
"registry-mirrors": ["https://w60mq0zz.mirror.aliyuncs.com"],
"exec-opts": ["native.cgroupdriver=systemd"]
}
EOF

9.8 The connection to the server localhost:8080 was refused – did you specify the right host or port?

(1).master节点运行
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

9.9 [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 19.03.1. Latest validated version: 18.09

(1).master节点运行
shell> yum list docker-ce --showduplicates | sort -r
shell> yum install docker-ce-18.09.8-3.el7 docker-ce-cli-18.09.8-3.el7 containerd.io
shell> docker version
打赏微海报分享