kubernetes部署笔记 - 熊吉的博客小站

安装kubernetes步骤

一、系统初始化

系统初始化分为几个部分，第一个部分是执行初始化脚本，关闭防火墙，关闭selinux
第二个部分是关闭swap，这是k8s的需求

1	swapoff -a && sed -i '/swap/d' /etc/fstab

二、安装docker服务

# 执行脚本直接安装docker-ce 使用阿里源
curl -fsSL https://get.docker.com | bash -s docker --mirror Aliyun
# 添加用户无需root直接执行docker
sudo usermod -aG docker server
# 开启docker开机自启
systemctl enable docker
# 开启docker服务
systemctl start docker

三、安装kubernetes服务

#先配置安装源，安装kubernetes
### 如果是Debian / Ubuntu执行以下
Debian / Ubuntu
apt-get update && apt-get install -y apt-transport-https
curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add - 
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
EOF  
apt-get update
apt-get install -y kubelet kubeadm kubectl

### 如果是CentOS / RHEL / Fedora执行以下
CentOS / RHEL / Fedora
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
setenforce 0
yum install -y kubelet kubeadm kubectl
systemctl enable kubelet

四、kubernetes初始化

直接执行systemctl start kubelet是并不能正确启动的，需要进行初始化
因为初始化需要镜像，可以执行以下脚本进行初始化镜像国内源下载。

#!/bin/bash
images=(
    kube-apiserver:v1.13.3
    kube-controller-manager:v1.13.3
    kube-scheduler:v1.13.3
    kube-proxy:v1.13.3
    pause:3.1
    etcd:3.2.24
    coredns:1.2.6


    pause-amd64:3.1

    kubernetes-dashboard-amd64:v1.10.0
    heapster-amd64:v1.5.4
    heapster-grafana-amd64:v5.0.4
    heapster-influxdb-amd64:v1.5.2
)

for imageName in ${images[@]} ; do
    docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName
    docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName k8s.gcr.io/$imageName
    docker rmi registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName
done

把脚本保存到本地进行相应修改并执行，下载镜像。

# kubernetes初始化，注意需要开代理从google下载镜像
kubeadm init --pod-network-cidr=10.244.0.0/16
# kubernetes启动
systemctl start kubelet

五、kubernetes加入节点

通过kubeadm初始化后，都会提供node加入的token。
默认token的有效期为24小时，当过期以后，该token就不可用了，解决方法如下：
重新生成新的token

[root@master server]# kubeadm token create
je5176.gdhi2d95q4edcg3n
[root@master server]# kubeadm token list
TOKEN                     TTL       EXPIRES                     USAGES                   DESCRIPTION                                                EXTRA GROUPS
c9owsq.7bpfx7mbfnhd7e20   5h        2018-09-20T23:48:15+08:00   authentication,signing   The default bootstrap token generated by 'kubeadm init'.   system:bootstrappers:kubeadm:default-node-token

je5176.gdhi2d95q4edcg3n   23h       2018-09-21T18:19:16+08:00   authentication,signing   <none>    system:bootstrappers:kubeadm:default-node-token

获取ca证书sha256编码hash值

1
2

[root@master server]# openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
84ac067eebeed57154fc6f03ad0e3d10a1076f2fe268077c11e00787d3689327

节点加入集群

1	kubeadm join 192.168.239.137:6443 --token je5176.gdhi2d95q4edcg3n --discovery-token-ca-cert-hash sha256:84ac067eebeed57154fc6f03ad0e3d10a1076f2fe268077c11e00787d3689327

六、CNI网络插件加入

通过CNI插件进行部署，如安装flannel插件。

1	kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/bc79dd1505b0c8681ece4de4c0d86c5cd2643275/Documentation/kube-flannel.yml

七、拷贝kubectl 配置进行管理

1
2
3

sudo cp /etc/kubernetes/admin.conf $HOME/
sudo chown $(id -u):$(id -g) $HOME/admin.conf
export KUBECONFIG=$HOME/admin.conf

六、加入服务进行管理

1、创建实例yaml文件。

apiVersion: v1
kind: ReplicationController
metadata:
  name: mysql
spec:
  replicas: 1
  selector:
      app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - name: mysql
        image: mariadb:latest
        ports:
        - containerPort: 3386
        env:
          - name: MYSQL_ROOT_PASSWORD
            value: "123456"

执行创建rc命令

1 2	[server@lq-docker-test0 examples]$ kubectl create -f mysql-rc.yaml replicationcontroller/mysql created

查看执行效果

[server@lq-docker-test0 examples]$ kubectl get rc
NAME      DESIRED   CURRENT   READY     AGE
mysql     1         1         0         5s
[server@lq-docker-test0 examples]$ kubectl get pods
NAME          READY     STATUS    RESTARTS   AGE
mysql-r2p6w   0/1       Pending   0          58s

执行describe命令查看没有创建成功的原因。

[server@lq-docker-test0 examples]$ kubectl describe pods mysql-r2p6w
Name:           mysql-r2p6w
Namespace:      default
Node:           <none>
Labels:         app=mysql
Annotations:    <none>
Status:         Pending
IP:
Controlled By:  ReplicationController/mysql
Containers:
  mysql:
    Image:      mariadb:latest
    Port:       3386/TCP
    Host Port:  0/TCP
    Environment:
      MYSQL_ROOT_PASSWORD:  123456
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-cgbn8 (ro)
Conditions:
  Type           Status
  PodScheduled   False
Volumes:
  default-token-cgbn8:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-cgbn8
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason            Age               From               Message
  ----     ------            ----              ----               -------
  Warning  FailedScheduling  1s (x15 over 2m)  default-scheduler  0/1 nodes are available: 1 node(s) had taints that the pod didn't tolerate.

问题原因是因为master节点默认不运行其他任务，我们手动执行配置一下。

1 2	[server@lq-docker-test0 examples]$ kubectl taint nodes --all node-role.kubernetes.io/master- node/lq-docker-test0 untainted

然后查看运行效果

[server@lq-docker-test0 examples]$ kubectl describe pods mysql-r2p6w
Name:           mysql-r2p6w
Namespace:      default
Node:           lq-docker-test0/192.168.5.4
Start Time:     Thu, 27 Sep 2018 18:02:02 +0800
Labels:         app=mysql
Annotations:    <none>
Status:         Pending
IP:
Controlled By:  ReplicationController/mysql
Containers:
  mysql:
    Container ID:
    Image:          mariadb:latest
    Image ID:
    Port:           3386/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Environment:
      MYSQL_ROOT_PASSWORD:  123456
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-cgbn8 (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  default-token-cgbn8:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-cgbn8
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason            Age                From                      Message
  ----     ------            ----               ----                      -------
  Warning  FailedScheduling  19s (x24 over 3m)  default-scheduler         0/1 nodes are available: 1 node(s) had taints that the pod didn't tolerate.
  Normal   Scheduled         12s                default-scheduler         Successfully assigned default/mysql-r2p6w to lq-docker-test0
  Normal   Pulling           11s                kubelet, lq-docker-test0  pulling image "mariadb:latest"

问题排查

安装过程中出现过几次问题，我记录如下

一、k8s-dns ipv6问题

k8s需要网卡配置ipv6才能正常启动k8s-dns，如果出现k8s-dns找不到ipv6相关配置，需要排查解决一下此问题。

1
2
3

# ifconfig -a | grep inet6
        inet6 fe80::211:aff:fe6a:9de4  prefixlen 64  scopeid 0x20
        inet6 ::1  prefixlen 128  scopeid 0x10[host]

[k8s]k8s 1.9(on the fly搭建) 1.9_cni-flannel部署排错 ipvs模式
 How to disable IPv6 on CentOS / RHEL 7

二、k8s master调度问题

k8s默认不在主节点上运行容器任务，需要手动执行命令后，才在master节点上执行任务。

1	$ kubectl taint nodes --all node-role.kubernetes.io/master-

kubernetes遇到问题
 如何安装Kubernetes 集群