试用kubeasz的aio k8s单节点一键部署
项目地址：https://github.com/easzlab/kubeasz/blob/master/docs/setup/quickStart.md
快速指南
以下为快速体验k8s集群的测试、开发环境--单节点部署(aio)，国内环境下比官方的minikube方便、简单很多。真的很适合学习使用！
1.基础系统配置
- 准备一台虚机配置内存2G/硬盘30G以上
- 最小化安装
Ubuntu 16.04 server或者CentOS 7 Minimal(本试验环境为) - 配置基础网络、更新源、SSH登录等
**本实验环境为CentOS 7.6.1810 4C2G **
2.下载文件
相关的dockfile文件：https://github.com/kubeasz/dockerfiles
# 下载工具脚本easzup，举例使用kubeasz版本2.1.0
[root@k8s-node2 ~]# export release=2.1.0
[root@k8s-node2 ~]# curl -C- -fLO --retry 3 https://github.com/easzlab/kubeasz/releases/download/${release}/easzup
[root@k8s-node2 ~]# chmod +x ./easzup
# 使用工具脚本下载
[root@k8s-node2 ~]# ./easzup -D
#从输出内容来看
[INFO] Action begin : download_all
...
[INFO] downloading docker binaries 18.09.9
...
[INFO] downloading kubeasz 2.1.0
...
[INFO] downloading kubernetes v1.16.2 binaries
...
[INFO] downloading extral binaries kubeasz-ext-bin:0.3.2
...
[INFO] downloading system packages kubeasz-sys-pkg:0.3.3
...
[INFO] downloading offline images
...
[INFO] Action successed : download_all
上述脚本运行成功后，所有文件（kubeasz代码、二进制、离线镜像）均已整理好放入目录/etc/ansilbe
/etc/ansible包含 kubeasz 版本为 ${release} 的发布代码/etc/ansible/bin包含 k8s/etcd/docker/cni 等二进制文件/etc/ansible/down包含集群安装时需要的离线容器镜像/etc/ansible/down/packages包含集群安装时需要的系统基础软件
3.配置 ssh 免密登录
文档说每个节点都要操作，啥意思，不是单节点吗
[root@k8s-node2 ~]# ssh-keygen -t rsa -b 2048 -N '' -f ~/.ssh/id_rsa
[root@k8s-node2 ~]# ssh-copy-id 192.168.200.131 #131是node1
[root@k8s-node2 ~]# ssh-copy-id 192.168.200.132
4.安装集群
4.1 容器化运行 kubeasz，详见文档
[root@k8s-node2 ~]# ./easzup -S
[INFO] Action begin : start_kubeasz_docker
[INFO] get host IP: 192.168.200.132
Loaded image: easzlab/kubeasz:2.1.0
[INFO] run kubeasz in a container
4ec599413218ebb1e686ab6ca3725cbda119ca7c6d88a422c31aeaaee2ca689a
[INFO] Action successed : start_kubeasz_docker
4.2 使用默认配置安装 aio 集群
可以看到是ansible部署的
[root@k8s-node2 ~]# docker exec -it kubeasz easzctl start-aio
....
[INFO] Action successed : start-aio
5.验证安装
验证集群版本
[root@k8s-node2 ~]# exec bash
[root@k8s-node2 ~]# kubectl version
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.2", GitCommit:"c97fe5036ef3df2967d086711e6c0c405941e14b", GitTreeState:"clean", BuildDate:"2019-10-15T19:18:23Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.2", GitCommit:"c97fe5036ef3df2967d086711e6c0c405941e14b", GitTreeState:"clean", BuildDate:"2019-10-15T19:09:08Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}
验证 scheduler/controller-manager/etcd等组件状态
[root@k8s-node2 ~]# kubectl get componentstatus
NAME AGE
scheduler <unknown>
etcd-0 <unknown>
controller-manager <unknown>
验证节点就绪 (Ready) 状态
[root@k8s-node2 ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
192.168.200.132 Ready master 2m29s v1.16.2
验证集群pod状态，默认已安装网络插件、coredns、metrics-server等
[root@k8s-node2 ~]# kubectl get pod --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-68567cdb47-tx7vx 1/1 Running 0 2m23s
kube-system coredns-68567cdb47-x6vkk 1/1 Running 0 2m23s
kube-system dashboard-metrics-scraper-76585494d8-8s29h 1/1 Running 0 2m14s
kube-system kube-flannel-ds-amd64-d82g4 1/1 Running 0 2m39s
kube-system kubernetes-dashboard-6446fb56fc-xrvw9 1/1 Running 0 2m14s
kube-system metrics-server-745cb4496f-gh49f 1/1 Running 0 2m20s
kube-system traefik-ingress-controller-6dbb6b7c96-42zmb 1/1 Running 0 2m10s
验证集群服务状态
[root@k8s-node2 ~]# kubectl get svc --all-namespaces
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes ClusterIP 10.68.0.1 <none> 443/TCP 3m57s
kube-system dashboard-metrics-scraper ClusterIP 10.68.105.145 <none> 8000/TCP 2m54s
kube-system kube-dns ClusterIP 10.68.0.2 <none> 53/UDP,53/TCP,9153/TCP 3m3s
kube-system kubernetes-dashboard NodePort 10.68.74.142 <none> 443:34310/TCP 2m54s
kube-system metrics-server ClusterIP 10.68.32.190 <none> 443/TCP 3m
kube-system traefik-ingress-service NodePort 10.68.33.209 <none> 80:23456/TCP,8080:26493/TCP 2m50s
6.dashboard访问
参考：https://github.com/easzlab/kubeasz/blob/master/docs/guide/dashboard.md
只能火狐访问，有需求参考https://xyz.uscwifi.xyz/post/kubeadm部署两主两从k8s集群/ 制作证书
查看端口
root@qqq:~# kubectl get svc -A
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes ClusterIP 10.68.0.1 <none> 443/TCP 18m
kube-system dashboard-metrics-scraper ClusterIP 10.68.31.143 <none> 8000/TCP 16m
kube-system kube-dns ClusterIP 10.68.0.2 <none> 53/UDP,53/TCP,9153/TCP 17m
kube-system kubernetes-dashboard NodePort 10.68.69.6 <none> 443:38421/TCP 16m
kube-system metrics-server ClusterIP 10.68.81.157 <none> 443/TCP 17m
kube-system traefik-ingress-service NodePort 10.68.130.218 <none> 80:23456/TCP,8080:36301/TCP 16m
火狐访问https://192.168.38.154:38421
令牌登陆admin
root@qqq:~# kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')
Name: admin-user-token-qf9sz
Namespace: kube-system
......
token: eyJhbGciOiJSUzI1NiIsImtpZCI6Ii1zNi1mUzdOSTRyYmlmT1hMSzFVRTd2TXMtM0s4RS1hTXNySmE0eVhPVUUifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi11c2VyLXRva2VuLXFmOXN6Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiIxYzBkOTQzMC04YzUxLTRlOWUtODRjMS1jZTQxM2VmNDNlOTUiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06YWRtaW4tdXNlciJ9.Ma0fcCSSWiuZgqUl7uALmbDzGzn1a0Ood638hj_jv_Z8EFY5o0qGIkPusP7ajllg9EycYHCyemqS8tgO8tMMeBx3l2AJjvfNOBjaxtHWrR_OWuZFf9kEY4GYEbDLbsUI_r7xerZNtOcZbyIGJfXAQL9_qC3_rivqxo1LtasyqWCvPzanBgtNHrO6bF19bwJFA6q_sbJUnmhYf5tuE4aQBwAfM9E916x8ZUBakbA4e16S6VF1fiQ1ce1Geipjj29WIxZKIuW6J-i2bLLA7UzRX37u_uqH2Bjd9oImfzM0gZOx_zHbuLxVNh0YEdM4c3sM62gS_BNyn4kFfYiOD3z--A
令牌登陆只读
root@qqq:~# kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep read-user | awk '{print $1}')
Name: dashboard-read-user-token-2ll8x
Namespace: kube-system
......
token: eyJhbGciOiJSUzI1NiIsImtpZCI6Ii1zNi1mUzdOSTRyYmlmT1hMSzFVRTd2TXMtM0s4RS1hTXNySmE0eVhPVUUifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtcmVhZC11c2VyLXRva2VuLTJsbDh4Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImRhc2hib2FyZC1yZWFkLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJkNGU2OWY2OC0zMTkzLTQ5MTMtYThhZS1jMWY2MzA4YWIyNDgiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06ZGFzaGJvYXJkLXJlYWQtdXNlciJ9.nFvoe5fa11SNiW0dbasv_wME3hgqTSNSKy0sMMqBrlpjoJQ7BAYihCpzaxKkFWufBM8ZESJCNngTM-uY0-mWEUUAzb8f5kYE8VcDfQm2fj-JG4m8gaQFLTC5gK3eFpnp-RuEmisd7btqQ6r6SNx1tfEjtizpVH2_h3H3Vzic1-5t09PVf77sLO5qjRDRv-jf4TZP1TxPkAv67eppy_PvoKi137us0oxK5j4hbgvARGPZPVOl1inOhpI3EvP8ASxZ21wl5tr4JiuNUTFksbJJgylsqP2h_25REd8LzVsiGvMXTQQv2jwjOnlb1-ANURsnmFmXNc12WjO0TmTHgfS-9A
7.测试
当前就一个节点测试下是否能运行pod
root@qqq:~# kubectl create deployment nginx --image=nginx
deployment.apps/nginx created
root@qqq:~# kubectl expose deployment nginx --port=80 --type=NodePort
service/nginx exposed
root@qqq:~# kubectl get pod,svc
NAME READY STATUS RESTARTS AGE
pod/nginx-86c57db685-42wdv 1/1 Running 0 98s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.68.0.1 <none> 443/TCP 32m
service/nginx NodePort 10.68.56.0 <none> 80:35963/TCP 30s
访问没有问题
root@qqq:~# kubectl scale --replicas=3 deployment/nginx
deployment.apps/nginx scaled
root@qqq:~# kubectl get pod,svc
NAME READY STATUS RESTARTS AGE
pod/nginx-86c57db685-42wdv 1/1 Running 0 3m36s
pod/nginx-86c57db685-gs6q2 0/1 ContainerCreating 0 3s
pod/nginx-86c57db685-rxv7t 0/1 ContainerCreating 0 3s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.68.0.1 <none> 443/TCP 34m
service/nginx NodePort 10.68.56.0 <none> 80:35963/TCP 2m28s
8.添加一个节点
看下帮助
root@qqq:~# docker exec -it kubeasz easzctl --help
Usage: easzctl COMMAND [args]
Cluster-wide operation:
checkout To switch to context <clustername>, or create it if not existed
destroy To destroy the current cluster, '--purge' to also delete the context
list To list all of clusters managed
setup To setup a cluster using the current context
start-aio To quickly setup an all-in-one cluster for testing (like minikube)
In-cluster operation:
add-etcd To add a etcd-node to the etcd cluster
add-master To add a kube-master(master node) to the k8s cluster
add-node To add a kube-node(work node) to the k8s cluster
del-etcd To delete a etcd-node from the etcd cluster
del-master To delete a kube-master from the k8s cluster
del-node To delete a kube-node from the k8s cluster
upgrade To upgrade the k8s cluster
Extra operation:
basic-auth To enable/disable basic-auth for apiserver
Use "easzctl help <command>" for more information about a given command.
# 分发密钥到node2
root@qqq:~# ssh-copy-id 192.168.38.155
# ubuntu默认是python3，ansible这个用的是python，做下软链接
root@qqq:~# ln -sf /usr/bin/python3 /usr/bin/python
#添加节点
root@qqq:~# docker exec -it kubeasz easzctl add-node 192.168.38.155
root@qqq:~# kubectl get node
NAME STATUS ROLES AGE VERSION
192.168.38.154 Ready master 65m v1.16.2
192.168.38.155 Ready node 83s v1.16.2
root@qqq:~# kubectl scale --replicas=5 deployment/nginx
root@qqq:~# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-86c57db685-42wdv 1/1 Running 0 36m 172.20.0.8 192.168.38.154 <none> <none>
nginx-86c57db685-bxhg4 1/1 Running 0 64s 172.20.1.2 192.168.38.155 <none> <none>
nginx-86c57db685-gs6q2 1/1 Running 0 33m 172.20.0.10 192.168.38.154 <none> <none>
nginx-86c57db685-rxv7t 1/1 Running 0 33m 172.20.0.9 192.168.38.154 <none> <none>
nginx-86c57db685-w55l5 1/1 Running 0 64s 172.20.1.3 192.168.38.155 <none> <none>
root@qqq:~# kubectl top node
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
192.168.38.154 323m 8% 1134Mi 89%
192.168.38.155 101m 2% 1174Mi 92%
root@qqq:~# kubectl top pod
NAME CPU(cores) MEMORY(bytes)
nginx-86c57db685-42wdv 0m 3Mi
nginx-86c57db685-bxhg4 0m 2Mi
nginx-86c57db685-gs6q2 0m 2Mi
nginx-86c57db685-rxv7t 0m 2Mi
nginx-86c57db685-w55l5 0m 2Mi
9.清理
以上步骤创建的K8S开发测试环境请尽情折腾，碰到错误尽量通过查看日志、上网搜索、提交issues等方式解决；当然你也可以清理集群后重新创建。
在宿主机上，按照如下步骤清理
- 1.清理集群
docker exec -it kubeasz easzctl destroy或docker exec -it kubeasz ansible-playbook /etc/ansible/99.clean.yml - 2.清理管理节点
- 清理运行的容器
easzup -C - 清理容器镜像
docker system prune -a - 停止docker服务
systemctl stop docker - 删除下载文件
rm -rf /etc/ansible /etc/docker /opt/kube - 删除docker文件
- 清理运行的容器
$ umount /var/run/docker/netns/default
$ umount /var/lib/docker/overlay
$ rm -rf /var/lib/docker /var/run/docker
上述清理脚本执行成功后，建议重启节点，以确保清理残留的虚拟网卡、路由等信息。
杂项
1、看下该程序配置的docker配置文件
root@qqq:~# cat /etc/docker/daemon.json
{
"registry-mirrors": [
"https://dockerhub.azk8s.cn",
"https://docker.mirrors.ustc.edu.cn",
"http://hub-mirror.c.163.com"
],
"max-concurrent-downloads": 10,
"log-driver": "json-file",
"log-level": "warn",
"log-opts": {
"max-size": "10m",
"max-file": "3"
},
"data-root": "/var/lib/docker"
}
2.部署kuboard
#下载这个yaml文件和镜像文件
root@qqq:~# wget https://github.com/sealstore/dashboard/releases/download/v1.0-1/kuboard.tar
root@qqq:~# tar xf kuboard.tar
root@qqq:~# kubectl apply -f manifests/
# 查看svc
root@qqq:~# kubectl get svc kuboard -n kube-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kuboard NodePort 10.68.13.250 <none> 80:32567/TCP 85s
#查看admin的token
root@qqq:~# kubectl -n kube-system get secret $(kubectl -n kube-system get secret | grep kuboard-user | awk '{print $1}') -o jsonpath='{.data.token}' | base64 --decode
eyJhbGciOiJSUzI1NiIsImtpZCI6Ii1zNi1mUzdOSTRyYmlmT1hMSzFVRTd2TXMtM0s4RS1hTXNySmE0eVhPVUUifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJrdWJvYXJkLXVzZXItdG9rZW4tbTI4ajkiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoia3Vib2FyZC11c2VyIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiZGMwYzNkNjUtYzBhNC00N2I1LWI0OGItYTcxYjIyNTVlMWZhIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmt1Ym9hcmQtdXNlciJ9.ZOZVxCyg5azml37cbs1QBTkwn6QgqWQGm8aWrkDdEW9k6cZONpAtdbPb6Qy80x5Mi0cSsiicfW_4iEZIbVy3eX6Qxk1fhmISCgNEzCLdlpq-gHh-6dAO0a0zFBnW1gs38mIwm3Uy-P6tbwYXlze0Gys0pMDp_W4fILj2phH2IQ6xwYUypmT1QW4N11GKZC66G-ApeE2XgcdJvsIQG2ofWJkwXU_S7jXL06L05tfrG0Vq359G18NxIaHre7_jWh5ecB_t7aKlw3eusjl1bnRdNw6L6vX4Jy75IJQBRxDNfOjJvWvztdplIKyEIxh4RyQIEzLLX9B4ewz6wnh8Erhwjg
#查看值只读用户的token
root@qqq:~# kubectl -n kube-system get secret $(kubectl -n kube-system get secret | grep kuboard-viewer | awk '{print $1}') -o jsonpath='{.data.token}' | base64 --decode
eyJhbGciOiJSUzI1NiIsImtpZCI6Ii1zNi1mUzdOSTRyYmlmT1hMSzFVRTd2TXMtM0s4RS1hTXNySmE0eVhPVUUifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJrdWJvYXJkLXZpZXdlci10b2tlbi04em0yayIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJrdWJvYXJkLXZpZXdlciIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjZjMTk5N2I0LTAwMDEtNDBlZS1iNDU4LWY1YTI5NmUxNTM4OCIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlLXN5c3RlbTprdWJvYXJkLXZpZXdlciJ9.kUBaEFSFReAC-iZRhnpM2FIeBz0gfvVeX7ViIoV6qp_KXzxeS-ojSNKYPH7bN57sTmVNSPTjU8kpAxQDd_HXE0qmqhRwUd7fjVi9xUriTBPuxTWE5dUE6FsE_p9R4Yfqk9ObRGLa86HMYDE4yf9N2QKP3ftLcDUgynt4oY-iaaHdH7TkNeTxf358R1SEZZadvcA9asNFEswdteEx38rSG53705QwymASGG1iDtlchWKsr6Nj2elKxE67Xxb4-TLnk5nE2dcgPDgvoJ5QKlbEhdZF4nx3Z4O8rF5WlVQcnBXUK-YoCytxQdv2cLKegFJxCh8Sdj0NktygaEB2JMob4Q
访问：http://node的IP:32567
