Kubernetes(k8s)1.18.3高可用集群部署(多master多node)(二)

Kubernetes(k8s)1.18.3高可用集群部署(多master多node)(二) 第二部分:部署负载与高可用:nginx与keepalived  (所有master节点部署) nginx推荐使用tcp进行反向代理进行各master间轮...

Kubernetes(k8s)1.18.3高可用集群部署(多master多node)(二)


第二部分:部署负载与高可用:nginx与keepalived  (所有master节点部署)


nginx推荐使用tcp进行反向代理进行各master间轮询

keepalived用于各master间高可用


增加nginx的yum源


vim /etc/yum.repos.d/nginx.repo

[nginx-stable]
name=nginx stable repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=1
enabled=1
gpgkey=https://nginx.org/keys/nginx_signing.key
module_hotfixes=true

[nginx-mainline]
name=nginx mainline repo
baseurl=http://nginx.org/packages/mainline/centos/$releasever/$basearch/
gpgcheck=1
enabled=0
gpgkey=https://nginx.org/keys/nginx_signing.key
module_hotfixes=true

将yum源scp到其它各master节点


scp /etc/yum.repos.d/nginx.repo [email protected]:/etc/yum.repos.d/nginx.repo
scp /etc/yum.repos.d/nginx.repo [email protected]:/etc/yum.repos.d/nginx.repo


yum-config-manager --enable nginx-mainline

各master节点安装nginx与keepalived:
yum -y install nginx keepalived

attachments-2020-05-XXD1YwRB5eccf395d2be8.png


nginx从1.9.0版本开始,新增了stream模块,使nginx支持四层(tcp)负载均衡  我们启动nginx并检查模块 是否含有 --with-stream 

# nginx && nginx -V


attachments-2020-05-V4AlrxIn5eccf59a78afd.png
一般yum安装1.9以后默认都包含stream模块,如果是编译安装需要在编译时添加 --with-stream

配置nginx TCP 反向代理


vim /etc/nginx/nginx.conf

在http模块配置部分的最后  加入stream模块配置      

http {
   xxxxxxxxxx     此处为http模块的配置内容  保持默认即可
}


stream {

    log_format proxy '$remote_addr [$time_local]'

                '$protocol $status $bytes_sent $bytes_received'

                '$session_time "$upstream_addr" '

                '"$upstream_bytes_sent" "$upstream_bytes_received" "$upstream_connect_time"';

    access_log /var/log/nginx/tcp-access.log proxy ;

    error_log  /var/log/nginx/tcp-error.log warn ;


   upstream k8s.nginx {
       server 192.168.40.81:6443;
       server 192.168.40.82:6443;
       server 192.168.40.83:6443;
}


  server {
       listen 16443;
       proxy_connect_timeout 8s;
       proxy_timeout 300s;
       proxy_pass k8s.nginx;

}

}

日志的配置很重要,可以通过日志查看tcp代理的情况


重启nginx   查看端口
attachments-2020-05-Nrxv9oTq5ece30da07e47.png
解释配置16443:k8s的api服务端口是6443  由于nginx与k8s master在一台服务器 因此需要自定义nginx端口,用来接受k8s客户端的请求再转发给真实的k8s api后端

所有其它master节点都需要此配置

attachments-2020-05-ypD78pCo5ece37c6204a7.png
attachments-2020-05-U5kJ98zy5ece37d7f04d1.png


配置keepalived高可用

 
安装keepalived监测机制用的killall       需要用killall来监测nginx是否正常
# yum install psmisc -y

# cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak && vim /etc/keepalived/keepalived.conf   
attachments-2020-05-qRv0sNfT5ecf90dc199ac.png
把keepalived配置文件scp到其它2个master      注意需要修改的内容

scp /etc/keepalived/keepalived.conf [email protected]:/etc/keepalived/keepalived.conf
scp /etc/keepalived/keepalived.conf [email protected]:/etc/keepalived/keepalived.conf

attachments-2020-05-6jiNoC6k5ecf91291e8e9.png
attachments-2020-05-fAFazgOE5ecf9150e1e90.png
解释:81 为主master   82-83 为backup    优先级(priority)主 40.81为90  40.82为80  40.83为70   这个需要与我们的监测机制相结合,我们监测机制是如果nginx进程不正常则优先级减21,这样81nginx挂 80的优先级本来是90 现在减去21 为 69  小于 82的80  因此82接替   82挂同理83接替  当81恢复则抢占vip

启动keepalived
#systemctl restart keepalived && systemctl enable keepalived

attachments-2020-05-fLuLmOII5ecf89b559097.png
通过测试当nginx进程丢失则master1权重减去21   vip漂移到master2 上面    

attachments-2020-05-KCQ9b1Dc5ecf91de318f9.png
attachments-2020-05-VZe7u1gZ5ecf91fe30c69.png
当进程恢复  master1抢占vip    

attachments-2020-05-QfdlATnR5ecf923a6ac7c.png

接下来我们进行第三部分 :部署k8s集群






  • 发表于 2020-05-26 18:59
  • 阅读 ( 202 )
  • 分类:Kubenetes

0 条评论

请先 登录 后评论
zhik8sadm9
zhik8sadm9

运维工程师

11 篇文章

作家榜 »

  1. zhik8sadm9 11 文章
  2. hebergemWob 0 文章
  3. liangjj 0 文章
  4. Lai Wei 0 文章
  5. noveluser 0 文章
  6. z1x2c34 0 文章
  7. zhisongwu 0 文章
  8. tankerwng 0 文章