網站集群組成所需服務器

主機名    eth0網卡           eth1網卡             服務簡介
lb01    10.0.0.5/24        172.16.1.5/24          負載服務主
lb02    10.0.0.6/24        172.16.1.6/24          負載服務備
web01   10.0.0.7/24        172.16.1.7/24          動態php服務
web02   10.0.0.8/24        172.16.1.8/24          https訪問、vpn服務
web03   10.0.0.9/24        172.16.1.9/24          ELK日志分析系統
db01    10.0.0.51/24       172.16.1.51/24         主數據庫服務
db02    10.0.0.52/24       172.16.1.52/24         從數據庫服務
nfs01   10.0.0.31/24       172.16.1.31/24         存儲服務
backup  10.0.0.41/24       172.16.1.41/24         備份服務
m01     10.0.0.61/24       172.16.1.61/24         管理、yum、監控
m02     10.0.0.62/24       172.16.1.62/24         跳板機服務

網站集群各服務器作用

A.web服務：負責處理用戶的請求，提供網頁服務，將上傳的數據需要放到存儲服務器上。涉及核心軟件：Nginx、PHP、Tomcat，NFS、定時任務服務等 B.負載均衡服務：負載均衡器，負責調度網站訪問請求，需要2臺主備服務器，同時做高可用。涉及核心軟件：Nginx和Keepalved C.數據庫服務：提供bbs、blog、www等文章文本內容的存儲和訪問。涉及核心軟件：MySQL、MariaDB、PHP等 D.存儲服務：提供blog、bbs、www等服務上的共享圖片、視頻、附件數據存儲。涉及核心軟件：NFS E.備份服務：提供全網數據的備份服務，以及異地災備、涉及軟件核心：Rsync服務 F.實時同步服務：提供nfs服務與rsync服務實時數據同步，實現數據無差別還原與備份。涉及核心軟件：sersync G.管理服務：管理所有服務器，包括批量分發文件、批量執行命令、批量配置服務、跳板機服務，遠程撥號服務，yum倉庫服務等。涉及核心軟件：SSH、Ansible、跳板機等

網站集群架構圖及思路

架構圖

思路：該架構是以web集群作為始發點，通過nginx+PHP相關服務搭建出aaa.xxx.com的基礎網站，其中https訪問，用戶在通過負載均衡的分配訪問到不同的web服務器上，進行瀏覽；當有數據發生變化的同時通過數據庫的遷移保證動態數據信息實時同步，而數據庫服務器與存儲服務器保持共享存儲從而達到靜態數據信息的實時同步，再通過sersync實時同步將nfs服務器與數據庫服務器中的數據備份到rsync服務器中,最后使用定時任務將整個架構中的重要數據、配置文件、腳本信息等進行壓縮備份。而內部人員可通過vpn、跳板機、管理服務器對整個集群架構進行統一管理，同時記錄集群日志和實時監控。

網站集群實現目標

實現目標為，當用戶上傳文本、音頻、附件等數據信息的同時，做到數據庫、存儲服務器皆有該數據信息，并在定時任務后備份服務器中也存在該數據。

網站集群各服務器基礎優化（模板機）

1.配置解析文件

 vim /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
10.0.0.5 lb01
10.0.0.6 lb02
10.0.0.7 web01
10.0.0.8 web02
10.0.0.9 web03
10.0.0.31 nfs01
10.0.0.41 backup
10.0.0.51 db01
10.0.0.61 m01
10.0.0.71 zabbix

2.修改主機名稱和網卡地址（需添加多塊網卡）

修改主機名稱: hostnamectl set-hostname 名稱

修改主機網卡地址: sed -ri 's#200#ip#g;/UUID|HWADDR/d'
/etc/sysconfig/network-scripts/ifcfg-eth[01] 3.更新yum源 a.curl -o
/etc/yum.repos.d/centos-Base.repo
http://mirrors.aliyun.com/repo/Centos-7.repo b.wget -O
/etc/yum.repos.d/epel.repo
http://mirrors.aliyun.com/repo/epel-7.repo c.rpm -Uvh
https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm d.rpm -Uvh
https://mirror.webtatic.com/yum/el7/webtatic-release.rpm e.yum clean all

4.關閉selinux

 vim /etc/selinux/config 
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

5.關閉防火墻 systemctl stop firewalld systemctl disable firewalld 6.修改為英文字符集 echo 'LANG="en_US.UTF-8"' >/etc/locale.conf source /etc/locale.conf echo $LANG 7.時間同步 ntpdate ntp1.aliyun.com 8.優化內核

cat >>/etc/sysctl.conf<<EOF
net.ipv4.tcp_fin_timeout = 2
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_keepalive_time = 600
net.ipv4.ip_local_port_range = 4000    65000
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_max_tw_buckets = 36000
net.ipv4.route.gc_timeout = 100
net.ipv4.tcp_syn_retries = 1
net.ipv4.tcp_synack_retries = 1
net.core.somaxconn = 16384
net.core.netdev_max_backlog = 16384
net.ipv4.tcp_max_orphans = 16384
EOF

9.加大文件描述符

echo '*               -       nofile          65535 ' >>/etc/security/limits.conf

10.設置/tmp權限為777 chmod 777 /tmp

網站集群架構開始

1）web01==>LNMP動態網站搭建

A.環境部署

安裝官方源的nginx服務

vim /etc/yum.repos.d/nginx.repo 
[nginx-stable]
name=nginx stable repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=1
enabled=1
gpgkey=https://nginx.org/keys/nginx_signing.key

yum install nginx -y systemctl start nginx systemctl enable nginx 安裝mariadb數據庫服務 yum install mariadb-server mariadb -y systemctl start mariadb.service systemctl enable mariadb.service 安裝php-fpm

yum remove php-mysql php php-fpm php-common
rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
rpm -Uvh https://mirror.webtatic.com/yum/el7/webtatic-release.rpm
yum install -y php71w php71w-cli php71w-common php71w-devel php71w-embedded  php71w-gd php71w-mcrypt php71w-mbstring php71w-pdo php71w-xml php71w-fpm php71w-mysqlnd php71w-opcache  php71w-pecl-memcached php71w-pecl-redis php71w-pecl-mongodb

vim /etc/php-fpm.d/www.conf
 user = nginx
 group = nginx

systemctl start  php-fpm

B.搭建aaa.xxx.com

下載指定軟件包：
https://cn.wordPress/ target=_blank class=infotextkey>WordPress.org/download/

vim /etc/nginx/cond.f/blog.conf
server         {
  listen       80;
  server_name  aaa.xxx.com;
  location  /  {
    root       /usr/share/nginx/html/blog;
    index      index.php index.html;
  }
  location ~ .php$ {
    root       /usr/share/nginx/html/blog;
    fastcgi_pass  127.0.0.1:9000;
    fastcgi_index index.php;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    include fastcgi_params;
    }
}

systemctl restart nginx

cd /usr/share/nginx/html/
rz -y 選擇軟件包位置，進行傳輸
tar xf    wordpress-5.2.3-zh_CN.tar.gz
mv wordpress  blog
rm -rf   wordpress-5.2.3-zh_CN.tar.gz
chown -R nginx.  blog/

創建數據庫信息

mysql -uroot -p123456
create database wordpress;
show databases;
grant all on wordpress.* to 'wordpress'@'localhost'  identified by '123456';
select user,host from mysql.user;

本地解析hosts 在瀏覽器輸入ip地址按步安裝即可，輸入對應的數據庫名稱及密碼

aaa.xxx.com 同理

2）web02==>動態網站實現https訪問

vim www.conf 
server {
       listen            80;
       server_name       www.oldboy.com;
       rewrite  ^/(.*)$  https://$host/$1 redirect;
}
server         {
  listen       443 ssl;
  server_name  www.oldboy.com;
  ssl_certificate       server.crt;
  ssl_certificate_key   server.key;
  location  /  {
    root       /usr/share/nginx/html/www/install_package;
    index      index.php index.html;
  }
  location ~ .php$ {
    root       /usr/share/nginx/html/www/install_package;
    fastcgi_pass  127.0.0.1:9000;
    fastcgi_index index.php;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    include fastcgi_params;
    }
}

3）lb01/lb02==>nginx+keppalived負載均衡高可用

負載均衡：

vim /etc/nginx/nginx.com
user  nginx;
worker_processes  1;
error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;
events {
    worker_connections  1024;
}
http {
    include       /etc/nginx/mime.types;
    default_type  Application/octet-stream;
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    access_log  /var/log/nginx/access.log  main;
    sendfile        on;
    #tcp_nopush     on;
    keepalive_timeout  65;
    #gzip  on;
    upstream oldboy {
       server   172.16.1.7:80;
    }
    server{
      listen      80;
      server_name localhost;
      location / {
           proxy_pass   http://oldboy;
           proxy_set_header Host  $host;
           proxy_set_header X-Forwarded-For  $remote_addr;
           proxy_next_upstream error timeout invalid_header http_404;
       }
    }
}

本地解析10.0.0.5 aaa.xxx.com 高可用：負載均衡環境準備

yum install -y keepalived
echo 'net.ipv4.ip_nonlocal_bind = 1' >>/etc/sysctl.conf
sysctl -p
vim /etc/nginx/nginx.conf
server {
      listen        10.0.0.3:80;
      server_name   localhost;
systemctl  restart nginx

lb01：

vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived      
global_defs {                           
notification_email {            
acassen@firewall.loc                
failover@firewall.loc
sysadmin@firewall.loc
 }
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id lb01                        
}
vrrp_instance oldboy {                
state MASTER                                   
interface eth0                  
virtual_router_id 63               
priority 110                    
advert_int 1                       
authentication {                 
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
             10.0.0.3                     
         }
     }

lb02

vim /etc/keepalived/keepalived.conf
 ! Configuration File for keepalived      
global_defs {                         
        notification_email {               
          acassen@firewall.loc                
          failover@firewall.loc
          sysadmin@firewall.loc
        }
        notification_email_from Alexandre.Cassen@firewall.loc
        smtp_server 192.168.200.1
        smtp_connect_timeout 30
        router_id lb02                     
     }
vrrp_instance oldboy {                   
         state BACKUP                                     
         interface eth0                   
         virtual_router_id 63        
         priority 100                        
         advert_int 1                      
         authentication {                     
             auth_type PASS
             auth_pass 1111
         }
         virtual_ipaddress {
             10.0.0.3                        
         }
     }

systemctl start keepalived.service

本地解析：10.0.0.3 腦裂及nginx服務停止保證主備轉換：

mkdir -p /server/scripts
vim /server/scripts/check_keepalivde.sh
 #!/bin/bash
       ip a|grep 10.0.0.3 &>/dev/null
       if [ $? -eq 0 ]
       then
          echo "master and backup change"|mail -s "check keepalived server" xxx@163.com
       fi
crontab -e
* * * * *   sh /server/scripts/check_keepalived.sh
vim /server/scripts/check_web.sh
      #!/bin/bash
       port_info=$(netstat -lntup|grep -w 80|wc -l)
       if [ $port_info -eq 0 ]
       then
          systemctl stop keepalived
       fi
chmod +x /server/srcipts/check_web.sh
vim /etc/keepalived/keepalived.conf
vrrp_script check_web {                   
           script "/server/scripts/check_web.sh" 
           interval 2                             
           weight 2 
}
        track_script {                              
             check_web 
}

4）mariadb==>實現數據庫遷移

web01服務器端(bbs、blog)

mysqldump -uroot -p123456  -A  > /tmp/mysql_bak.sql
rsync -avz /tmp/mysql_bak.sql 172.16.1.51:/tmp
vim /usr/share/nginx/html/blog/wp-config.php 
define('DB_HOST', '172.16.1.51');
vim /usr/share/nginx/html/bbs/config/config_global.php
$_config['db']['1']['dbhost'] = '172.16.1.51';
vim /usr/share/nginx/html/bbs/config/config_ucenter.php
define('UC_DBHOST', '172.16.1.51');

web02服務器端（www）

mysqldump -uroot -p123456  -A  > /tmp/phpcms_bak.sql
rsync -avz /tmp/phpcms_bak.sql 172.16.1.51:/tmp
vim /usr/share/nginx/html/www/install_package/caches/configs/database.php
'hostname' => '172.16.1.51',

數據庫服務器端

mysql  -uroot -p123456 <  /tmp/mysql_bak.sql
mysql -uroot -p123456 < /tmp/phpcms_bak.sql
mysql -uroot -p123456
grant all on wordpress.* to 'wordpress'@'172.16.1.%' identified by '123456';
grant all on ultrax.* to ''root@'172.16.1.%' identified by '123456';
grant all on phpcmsv9.* to 'phpcmsv9'@'172.16.1.%' identified by '123456';
exit
systemctl restart mariadb

5）nfs==>實現網站上傳數據實時存儲

web01服務器

mkdir {/tmp/wordpress_bak，/tmp/discuz_bak}
mv  /usr/share/nginx/html/blog/wp-content/uploads/*  /tmp/wordpress_bak
mv /usr/share/nginx/html/bbs/data/attachment/forum/* /tmp/discuz_bak/
mount -t nfs 172.16.1.31:/data/bbs  /usr/share/nginx/html/bbs/data/attachment/forum
mount -t nfs 172.16.1.31:/data/blog  /usr/share/nginx/html/blog/wp-content/uploads 
mv   /tmp/wordpress_bak/*   /usr/share/nginx/html/blog/wp-content/uploads
mv /tmp/discuz_bak/* /usr/share/nginx/html/bbs/data/attachment/forum

web02服務器

mkdir  /tmp/phpcms_bak
mv /usr/share/nginx/html/www/install_package/uploadfile/poster/* /tmp/phpcms_bak/
mount -t nfs 172.16.1.31:/data/www  /usr/share/nginx/html/www/install_package/uploadfile/poster
mv /tmp/phpcms_bak/* /usr/share/nginx/html/www/install_package/uploadfile/poster

nfs服務器

mkdir -p /data/{bbs,blog,www}
vim /etc/exports
/data/blog 172.16.1.0/24(rw,sync,no_root_squash)
/data/www 172.16.1.0/24(rw,sync,no_root_squash)
/data/bbs 172.16.1.0/24(rw,sync,no_root_squash)
useradd -M -s /sbin/nologin nginx  -u 995（與相關服務的屬主屬組及uid保持一致）
chown  -R nginx.   /data
systemctl restart nfs

配置自動掛載文件

6）rsync==>定時任務實現與nfs服務器數據備份與還原

backup服務器端

yum install -y rsync
vim /etc/rsyncd.conf
uid = rsync            
gid = rsync       
port = 873             
fake super = yes 
use chroot = no        
max connections = 200  
timeout = 300      
lock file = /var/run/rsync.lock  
log file = /var/log/rsyncd.log     
ignore errors                      
read only = false             
list = false                     
hosts allow = 172.16.1.0/24      
hosts deny = 0.0.0.0/32        
auth users = rsync_backup
secrets file = /etc/rsync.password 
[backup_db]                        
comment = "backup dir by oldboy"  
path = /backup/db/
[backup_nfs]                        
comment = "backup dir by oldboy"  
path = /backup/nfs/                   
useradd  -M  -s  /sbin/nologin  rsync
echo rsync_backup:123456  >  /etc/rsync.password
chmod 600 /etc/rsync.password
mkdir  /dackup
chown  rsync.rsync  /dackup
systemctl start  rsyncd

nfs服務器端

yum install -y rsync
echo  123456   >/etc/rsync.password
chmod 600 /etc/rsync.password

nfs服務器定時任務

crontab -e
30  2  *  *  *  /usr/bin/rsync -avz  /data/  rsync_backup@172.16.1.41::backup_nfs   --password-file=/etc/rsync.password

7）sersync==>實時同步數據庫服務與備份服務

數據庫服務

yum install -y lrzsz
mkdir -p /server/tools
rz -y ==>選擇路徑
unzip sersync_installdir_64bit.zip
cd /server/tools/sersync_installdir_64bit
mv sersync/ /usr/local/
vim /usr/local/sersync/conf/confxml.xml
    <filter start="false">
        <exclude expression="(.*).svn"></exclude>
        <exclude expression="(.*).gz"></exclude>
        <exclude expression="^info/*"></exclude>
        <exclude expression="^static/*"></exclude>
    </filter>
    <inotify>
        <delete start="true"/>
        <createFolder start="true"/>
        <createFile start="false"/>
        <closeWrite start="true"/>
        <moveFrom start="true"/>
        <moveTo start="true"/>
        <attrib start="false"/>
        <modify start="false"/>
    </inotify>

    <sersync>
        <localpath watch="/var/lib/mysql/">
            <remote ip="172.16.1.41" name="backup_db"/>
            <!--<remote ip="192.168.8.39" name="tongbu"/>-->
            <!--<remote ip="192.168.8.40" name="tongbu"/>-->
        </localpath>
        <rsync>
            <commonParams params="-az"/>
            <auth start="true" users="rsync_backup" passwordfile="/etc/rsync.password"/>
            <userDefinedPort start="false" port="874"/><!-- port=874 -->
            <timeout start="false" time="100"/><!-- timeout=100 -->
            <ssh start="false"/>
        </rsync>
        <failLog path="/tmp/rsync_fail_log.sh" timeToExecute="60"/><!--default every 60mins execute once-->
        <crontab start="false" schedule="600"><!--600mins-->
            <crontabfilter start="false">
                <exclude expression="*.php"></exclude>
                <exclude expression="info/*"></exclude>
            </crontabfilter>
        </crontab>
        <plugin start="false" name="command"/>
    </sersync>

    <plugin name="command">
        <param prefix="/bin/sh" suffix="" ignoreError="true"/>  <!--prefix /opt/tongbu/mmm.sh suffix-->
        <filter start="false">
            <include expression="(.*).php"/>
            <include expression="(.*).sh"/>
        </filter>
    </plugin>

    <plugin name="socket">
        <localpath watch="/opt/tongbu">
            <deshost ip="192.168.138.20" port="8009"/>
        </localpath>
    </plugin>
    <plugin name="refreshCDN">
        <localpath watch="/data0/htdocs/aaa.xxx.com/site/">
            <cdninfo domainname="aaa.xxx.com" port="80" username="xxxx" passwd="xxxx"/>
            <sendurl base="http://pic.xoyo.com/cms"/>
            <regexurl regex="false" match="cms.xoyo.com/site([/a-zA-Z0-9]*).xxx.com/images"/>
        </localpath>
    </plugin>
</head>
cd /usr/local/sersync/bin/
chmod +x sersync
yum install -y rsync
echo  123456   >/etc/rsync.password
chmod 600 /etc/rsync.password
./sersync -dro /usr/local/sersync/conf/confxml.xml

8）ansible==>實現一鍵化ssh遠程連接

vim /server/scripts/distribute_key.sh 
#!/bin/bash
. /etc/init.d/functions

# 創建秘鑰對
if [ ! -f /root/.ssh/id_dsa ]
then
  ssh-keygen -t dsa -f /root/.ssh/id_dsa -P "" >/dev/null
  action "key pair create"  /bin/true
  else
      action "key pair already exists" /bin/false
  fi

# 分發公鑰信息
for ip in 7 31 41 5 6 51 8
do
  sshpass -p123456 ssh-copy-id -i ~/.ssh/id_dsa.pub 172.16.1.$ip -o StrictHostKeyChecking=no &>/dev/null
  if [ $? -eq 0 ]
  then
    action "host 172.16.1.$ip pub_key distribute"  /bin/true
    echo ""
    else
      action "host 172.16.1.$ip pub_key distribute"  /bin/false
      echo ""
    fi
done

9）rsync全網備份

nfs端： mkdir -p /server/srcipts

vim backup.sh
#!/bin/bash

Backup_IP=$(hostname -I|awk '{print $NF}')

mkdir -p /backup/$Backup_IP
  
# 1.
tar zchPf /backup/$Backup_IP/system_backup_$(date +%F_%w).tar.gz /var/spool/cron/root /etc/rc.local /server/scripts /etc/
exports

# 2.
find /backup/ -type f -name "*.tar.gz"|xargs md5sum >/backup/$Backup_IP/finger.txt

# 3.
rsync -az /backup/  rsync_backup@172.16.1.41::backup_backup --password-file=/etc/rsync.password

# 4.
find /backup/ -type f -name  "*.tar.gz"  -mtime +7 -delete

crontab -e
0 2 * * * /usr/bin/sh /server/scripts/backup.sh

web01端： mkdir /server/srcipts

vim backup.sh
#!/bin/bash

Backup_IP=$(hostname -I|awk '{print $NF}')
Backuo_html="/usr/share/nginx/html"
Backup_mysql="/var/lib/mysql"

mkdir -p /backup/$Backup_IP

# 1.
tar zchPf /backup/$Backup_IP/web_info_backup_$(date +%F_%w).tar.gz $Backup_html/bbs $Backup_html/blog
tar zchPf /backup/$Backup_IP/web_log_backup_$(date +%F_%w).tar.gz $Backup_mysql/ultrax $Backup_mysql/wordpress
tar zchPf /backup/$Backup_IP/system_backup_$(date +%F_%w).tar.gz /etc/rc.local /server/srcipts /var/spool/cron/root /etc/
nginx/conf.d/bbs.conf /etc/nginx/nginx.conf /etc/nginx/conf.d/blog.conf
# 2.
find /backup/ -type f -name "*.tar.gz"|xargs md5sum >/backup/$Backup_IP/finger.txt

# 3.
rsync -az /backup/ rsync_backup@172.16.1.41::backup_backup --password-file=/etc/rsync.password

# 4.
find /backup/ -type f -name  "*.tar.gz"  -mtime +7 -delete

echo 123456 > /etc/rsync.password
crontab -e
0 2 * * * /usr/bin/sh /server/scripts/backup.sh

web02端：同理 backup端：

mkdir -p /server/srcipts
vim /server/scripts/backup.sh
#!/bin/bash

# 1.
find /backup/backup/ -type f -name "finger.txt"|xargs md5sum -c >/tmp/check_data.txt

# 2.
find /backup/backup/ -type f -name "*.tar.gz" -mtime +180 ! -iname "*_2.tar.gz" -delete

crontab -e
0 6 * * * /usr/bin/sh /server/scripts/backup.sh

日日操夜夜添-日日操影院-日日草夜夜操-日日干干-精品一区二区三区波多野结衣-精品一区二区三区高清免费不卡

基于Linux系統的高可用中小型網站集群架構說明網站集群組成所需服務器網站集群各服務器作用網站集群架構圖及思路網站集群實現目標網站集群各服務器基礎優化（模板機）網站集群架構開始