本文僅介紹在linux系統(tǒng)中部署minio
軟件下載
！！！強(qiáng)烈建議同一個(gè)業(yè)務(wù)系統(tǒng)使用同一固定版本，防止版本變動(dòng)帶來(lái)的不必要的問(wèn)題
下載最新版文件
wget https://dl.min.io/server/minio/release/linux-amd64/minio wget https://dl.min.io/client/mc/release/linux-amd64/mc
注意：
- 生產(chǎn)環(huán)境部署時(shí)，密碼一定要更換！！！
- 文中的IP地址一定要換成自己的實(shí)際IP地址。
- 注意防火墻的端口開(kāi)放[9000-9010]，[19000-19010]，當(dāng)然也可以更換為自己想要的端口。
部署運(yùn)行 安裝minio 和 mc chmod +x minio mv minio /usr/local/bin/ chmod +x mc mv mc /usr/local/bin/ 創(chuàng)建minio用戶 # 創(chuàng)建用戶組：-r創(chuàng)建一個(gè)系統(tǒng)賬戶 groupadd -r minio # 創(chuàng)建用戶 -M不創(chuàng)建用戶的主目錄 -r創(chuàng)建一個(gè)系統(tǒng)賬戶 -g新賬戶主組的名稱(chēng)或 ID useradd -M -r -g minio minio 腳本方式部署
首先我們建議將腳本統(tǒng)一放在/opt/minio目錄下，并授權(quán)給minio用戶，包括后面自行編寫(xiě)的導(dǎo)入導(dǎo)出腳本
# 授權(quán)用戶目錄權(quán)限 chown minio:minio /opt/minio # 賦予腳本執(zhí)行權(quán)限 chmod 777 /opt/minio/*.sh
單節(jié)點(diǎn)單驅(qū)動(dòng)器部署
不具有糾刪碼校驗(yàn)修復(fù)能力 簡(jiǎn)單、快速部署
1.創(chuàng)建和授權(quán)minio用戶數(shù)據(jù)目錄
mkdir /mnt/data chown minio:minio /mnt/data
2.啟動(dòng)腳本 /opt/minio/minio-simple-start.sh
#!/bin/bash RUNNING_USER=minio MINIO_root_USER=minioadmin MINIO_ROOT_PASSword=miniopwd MINIO_DATA=/mnt/data START_CMD="MINIO_ROOT_USER=${MINIO_ROOT_USER} MINIO_ROOT_PASSWORD=${MINIO_ROOT_PASSWORD} nohup minio server --console-address ":19000" ${MINIO_DATA} > ./minio-simple-run.log 2>&1 &" su ${RUNNING_USER} -c "${START_CMD}" echo "${START_CMD}"
3.管理操作
# 啟動(dòng)實(shí)例 ./minio-simple-start.sh # 訪問(wèn)控制臺(tái) （注意：訪問(wèn)http://ip:9000也會(huì)直接重定向到19000端口） http://ip:19000 # 查看日志 tail -100f minio-simple-run.log # 查看所有minio進(jìn)程 ps -ef |grep minio # 殺死所有minio進(jìn)程 ps -aux | grep minio | awk '{print $2}' | xargs kill
單節(jié)點(diǎn)多驅(qū)動(dòng)器部署
具有糾刪碼校驗(yàn)修復(fù)能力 下例中/mnt/disk{1...4}，用的是目錄模擬的驅(qū)動(dòng)器，實(shí)際生產(chǎn)環(huán)境中，應(yīng)掛載為4個(gè)磁盤(pán)
1.創(chuàng)建和授權(quán)minio用戶數(shù)據(jù)模擬的驅(qū)動(dòng)器目錄
mkdir /mnt/disk1 /mnt/disk2 /mnt/disk3 /mnt/disk4 chown minio:minio /mnt/disk1 /mnt/disk2 /mnt/disk3 /mnt/disk4
2.啟動(dòng)腳本 /opt/minio/minio-start.sh
#!/bin/bash RUNNING_USER=minio MINIO_ROOT_USER=minioadmin MINIO_ROOT_PASSWORD=miniopwd MINIO_DATA=/mnt/disk{1...4} # CI=true 允許根目錄磁盤(pán)作為驅(qū)動(dòng)器（模擬時(shí)使用，生產(chǎn)環(huán)境可去除） START_CMD="MINIO_ROOT_USER=${MINIO_ROOT_USER} MINIO_ROOT_PASSWORD=${MINIO_ROOT_PASSWORD} CI=true nohup minio server --console-address ":19000" ${MINIO_DATA} > ./minio-run.log 2>&1 &" su ${RUNNING_USER} -c "${START_CMD}" echo "${START_CMD}"
3.管理操作
# 啟動(dòng)實(shí)例 ./minio-start.sh # 訪問(wèn)控制臺(tái) （注意：訪問(wèn)http://ip:9000也會(huì)直接重定向到19000端口） http://ip:19000 # 查看日志 tail -100f minio-run.log # 查看所有minio進(jìn)程 ps -ef |grep minio # 殺死所有minio進(jìn)程 ps -aux | grep minio | awk '{print $2}' | xargs kill
多節(jié)點(diǎn)多驅(qū)動(dòng)器部署
具有糾刪碼校驗(yàn)修復(fù)能力 模擬準(zhǔn)備四個(gè)節(jié)點(diǎn)，每個(gè)節(jié)點(diǎn)四個(gè)驅(qū)動(dòng)器 本例中用的是目錄模擬的節(jié)點(diǎn)和驅(qū)動(dòng)器，實(shí)際生產(chǎn)環(huán)境中，應(yīng)以4個(gè)服務(wù)器作為節(jié)點(diǎn)，每個(gè)節(jié)點(diǎn)掛載為4個(gè)磁盤(pán)
1.創(chuàng)建和授權(quán)minio用戶數(shù)據(jù)模擬的節(jié)點(diǎn)以及驅(qū)動(dòng)器目錄
mkdir /mnt/mdata && mkdir /mnt/mdata/node1 /mnt/mdata/node2 /mnt/mdata/node3 /mnt/mdata/node4 mkdir /mnt/mdata/node1/disk1 /mnt/mdata/node1/disk2 /mnt/mdata/node1/disk3 /mnt/mdata/node1/disk4 mkdir /mnt/mdata/node2/disk1 /mnt/mdata/node2/disk2 /mnt/mdata/node2/disk3 /mnt/mdata/node2/disk4 mkdir /mnt/mdata/node3/disk1 /mnt/mdata/node3/disk2 /mnt/mdata/node3/disk3 /mnt/mdata/node3/disk4 mkdir /mnt/mdata/node4/disk1 /mnt/mdata/node4/disk2 /mnt/mdata/node4/disk3 /mnt/mdata/node4/disk4 # 賦予用戶和文件夾讀寫(xiě)權(quán)限，否則會(huì)無(wú)權(quán)限錯(cuò)誤 chown minio:minio /mnt/mdata # 賦予所有節(jié)點(diǎn)可讀寫(xiě)權(quán)限，否則建立驅(qū)動(dòng)器時(shí)會(huì)報(bào)錯(cuò) chmod -R 777 /mnt/mdata
2.啟動(dòng)腳本 /opt/minio/minio-multi-start.sh
#!/bin/bash RUNNING_USER=minio MINIO_HOST=192.168.1.110 MINIO_ROOT_USER="minioadmin" MINIO_ROOT_PASSWORD="miniopwd" for i in {1..4}; do START_CMD="MINIO_ROOT_USER=${MINIO_ROOT_USER} MINIO_ROOT_PASSWORD=${MINIO_ROOT_PASSWORD} nohup minio server --address "${MINIO_HOST}:900${i}" --console-address "${MINIO_HOST}:1900${i}" http://${MINIO_HOST}:9001/mnt/mdata/node1/disk{1...4} http://${MINIO_HOST}:9002/mnt/mdata/node2/disk{1...4} http://${MINIO_HOST}:9003/mnt/mdata/node3/disk{1...4} http://${MINIO_HOST}:9004/mnt/mdata/node4/disk{1...4} > ./minio-multi-run${i}.log 2>&1 &" su ${RUNNING_USER} -c "${START_CMD}" echo "${START_CMD}" done
3.管理操作
# 啟動(dòng)實(shí)例 ./minio-multi-start.sh # 訪問(wèn)控制臺(tái) （注意：訪問(wèn)http://ip:9001也會(huì)直接重定向到19001端口） http://ip:19001 http://ip:19002 ... # 查看日志 tail -100f minio-multi-run1.log tail -100f minio-multi-run2.log ... # 查看所有minio進(jìn)程 ps -ef |grep minio # 殺死所有minio進(jìn)程 ps -aux | grep minio | awk '{print $2}' | xargs kill
4.多節(jié)點(diǎn)Nginx負(fù)載均衡
upstream minio { server 192.168.1.110:9001; server 192.168.1.110:9002; server 192.168.1.110:9003; server 192.168.1.110:9004; } server{ listen 9000; server_name 192.168.1.110; ignore_invalid_headers off; proxy_buffering off; location / { proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-Host $host:$server_port; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $http_x_forwarded_proto; proxy_set_header Host $http_host; proxy_connect_timeout 300; #秒 proxy_http_version 1.1; client_body_buffer_size 100m; client_max_body_size 1000m; chunked_transfer_encoding off; proxy_ignore_client_abort on; proxy_pass http://minio; } }
然后就可以通過(guò)：192.168.1.110:9000 直接訪問(wèn)API了，當(dāng)然訪問(wèn)控制臺(tái)還是會(huì)重定向到1900{1...4}端口。
服務(wù)方式部署 官方文檔
官方文檔
https://min.io/docs/minio/linux/operations/install-deploy-manage/deploy-minio-single-node-single-drive.html
使用多驅(qū)動(dòng)器的建議
- 如果使用多驅(qū)動(dòng)器，強(qiáng)烈建議使用具有 XFS 格式磁盤(pán)的直連 JBOD 陣列，以獲得最佳性能。
- 多個(gè)磁盤(pán)確保具有相同的容量和格式，否則minio會(huì)限制每個(gè)磁盤(pán)的容量為最小的那個(gè)。
- 掛在磁盤(pán)時(shí)確保使用/etc/fstab，保證每次重啟自動(dòng)掛載磁盤(pán)。
- MinIO 不支持將具有現(xiàn)有 MinIO 數(shù)據(jù)的驅(qū)動(dòng)器任意遷移到新的掛載位置
mkfs.xfs /dev/sdb -L DISK1 mkfs.xfs /dev/sdc -L DISK2 mkfs.xfs /dev/sdd -L DISK3 mkfs.xfs /dev/sde -L DISK4 nano /etc/fstab # LABEL=DISK1 /mnt/disk1 xfs defaults,noatime 0 2 LABEL=DISK2 /mnt/disk2 xfs defaults,noatime 0 2 LABEL=DISK3 /mnt/disk3 xfs defaults,noatime 0 2 LABEL=DISK4 /mnt/disk4 xfs defaults,noatime 0 2 單節(jié)點(diǎn)單/多驅(qū)動(dòng)器部署
1.創(chuàng)建 /etc/systemd/system/minio.service 服務(wù)
cd /etc/systemd/system/ touch minio.service
/etc/systemd/system/minio.service
[Unit] Description=MinIO Documentation=https://min.io/docs/minio/linux/index.html Wants.NETwork-online.target After=network-online.target AssertFileIsExecutable=/usr/local/bin/minio [Service] WorkingDirectory=/usr/local # Run Os User User=minio Group=minio ProtectProc=invisible EnvironmentFile=/etc/default/minio ExecStartPre=/bin/bash -c "if [ -z "${MINIO_VOLUMES}" ]; then echo "Variable MINIO_VOLUMES not set in /etc/default/minio"; exit 1; fi" ExecStart=/usr/local/bin/minio server $MINIO_OPTS $MINIO_VOLUMES # Let systemd Restart this service always Restart=always # Specifies the maximum file descriptor number that can be opened by this process LimitNOFILE=65536 # Specifies the maximum number of threads this process can create TasksMax=infinity # Disable timeout logic and wait until process is stopped TimeoutStopSec=infinity SendSIGKILL=no [Install] WantedBy=multi-user.target # Built for ${project.name}-${project.version} (${project.name})
2.創(chuàng)建和授權(quán)minio用戶數(shù)據(jù)模擬的驅(qū)動(dòng)器目錄
# 用于模擬多驅(qū)動(dòng)器的目錄 mkdir /mnt/disk1 /mnt/disk2 /mnt/disk3 /mnt/disk4 chown minio:minio /mnt/disk1 /mnt/disk2 /mnt/disk3 /mnt/disk4 # 用于單驅(qū)動(dòng)器的目錄 mkdir /mnt/data chown minio:minio /mnt/data
3.創(chuàng)建環(huán)境變量（注意更改用戶名/密碼）
# windows的在C:minioconfig touch /etc/default/minio vi /etc/default/minio
/etc/default/minio 文件內(nèi)容如下
# MINIO_ROOT_USER and MINIO_ROOT_PASSWORD sets the root account for the MinIO server. # This user has unrestricted permissions to perform S3 and administrative API operations on any resource in the deployment. # Omit to use the default values 'minioadmin:minioadmin'. # MinIO recommends setting non-default values as a best practice, regardless of environment MINIO_ROOT_USER=minioadmin MINIO_ROOT_PASSWORD=miniopwd # MINIO_VOLUMES sets the storage volume or path to use for the MinIO server. # 單驅(qū)動(dòng)器 # MINIO_VOLUMES="/mnt/data" # 多驅(qū)動(dòng)器（至少4個(gè)以上） MINIO_VOLUMES="/mnt/disk{1...4}" # 控制臺(tái)訪問(wèn)端口 MINIO_OPTS="--console-address :19000" # 該參數(shù)設(shè)置為true支持掛載根目錄的多驅(qū)動(dòng)器部署（不建議） CI=true # MINIO_SERVER_URL sets the hostname of the local machine for use with the MinIO Server # MinIO assumes your network control plane can correctly resolve this hostname to the local machine # Uncomment the following line and replace the value with the correct hostname for the local machine. #MINIO_SERVER_URL="http://minio.example.net"
4.管理操作
# 啟用服務(wù) systemctl enable minio.service # 啟動(dòng) sudo systemctl start minio.service systemctl start minio.service # 重啟 systemctl restart minio.service # 停止 systemctl stop minio.service # 查看狀態(tài) systemctl status minio.service journalctl -f -u minio.service 多節(jié)點(diǎn)多驅(qū)動(dòng)器
對(duì)于多節(jié)點(diǎn)驅(qū)動(dòng)器部署，至少需要4個(gè)以上的節(jié)點(diǎn)才能滿足最低的糾刪碼校驗(yàn)修復(fù)功能
多節(jié)點(diǎn)多驅(qū)動(dòng)器服務(wù)部署是官方的方式目前沒(méi)有實(shí)驗(yàn)過(guò),所以本文檔可能參考的意義不多。
建議可直接翻閱官方文檔。
多節(jié)點(diǎn)多驅(qū)動(dòng)器官方文檔
https://min.io/docs/minio/linux/operations/install-deploy-manage/deploy-minio-multi-node-multi-drive.html
MinIO 強(qiáng)烈建議為部署中的所有節(jié)點(diǎn)選擇基本相似的硬件配置。確保硬件（CPU、內(nèi)存、主板、存儲(chǔ)適配器）和軟件（操作系統(tǒng)、內(nèi)核設(shè)置、系統(tǒng)服務(wù)）在所有節(jié)點(diǎn)上保持一致。
1.按照單節(jié)點(diǎn)的步驟部署4臺(tái)存儲(chǔ)服務(wù)
部署具有單個(gè)服務(wù)器池，該池由四個(gè)具有順序主機(jī)名的 MinIO 服務(wù)器主機(jī)組成。
# IP 192.168.10.101 192.168.10.102 192.168.10.103 192.168.10.104 # 或主機(jī)名 minio1.example.com minio3.example.com minio2.example.com minio4.example.com
每個(gè)服務(wù)器上的 /etc/default/minio 文件內(nèi)容調(diào)整如下
# Set the hosts and volumes MinIO uses at startup # The command uses MinIO expansion notation {x...y} to denote a # sequential series. # # The following example covers four MinIO hosts # with 4 drives each at the specified hostname and drive locations. # The command includes the port that each MinIO server listens on # (default 9000) # MINIO_VOLUMES="http://192.168.10.10{1...4}:9000/mnt/disk{1...4}" MINIO_VOLUMES="https://minio{1...4}.example.net:9000/mnt/disk{1...4}" # Set all MinIO server options # # The following explicitly sets the MinIO Console listen address to # port 9001 on all network interfaces. The default behavior is dynamic # port selection. MINIO_OPTS="--console-address :19000" # Set the root username. This user has unrestricted permissions to # perform S3 and administrative API operations on any resource in the # deployment. # # Defer to your organizations requirements for superadmin user name. MINIO_ROOT_USER=minioadmin # Set the root password # # Use a long, random, unique string that meets your organizations # requirements for passwords. MINIO_ROOT_PASSWORD=miniopwd # 負(fù)載均衡URL設(shè)置 # Set to the URL of the load balancer for the MinIO deployment # This value *must* match across all MinIO servers. If you do # not have a load balancer, set this value to to any *one* of the # MinIO hosts in the deployment as a temporary measure. # MINIO_SERVER_URL="http://192.168.10.100:9000" MINIO_SERVER_URL="https://minio.example.net:9000"
可能的問(wèn)題 1. 遇到類(lèi)似 Unknown lvalue 'ProtectProc' in section 'Service' # 升級(jí)systemd yum install systemd-* -y 2. Error: Disk /mnt/disk1 is part of root disk # 將 /etc/default/minio 文件下 設(shè)置CI=true # 這是因?yàn)閙inio默認(rèn)不允許多驅(qū)動(dòng)器模式直接使用根目錄的磁盤(pán) CI=true
存儲(chǔ)遷移
主要是利用mc進(jìn)行遷移操作
# 從機(jī)器1 遷移到 機(jī)器2 # 設(shè)置機(jī)器1別名 mc alias set mdata http://127.0.0.1:9000 minioadmin miniopwd # 查看已設(shè)置的別名列表 mc alias list # 導(dǎo)出備份 mdata：別名，base：存儲(chǔ)桶 mc cp --recursive mdata/base/ /mnt/minio_data_bak/ # 將minio_data_bak打包遷移到機(jī)器2 # 設(shè)置機(jī)器2別名 mc alias set mdata http://127.0.0.1:9000 minioadmin miniopwd # 導(dǎo)入備份文件 mdata：別名，base：存儲(chǔ)桶 mc cp --recursive /mnt/minio_data_bak/ mdata/base/
更多MC命令可參考官方文檔：mc命令操作說(shuō)明
https://min.io/docs/minio/linux/reference/minio-mc.html#minio-client
結(jié)語(yǔ)
如有錯(cuò)誤請(qǐng)指正，非常感謝！
