目錄
- 前言
- 什么是es
- 使用docker搭建es集群
前言
該系列默認(rèn)開啟Nacos 服務(wù),還不會搭建的小伙伴可以參考往期文章~
本節(jié)重點是給大家介紹利用docker來搭建Es集群,廢話不多說直接開整吧~
什么是es
同樣的，在學(xué)習(xí)之前，先了解一下這玩意到底是個啥?
es這個名詞或許大家都聽過，它的全稱是Elasticsearch,它是一個分布式文檔儲存中間件，它不會將信息儲存為列數(shù)據(jù)行，而是儲存已序列化為 JSON 文檔的復(fù)雜數(shù)據(jù)結(jié)構(gòu)。當(dāng)你在一個集群中有多個節(jié)點時，儲存的文檔分布在整個集群里面，并且立刻可以從任意節(jié)點去訪問。
當(dāng)文檔被儲存時，它將建立索引并且近實時（1s）被搜索。 Elasticsearch 使用一種被稱為倒排索引的數(shù)據(jù)結(jié)構(gòu)，該結(jié)構(gòu)支持快速全文搜索。在倒排索引里列出了所有文檔中出現(xiàn)的每一個唯一單詞并分別標(biāo)識了每個單詞在哪一個文檔中。有時候面試官會問，es為什么這么快?這也是一個小的知識點。
通過上面簡單的介紹，我們大體可以知道，它是用來做數(shù)據(jù)檢索的，而且速度特別快。
不知道小伙伴們有沒有遇到過這樣一個問題，比方說我們在用sql查商品庫表的時候,想要通過某個關(guān)鍵詞來匹配相應(yīng)的商品，當(dāng)數(shù)據(jù)量很小的時候ok,但是隨著商品數(shù)據(jù)的不斷導(dǎo)入，后期的數(shù)據(jù)量越來越大，而且都是關(guān)聯(lián)著好幾張表，這時候我們用sql去查詢我們想要的數(shù)據(jù)的時候，會顯得特別吃力，這種是相當(dāng)危險的操作，因為可能會把整張表鎖死，導(dǎo)致我們的系統(tǒng)出現(xiàn)故障，如果其它系統(tǒng)也使用這個庫，那么也會受到影響。所以這時候，我們就需要借助es這種中間件來幫我們處理這種需求，系統(tǒng)的性能也會有顯著的提升，當(dāng)然，維護(hù)上也會增加一些難度，當(dāng)然也不是啥都上es的。其實我們也可以使用其它的比如mongo，如何選取，取決于系統(tǒng)架構(gòu)和實際的業(yè)務(wù)場景。
使用docker搭建es集群
為了大家快速的體驗到es，這里推薦大家使用docker來搭建，因為它比較方便。但是生產(chǎn)中，如果你對docker不是很熟悉，維護(hù)會稍微有點麻煩，那么建議你還是到官網(wǎng)去下載具體的安裝包，本節(jié)默認(rèn)大家都已經(jīng)安裝好了docker。如果你還不知道docker是啥也沒關(guān)系，這個后邊我會專門給大家講講，本節(jié)跟著我敲就可以了。
docker的安裝非常簡單，官網(wǎng)都有具體的平臺的安裝包,win和mac都有，無腦安裝就好了。win11安裝可能會遇到wsl的問題，需要開啟linux子系統(tǒng)，如果啟動錯誤，直接百度錯誤就好了，已經(jīng)有人踩過坑了。
下面，我們進(jìn)入正題,首先啟動好docker，本節(jié)帶大家安裝的是7.6.2的版本，這個版本相對好一些，控制臺的功能也都很完善。
執(zhí)行已下命令獲取官方鏡像， 打開cmd/mac終端：
# es鏡像 docker pull docker.elastic.co/elasticsearch/elasticsearch:7.6.2 # kibana鏡像 docker pull docker.elastic.co/kibana/kibana:7.6.2
kibana它是一個可視化的平臺，我們查看數(shù)據(jù)就是通過它，es只是用作數(shù)據(jù)引擎，市面上也有一些第三方的工具，但是官方的這個已經(jīng)非常完善了，界面也很美觀。
緊接著，進(jìn)入指定安裝目錄,比方說當(dāng)前目錄叫es,終端進(jìn)入這個目錄后執(zhí)行一下命令:
# kibana數(shù)據(jù)掛載的目錄 mkdir data/kibana # 三個節(jié)點數(shù)據(jù)掛載的目錄 mkdir data/node1 mkdir data/node2 mkdir data/node3
這一步主要是創(chuàng)建相關(guān)的目錄，因為后邊docker的數(shù)據(jù)卷會映射到該目錄，這樣做的目的是防止容器意外銷毀后的數(shù)據(jù)丟失。這里為什么是三個節(jié)點,因為es集群至少需要三個節(jié)點，這是跟它的內(nèi)部機制有關(guān)，為了防止腦裂現(xiàn)象，這里就不給大家過多展開了
接下來進(jìn)入data/kibana目錄,新建kibana.yml,這個文件是它的配置文件，后邊我們會把它映射到docker容器內(nèi)部
# ## ** THIS IS AN AUTO-GENERATED FILE ** ## # # # Default Kibana configuration for docker target server.name: kibana server.host: "0" elasticsearch.hosts: [ "http://es01:9200","http://es02:9200","http://es03:9200" ] xpack.monitoring.ui.container.elasticsearch.enabled: true i18n.locale: zh-CN
elasticsearch.hosts指的是三個es節(jié)點,會和這些節(jié)點進(jìn)行通信
進(jìn)入node1，同樣新建配置文件elasticsearch.yml
# ======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: es-cluster # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: es01 # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # #path.data: /path/to/data # # Path to log files: # #path.logs: /path/to/logs # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # #bootstrap.memory_lock: true # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 0.0.0.0 # # Set a custom port for HTTP: # http.port: 9200 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when this node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # discovery.seed_hosts: ["es01","es02","es03"] # # Bootstrap the cluster using an initial set of master-eligible nodes: # cluster.initial_master_nodes: ["es01","es02","es03"] # bootstrap.memory_lock: true # # For more information, consult the discovery and cluster formation module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # #gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true http.cors.enabled: true http.cors.allow-origin: '*' http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type node.master: true
我們把node1作為主節(jié)點,也就是老大，node.master: true可以配置。為了使它支持中文分詞,我們給它安裝一下插件, 到倉庫下載指定版本的插件https://github.com/medcl/elasticsearch-analysis-ik/releases,然后我們解壓到node1根目錄，然后重新命名為ik目錄,然后再新建一個Dockerfile用來重構(gòu)“`es““鏡像，沒錯，后邊我們就使用我們重構(gòu)好的鏡像，這樣就自動安裝好了插件
Dockerfile文件內(nèi)容
FROM docker.elastic.co/elasticsearch/elasticsearch:7.6.2 COPY --chown=elasticsearch:elasticsearch elasticsearch.yml /usr/share/elasticsearch/config/ ADD ik /usr/share/elasticsearch/plugins/ik ADD ik/config /data/erms/es/node1/ik/config
下面我們進(jìn)入node2目錄，這個目錄只需要放配置文件就好了
# ======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: es-cluster # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: es02 # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # #path.data: /path/to/data # # Path to log files: # #path.logs: /path/to/logs # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # #bootstrap.memory_lock: true # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 0.0.0.0 # # Set a custom port for HTTP: # http.port: 9200 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when this node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # discovery.seed_hosts: ["es01","es02","es03"] # # Bootstrap the cluster using an initial set of master-eligible nodes: # cluster.initial_master_nodes: ["es01","es02","es03"] # bootstrap.memory_lock: true # # For more information, consult the discovery and cluster formation module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # #gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true http.cors.enabled: true http.cors.allow-origin: '*' http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type node.data: true
這里我們指定為數(shù)據(jù)節(jié)點node.data: true用來做副本
同樣的node3
# ======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: es-cluster # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: es03 # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # #path.data: /path/to/data # # Path to log files: # #path.logs: /path/to/logs # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # #bootstrap.memory_lock: true # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 0.0.0.0 # # Set a custom port for HTTP: # http.port: 9200 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when this node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # discovery.seed_hosts: ["es01","es02","es03"] # # Bootstrap the cluster using an initial set of master-eligible nodes: # cluster.initial_master_nodes: ["es01","es02","es03"] # bootstrap.memory_lock: true # # For more information, consult the discovery and cluster formation module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # #gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true http.cors.enabled: true http.cors.allow-origin: '*' http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type node.data: true
然后我們回到根目錄(es),新建一個docker-compose.yaml,我們使用docker-compose來編排我們的容器，默認(rèn)安裝好docker desktop就自動給我們安裝好了docker-compose
version: '3'
services:
es01:
image: ${image}
container_name: es01
environment:
- discovery.seed_hosts=es02,es03
- cluster.initial_master_nodes=es01,es02,es03
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./data/node1/data:/usr/share/elasticsearch/data
- ./data/node1/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
- ./data/node1/plugins:/usr/share/elasticsearch/plugins
ports:
- 9200:9200
networks:
- elastic
es02:
image: ${image}
container_name: es02
environment:
- discovery.seed_hosts=es01,es03
- cluster.initial_master_nodes=es01,es02,es03
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./data/node2/data:/usr/share/elasticsearch/data
- ./data/node2/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
- ./data/node2/plugins:/usr/share/elasticsearch/plugins
ports:
- 9201:9201
networks:
- elastic
es03:
image: ${image}
container_name: es03
environment:
- discovery.seed_hosts=es01,es02
- cluster.initial_master_nodes=es01,es02,es03
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./data/node3/data:/usr/share/elasticsearch/data
- ./data/node3/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
- ./data/node3/plugins:/usr/share/elasticsearch/plugins
ports:
- 9202:9202
networks:
- elastic
kibana:
image: ${image_kibana}
container_name: kibana
depends_on:
- es01
environment:
ELASTICSEARCH_URL: http://es01:9200
ELASTICSEARCH_HOSTS: http://es01:9200
volumes:
- ./data/kibana/kibana.yml:/usr/share/kibana/config/kibana.yml
networks:
- elastic
ports:
- 5601:5601
networks:
elastic:
driver: bridge
這個文件有點長，不懂沒關(guān)系，跟著配就完了。${image}是一個占位符，所以我們還需要指定環(huán)境變量,然后新建一個.env
image=m/es image_kibana=docker.elastic.co/kibana/kibana:7.6.2
m/es這個是我們重構(gòu)后的鏡像名稱，下面我們就來重構(gòu)鏡像
進(jìn)入data/node1執(zhí)行
docker build -t m/es .
執(zhí)行完成后，到根目錄執(zhí)行啟動命令:
docker-compose up -d
如果你想看實時日志，把-d去掉，這個是后臺運行，初次啟動，可能要花費一些時間。
啟動成功后，我們可以訪問一些es1的節(jié)點localhost:9200,可以查看節(jié)點的信息,如果顯示正常，說明已經(jīng)搭建成功了，下面我們直接進(jìn)入kibana控制臺
http://localhost:5601/,初次進(jìn)入會讓你設(shè)置控制臺的密碼
我們進(jìn)入控制臺,執(zhí)行一下，有如下輸出，至此我們就搭建成功了
如果你想卸載它們，執(zhí)行docker-compose down就可以了，畢竟這幾個家伙特別的吃資源。這里提醒一下大家，如果想嘗試到服務(wù)器安裝，建議新開一個機器，不要直接在生產(chǎn)環(huán)境里安裝，因為挺吃硬件資源的，會容易出問題
以上就是docker搭建es集群實現(xiàn)過程詳解的詳細(xì)內(nèi)容，更多關(guān)于docker搭建es集群的資料請關(guān)注其它相關(guān)文章！
