目錄
- 前言
- 什么是es
- 使用docker搭建es集群
前言
該系列默認開啟Nacos 服務,還不會搭建的小伙伴可以參考往期文章~
本節重點是給大家介紹利用docker來搭建Es集群,廢話不多說直接開整吧~
什么是es
同樣的，在學習之前，先了解一下這玩意到底是個啥?
es這個名詞或許大家都聽過，它的全稱是Elasticsearch,它是一個分布式文檔儲存中間件，它不會將信息儲存為列數據行，而是儲存已序列化為 JSON 文檔的復雜數據結構。當你在一個集群中有多個節點時，儲存的文檔分布在整個集群里面，并且立刻可以從任意節點去訪問。
當文檔被儲存時，它將建立索引并且近實時（1s）被搜索。 Elasticsearch 使用一種被稱為倒排索引的數據結構，該結構支持快速全文搜索。在倒排索引里列出了所有文檔中出現的每一個唯一單詞并分別標識了每個單詞在哪一個文檔中。有時候面試官會問，es為什么這么快?這也是一個小的知識點。
通過上面簡單的介紹，我們大體可以知道，它是用來做數據檢索的，而且速度特別快。
不知道小伙伴們有沒有遇到過這樣一個問題，比方說我們在用sql查商品庫表的時候,想要通過某個關鍵詞來匹配相應的商品，當數據量很小的時候ok,但是隨著商品數據的不斷導入，后期的數據量越來越大，而且都是關聯著好幾張表，這時候我們用sql去查詢我們想要的數據的時候，會顯得特別吃力，這種是相當危險的操作，因為可能會把整張表鎖死，導致我們的系統出現故障，如果其它系統也使用這個庫，那么也會受到影響。所以這時候，我們就需要借助es這種中間件來幫我們處理這種需求，系統的性能也會有顯著的提升，當然，維護上也會增加一些難度，當然也不是啥都上es的。其實我們也可以使用其它的比如mongo，如何選取，取決于系統架構和實際的業務場景。
使用docker搭建es集群
為了大家快速的體驗到es，這里推薦大家使用docker來搭建，因為它比較方便。但是生產中，如果你對docker不是很熟悉，維護會稍微有點麻煩，那么建議你還是到官網去下載具體的安裝包，本節默認大家都已經安裝好了docker。如果你還不知道docker是啥也沒關系，這個后邊我會專門給大家講講，本節跟著我敲就可以了。
docker的安裝非常簡單，官網都有具體的平臺的安裝包,win和mac都有，無腦安裝就好了。win11安裝可能會遇到wsl的問題，需要開啟linux子系統，如果啟動錯誤，直接百度錯誤就好了，已經有人踩過坑了。
下面，我們進入正題,首先啟動好docker，本節帶大家安裝的是7.6.2的版本，這個版本相對好一些，控制臺的功能也都很完善。
執行已下命令獲取官方鏡像， 打開cmd/mac終端：
# es鏡像 docker pull docker.elastic.co/elasticsearch/elasticsearch:7.6.2 # kibana鏡像 docker pull docker.elastic.co/kibana/kibana:7.6.2
kibana它是一個可視化的平臺，我們查看數據就是通過它，es只是用作數據引擎，市面上也有一些第三方的工具，但是官方的這個已經非常完善了，界面也很美觀。
緊接著，進入指定安裝目錄,比方說當前目錄叫es,終端進入這個目錄后執行一下命令:
# kibana數據掛載的目錄 mkdir data/kibana # 三個節點數據掛載的目錄 mkdir data/node1 mkdir data/node2 mkdir data/node3
這一步主要是創建相關的目錄，因為后邊docker的數據卷會映射到該目錄，這樣做的目的是防止容器意外銷毀后的數據丟失。這里為什么是三個節點,因為es集群至少需要三個節點，這是跟它的內部機制有關，為了防止腦裂現象，這里就不給大家過多展開了
接下來進入data/kibana目錄,新建kibana.yml,這個文件是它的配置文件，后邊我們會把它映射到docker容器內部
# ## ** THIS IS AN AUTO-GENERATED FILE ** ## # # # Default Kibana configuration for docker target server.name: kibana server.host: "0" elasticsearch.hosts: [ "http://es01:9200","http://es02:9200","http://es03:9200" ] xpack.monitoring.ui.container.elasticsearch.enabled: true i18n.locale: zh-CN
elasticsearch.hosts指的是三個es節點,會和這些節點進行通信
進入node1，同樣新建配置文件elasticsearch.yml
# ======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: es-cluster # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: es01 # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # #path.data: /path/to/data # # Path to log files: # #path.logs: /path/to/logs # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # #bootstrap.memory_lock: true # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 0.0.0.0 # # Set a custom port for HTTP: # http.port: 9200 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when this node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # discovery.seed_hosts: ["es01","es02","es03"] # # Bootstrap the cluster using an initial set of master-eligible nodes: # cluster.initial_master_nodes: ["es01","es02","es03"] # bootstrap.memory_lock: true # # For more information, consult the discovery and cluster formation module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # #gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true http.cors.enabled: true http.cors.allow-origin: '*' http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type node.master: true
我們把node1作為主節點,也就是老大，node.master: true可以配置。為了使它支持中文分詞,我們給它安裝一下插件, 到倉庫下載指定版本的插件https://github.com/medcl/elasticsearch-analysis-ik/releases,然后我們解壓到node1根目錄，然后重新命名為ik目錄,然后再新建一個Dockerfile用來重構“`es““鏡像，沒錯，后邊我們就使用我們重構好的鏡像，這樣就自動安裝好了插件
Dockerfile文件內容
FROM docker.elastic.co/elasticsearch/elasticsearch:7.6.2 COPY --chown=elasticsearch:elasticsearch elasticsearch.yml /usr/share/elasticsearch/config/ ADD ik /usr/share/elasticsearch/plugins/ik ADD ik/config /data/erms/es/node1/ik/config
下面我們進入node2目錄，這個目錄只需要放配置文件就好了
# ======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: es-cluster # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: es02 # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # #path.data: /path/to/data # # Path to log files: # #path.logs: /path/to/logs # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # #bootstrap.memory_lock: true # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 0.0.0.0 # # Set a custom port for HTTP: # http.port: 9200 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when this node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # discovery.seed_hosts: ["es01","es02","es03"] # # Bootstrap the cluster using an initial set of master-eligible nodes: # cluster.initial_master_nodes: ["es01","es02","es03"] # bootstrap.memory_lock: true # # For more information, consult the discovery and cluster formation module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # #gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true http.cors.enabled: true http.cors.allow-origin: '*' http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type node.data: true
這里我們指定為數據節點node.data: true用來做副本
同樣的node3
# ======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: es-cluster # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: es03 # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # #path.data: /path/to/data # # Path to log files: # #path.logs: /path/to/logs # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # #bootstrap.memory_lock: true # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 0.0.0.0 # # Set a custom port for HTTP: # http.port: 9200 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when this node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # discovery.seed_hosts: ["es01","es02","es03"] # # Bootstrap the cluster using an initial set of master-eligible nodes: # cluster.initial_master_nodes: ["es01","es02","es03"] # bootstrap.memory_lock: true # # For more information, consult the discovery and cluster formation module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # #gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true http.cors.enabled: true http.cors.allow-origin: '*' http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type node.data: true
然后我們回到根目錄(es),新建一個docker-compose.yaml,我們使用docker-compose來編排我們的容器，默認安裝好docker desktop就自動給我們安裝好了docker-compose
version: '3'
services:
es01:
image: ${image}
container_name: es01
environment:
- discovery.seed_hosts=es02,es03
- cluster.initial_master_nodes=es01,es02,es03
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./data/node1/data:/usr/share/elasticsearch/data
- ./data/node1/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
- ./data/node1/plugins:/usr/share/elasticsearch/plugins
ports:
- 9200:9200
networks:
- elastic
es02:
image: ${image}
container_name: es02
environment:
- discovery.seed_hosts=es01,es03
- cluster.initial_master_nodes=es01,es02,es03
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./data/node2/data:/usr/share/elasticsearch/data
- ./data/node2/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
- ./data/node2/plugins:/usr/share/elasticsearch/plugins
ports:
- 9201:9201
networks:
- elastic
es03:
image: ${image}
container_name: es03
environment:
- discovery.seed_hosts=es01,es02
- cluster.initial_master_nodes=es01,es02,es03
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./data/node3/data:/usr/share/elasticsearch/data
- ./data/node3/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
- ./data/node3/plugins:/usr/share/elasticsearch/plugins
ports:
- 9202:9202
networks:
- elastic
kibana:
image: ${image_kibana}
container_name: kibana
depends_on:
- es01
environment:
ELASTICSEARCH_URL: http://es01:9200
ELASTICSEARCH_HOSTS: http://es01:9200
volumes:
- ./data/kibana/kibana.yml:/usr/share/kibana/config/kibana.yml
networks:
- elastic
ports:
- 5601:5601
networks:
elastic:
driver: bridge
這個文件有點長，不懂沒關系，跟著配就完了。${image}是一個占位符，所以我們還需要指定環境變量,然后新建一個.env
image=m/es image_kibana=docker.elastic.co/kibana/kibana:7.6.2
m/es這個是我們重構后的鏡像名稱，下面我們就來重構鏡像
進入data/node1執行
docker build -t m/es .
執行完成后，到根目錄執行啟動命令:
docker-compose up -d
如果你想看實時日志，把-d去掉，這個是后臺運行，初次啟動，可能要花費一些時間。
啟動成功后，我們可以訪問一些es1的節點localhost:9200,可以查看節點的信息,如果顯示正常，說明已經搭建成功了，下面我們直接進入kibana控制臺
http://localhost:5601/,初次進入會讓你設置控制臺的密碼
我們進入控制臺,執行一下，有如下輸出，至此我們就搭建成功了
如果你想卸載它們，執行docker-compose down就可以了，畢竟這幾個家伙特別的吃資源。這里提醒一下大家，如果想嘗試到服務器安裝，建議新開一個機器，不要直接在生產環境里安裝，因為挺吃硬件資源的，會容易出問題
以上就是docker搭建es集群實現過程詳解的詳細內容，更多關于docker搭建es集群的資料請關注其它相關文章！
