elasticsearch和mysql的對比

https://blog.csdn.net/qq_21383435/article/details/79323383

Mapping ~ Schema

master: 負(fù)責(zé)在集群范圍內(nèi)創(chuàng)建/刪除索引,將分片分配給這些節(jié)點(diǎn). data: 用來保存數(shù)據(jù)和倒排索引,node.data=ture client: 將節(jié)點(diǎn)配置為客戶端節(jié)點(diǎn)，并充當(dāng)負(fù)載平衡器，將傳入的請求路由到集群中的不同節(jié)點(diǎn)。node.master和node.data設(shè)置為false

https://blog.csdn.net/sdksdk0/article/details/78469190

Index templates

索引可使用預(yù)定義的模板進(jìn)行創(chuàng)建,這個(gè)模板稱作Index templates。模板設(shè)置包括settings和mappings，通過模式匹配的方式使得多個(gè)索引重用一個(gè)模板，例如：

curl -XPUT localhost:9200/_template/template_1 -d ' {"template" : "te*","settings" : {"number_of_shards" : 1},"mappings" : {"type1" : {"_source" : {"enabled" : false }}} }

elk設(shè)計(jì)架構(gòu)

參考
k8sgithub上 https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/fluentd-elasticsearch

Elasticsearch最佳實(shí)踐建議將這些節(jié)點(diǎn)分成三個(gè)角色：Master 節(jié)點(diǎn) - 僅用于集群管理，沒有數(shù)據(jù)，沒有HTTP API Client 節(jié)點(diǎn) - 用于客戶端使用，無數(shù)據(jù)， 使用HTTP API Data 節(jié)點(diǎn) - 旨在存儲(chǔ)和索引數(shù)據(jù)， 沒有HTTP API

分為 管理區(qū) 存儲(chǔ)區(qū) 讀寫區(qū)

定制鏡像

- es 0. 我把鏡像推到了dockerhub: lanny/quay.io_pires_docker-elasticsearch-kubernetes:5.6.0 1. 鏡像里用到的yaml: https://github.com/lannyMa/docker-elasticsearch-kubernetes 2. 可以安裝自定義插件后重新build- kibana 0. 用到這個(gè)人的這個(gè)鏡像 cfontes/kibana-xpack-less:5.5.0 https://hub.docker.com/r/cfontes/kibana-xpack-less/這個(gè)人的git(包含了elk的k8s yaml): https://github.com/cfontes/kubernetes-elasticsearch-cluster- k8s的elk集群的yaml https://github.com/pires/kubernetes-elasticsearch-cluster 鏡像到這個(gè)地址: https://github.com/lannyMa/kubernetes-elasticsearch-cluster內(nèi)含elk 普羅 harbor的yaml,比較實(shí)用 https://github.com/cnych/k8s-repo

集群yaml里搜HTTP_ENABLE,統(tǒng)一打開head訪問權(quán)限

es-master.yaml

- name: HTTP_ENABLEvalue: "true"- name: HTTP_CORS_ALLOW_ORIGINvalue: "*"

創(chuàng)建集群

創(chuàng)建pv

$ cat es-data_claim.yaml kind: PersistentVolumeClaim apiVersion: v1 metadata:name: opspvcnamespace: kube-opsannotations:volume.beta.kubernetes.io/storage-class: "managed-nfs-storage" spec:accessModes:- ReadWriteManyresources:requests:storage: 1Gi

先創(chuàng)建svc類型

后依次創(chuàng)建 data master client kibana

訪問es api

訪問kibana

打開kibana,插入示例數(shù)據(jù)

PUT _bulk {"index":{"_index":"lagou","_type":"job","_id":"1"}} {"title":"python分布式爬蟲開發(fā)","salary_min":15000,"city":"深圳","company":{"name":"騰訊","company_addr":"深圳市軟件園"},"publish_date":"2017-11-11","comments":15} {"index":{"_index":"lagou","_type":"job","_id":"2"}} {"title":"django開發(fā)","salary_min":15000,"city":"上海","company":{"name":"阿里","company_addr":"廣州市軟件園"},"publish_date":"2017-11-12","comments":20}

打開head查看數(shù)據(jù)

docker run -d -v /etc/localtime:/etc/localtime --restart=always -p 9100:9100 mobz/elasticsearch-head:5

查看數(shù)據(jù)

todo:Curator(館長) 管理索引的工具

Curator的主要用途:

• 獲取索引列表

• 過濾索引,刪除/快照

• 可以對索引做這些操作

name用途

Alias	別號
Allocation	分配
Close	關(guān)
Cluster Routing	群集路由
Create Index	創(chuàng)建索引
Delete Indices	刪除索引
Delete Snapshots	刪除快照
forceMerge	forceMerge
Index Settings	索引設(shè)置
Open	打開
Reindex	重新編制
Replicas	副本
Restore	恢復(fù)
Rollover	滾下
Shrink	收縮
Snapshot	快照

elk整體鏈條架構(gòu)

es版本兼容

"minimum_wire_compatibility_version" : "5.6.0","minimum_index_compatibility_version" : "5.0.0" curl http://10.100.32.137:9200{"name" : "es-data-5c5969967-wb2b8","cluster_name" : "myesdb","cluster_uuid" : "qSps-b9dRI2ngGHBguJ44Q","version" : {"number" : "6.3.0","build_flavor" : "default","build_type" : "tar","build_hash" : "424e937","build_date" : "2018-06-11T23:38:03.357887Z","build_snapshot" : false,"lucene_version" : "7.3.1","minimum_wire_compatibility_version" : "5.6.0","minimum_index_compatibility_version" : "5.0.0"},"tagline" : "You Know, for Search" }

轉(zhuǎn)載于:https://www.cnblogs.com/iiiiher/p/8266930.html

總結(jié)

以上是生活随笔為你收集整理的[k8s]elk架构设计-k8s集群里搭建的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。