基于EFK构建日志收集平台

一、知识点:

Elasticsearch、Fluentd和Kibana(简称EFK)是一个强大的日志管理解决方案。在Kubernetes(K8s)环境中部署EFK集群,可以帮助您轻松管理和分析日志。

ElasticSearch:分布式存储检索引擎,用来搜索、存储日志

Fluentd:日志采集

Kibana:读取es中数据进行可视化web界面展示

二、实验环境:

一台管理节点master(180),两台工作节点node1(183)、node2(184),一台存储服务器NFS(185)

需要在配置完单节点集群的环境下进行操作

三、创建NFS存储服务器

3.1、给新系统的主机在原有一块硬盘的基础上再添加4块容量为100G的SCSI类型硬盘

3.2查看磁盘分区情况(主要查看磁盘是否sda已经分区,反之需要重启):fdisk -l

3.3 给添加的4块磁盘进行分区

四、nfs机器上安装raid软件

[root@nfs ~]# dnf -y install mdadm

五、nfs机器上将sdb1-sde1四块硬盘创建raid5,其中三块硬盘做raid5,一块硬盘做热备

[root@nfs ~]#mdadm -C /dev/md5 -ayes -l5 -n3 -x1 /dev/sd[b-e]1   将sdb1-sde1四块硬盘创建raid5,其中三块硬盘做raid5,一块硬盘做热备,(/dev/md5:raid5创建好的设备名称)

六、nfs机器上查看raid信息

[root@nfs ~]#cat /proc/mdstat                               查看raid信息(最低三块,最多坏两块) 

七、nfs机器上查看raid5磁盘的详细信息

[root@nfs ~]# mdadm -D /dev/md5
/dev/md5:
           Version : 1.2
     Creation Time : Mon Jul  7 23:03:14 2025
        Raid Level : raid5【raid的级别:raid5】
        Array Size : 209579008 (199.87 GiB 214.61 GB)
     Used Dev Size : 104789504 (99.94 GiB 107.30 GB)
      Raid Devices : 3【raid设备:3块磁盘】
     Total Devices : 4【总共设备:4块磁盘】
       Persistence : Superblock is persistent

八、nfs机器上将四块磁盘的信息追加到raid的配置文件中

[root@nfs ~]# echo 'DEVICE /dev/sd[b-e]1' >> /etc/mdadm.conf 将四块磁盘的信息追加到raid的配置文件中

九、nfs机器上将当前系统中所有raid设备的配置信息以动态方式追加到配置文件中

[root@nfs ~]# mdadm -Ds  >> /etc/mdadm.conf 将当前系统中所有raid设备的配置信息以动态方式追加到配置文件中(-D:raid的详细信息)(-s:扫描系统并输出所有已激活的raid设备的配置信息,以适合放入配置文件的格式呈现)

十、查看raid的配置文件

[root@nfs ~]# cat /etc/mdadm.conf 查看raid的配置文件

十一、格式化raid5的磁盘

[root@nfs ~]# mkfs.xfs /dev/md5 格式化raid5的磁盘

十二、在nfs主机上创建data目录、获取块设备的详细信息

[root@nfs ~]# mkdir /data 创建data目录
[root@nfs ~]# blkid 获取块设备的详细信息:如文件类型UUID等

十三、nfs上编辑开机挂载配置文件

[root@nfs ~]# vim /etc/fstab 编辑开机挂载配置文件
raid5的磁盘挂载到data目录下,文件类型为xfs,默认参数,不备份,不检查磁盘
[root@nfs ~]# systemctl daemon-reload 重新加载系统进程
[root@nfs ~]# mount -a 重新加载挂载项
[root@nfs ~]# df -hT 查看磁盘容量

十四、在nfs主机的在data目录下创建v1/v2目录

[root@nfs ~]# mkdir /data/v1
[root@nfs ~]# mkdir /data/v2

十五、在nfs主机安装nfs

[root@nfs ~]# dnf -y install nfs-utils

十六、编辑nfs配置文件

[root@nfs ~]#  vim /etc/exports
/data/v1 *(rw,no_root_squash)【将data下的v1目录共享给任何IP,可写可读,以匿名用户执行】
/data/v2 *(rw,no_root_squash)【将data下的v2目录共享给任何IP,可写可读,以匿名用户执行】
[root@nfs ~]# systemctl enable --now nfs-server 启动nfs并设置开机启动
[root@nfs ~]# exportfs -avr 将nfs发布出去
[root@nfs ~]# showmount -e 查看nfs挂载项

nfs创建完成

十七、node1/2 master1所有客户端安装nfs

[root@node1/2 master1 ~]# dnf -y install nfs-utils 所有客户端安装
nfs

十八、 node1/2 master1清除缓存

[root@node1/2 master1 ~ ~]# dnf clean all	            清除缓存	 			
37 files removed

十九、启动nfs,并设置开机启动

[root@node1/2 master1 ~]# systemctl enable --now nfs-server		启动nfs,并设置开机启动
[root@node1/2 master1 ~]# showmount -e 192.168.7.185	   查看192.168.7.185 nfs的挂载项
Export list for 192.168.7.185:
/data/v2 *
/data/v1 *

二十、node1/2 master1所有客户端都上传nfs-subdir-external-provisioner.tar压缩包

[root@node1/2 master1 ~]# ls		
anaconda-ks.cfg
calico.tar.gz
calico.yaml
kubeadm.yaml
nfs-subdir-external-provisioner.tar

二十一、所有客户端都导入并运行nfs-subdir-external-provisioner.tar镜像

[root@node1/2 master1 ~]# ctr -n k8s.io images import nfs-subdir-external-provisioner.tar  (用于配合存储类动态生成PV)所有客户端都导入并运行nfs-subdir-external-provisioner.tar镜像
unpacking k8s.gcr.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2 (sha256:925a999977a8fe14dd85d14aadd741144c7f8acfb2fe79f41c353bf26a0c58b7)...done
[root@master1 ~]#

二十二、在master1节点上创建efk文件夹

[root@master1 ~]# mkdir efk
[root@master1 ~]#
[root@master1 ~]# cd efk/
[root@master1 efk]#

二十三、编辑serviceaccount配置文件

[root@master1 efk]# vim serviceaccount.yaml	
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-provisioner	
[root@master1 efk]#
[root@master1 efk]# kubectl apply -f serviceaccount.yaml
serviceaccount/nfs-provisioner created  应用serviceaccount.yaml配置文件
[root@master1 efk]# kubectl create clusterrolebinding nfs-provisioner^Bclusterrolebinding --clusterrole=cluster-admin --serviceaccount=default:nfs-provisioner  #创建一个名为 nfs-provisioner-clusterrolebinding 的 ClusterRoleBinding 资源
clusterrolebinding.rbac.authorization.k8s.io/nfs-provisioner^Bclusterrolebinding created
[root@master1 efk]#

二十四、创建nfs无状态部署的配置文件

[root@master1 efk]# vim nfs-deployment.yaml		创建nfs无状态部署的配置文件
kind: Deployment		
apiVersion: apps/v1
metadata:
  name: nfs-provisioner
spec:
  selector:
    matchLabels:
       app: nfs-provisioner
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: nfs-provisioner
    spec:
      serviceAccount: nfs-provisioner
      containers:
        - name: nfs-provisioner
          image: registry.cn-beijing.aliyuncs.com/mydlq/nfs-subdir-external-provisioner:v4.0.0
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: example.com/nfs
            - name: NFS_SERVER
              value: 192.168.7.185
            - name: NFS_PATH
              value: /data/v1
      volumes:
        - name: nfs-client-root
          nfs:
            server: 192.168.7.185
            path: /data/v1
[root@master1 efk]# kubectl apply -f nfs-deployment.yaml		应用nfs无状态部署配置文件
deployment.apps/nfs-provisioner created

二十五、master1查看pod

[root@master1 efk]# kubectl get pods		查看pod
NAME                              READY   STATUS              RESTARTS   AGE
nfs-provisioner-69d569cc5-b5jf9   0/1     ContainerCreating 容器正在创建  0          10s
[root@master1 efk]# kubectl get pods		查看pod
NAME                              READY   STATUS    RESTARTS   AGE
nfs-provisioner-69d569cc5-tt9ns   1/1     Running   0          10s
[root@master1 efk]#

二十六、创建nfs

[root@master1 efk]# vim nfs-storageclass.yaml		创建nfs
kind: StorageClass			类型存储类
apiVersion: storage.k8s.io/v1		api版本存储型号V1
metadata:				元数据名称nfs
  name: nfs		
provisioner: example.com/nfs	提供者:example.com/nfs
[root@master1 efk]# kubectl apply -f nfs-storageclass.yaml		应用nfs存储类的配置文件
storageclass.storage.k8s.io/nfs created
[root@master1 efk]# kubectl get storageclass				查看存储类
NAME   PROVISIONER       RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE                                                           
nfs    example.com/nfs   Delete          Immediate           false     23s                                                                                             

二十七、编辑pvc-sc的配置文件

[root@master1 efk]# vim pvc-sc.yaml		编辑pvc-sc的配置文件
kind: PersistentVolumeClaim		类别:pvc 持久卷消费
apiVersion: v1					api版本v1
metadata:					元数据名称:pvc-sc
  name: pvc-sc				
spec:						定义资源对象
  storageClassName: nfs		存储类的名称:nfs
  accessModes:				访问的模式
    - ReadWriteMany			多重读写
  resources:					资源
    requests:					要求:存储容量50G
      storage: 50Gi			50Gi	

二十八、应用PVC-sc的配置文件

[root@master1 efk]# kubectl apply -f pvc-sc.yaml			应用PVC-sc的配置文件 
persistentvolumeclaim/pvc-sc created

二十九、查看pvc

[root@master1 efk]# kubectl get pvc					查看pvc
NAME     STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
pvc-sc   Bound 绑定   pvc-c74653fc-2575-4e69-acd5-1a891d72e8d8   50Gi       RWX            nfs            <unset>                 8s
[root@master1 efk]#

三十、创建nginx的配置文件

[root@master1 efk]# vim nginx.yaml		创建nginx的配置文件(为测试所有节点连接到nfs用的)
apiVersion: v1
kind: Service
metadata:
  labels:
    app: nginx
  name: nginx
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx:1.21.6
        name: nginx
        volumeMounts:
          - name: html-files
            mountPath: "/usr/share/nginx/html"
      volumes:
        - name: html-files
          persistentVolumeClaim:
            claimName: pvc-sc
[root@master1 efk]#

三十一、node1、node2上传nginx-1.21.6

三十二、node1、node2导入并运行nginx-1.21.6.tar镜像

[root@node1 ~]# ctr -n k8s.io images import nginx-1.21.6.tar                    
unpacking docker.io/library/nginx:1.21.6 (sha256:94b808e393739b5363decf631a746d0241083d40eb05f07200a6d1c0c16f54b8)...done
[root@node1 ~]#
[root@node2 ~]# ctr -n k8s.io images import nginx-1.21.6.tar
unpacking docker.io/library/nginx:1.21.6 (sha256:94b808e393739b5363decf631a746d0241083d40eb05f07200a6d1c0c16f54b8)...done
[root@node2 ~]#

三十三、在master1上运行nginx.yaml配置文件

[root@master1 efk]# kubectl apply -f nginx.yaml
service/nginx created
deployment.apps/nginx created
[root@master1 efk]#

三十四、查看pod信息、查看nginx服务为集群IP、访问nginx的集群IP

[root@master1 efk]# kubectl get pod     查看pod信息
NAME                              READY   STATUS    RESTARTS   AGE
nfs-provisioner-69d569cc5-tt9ns   1/1     Running   0          34m
nginx-8444fcb447-fcg5r            1/1     Running   0          9s
[root@master1 efk]# kubectl get svc		查看nginx服务为集群IP
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP   20d
nginx        ClusterIP   10.101.12.184   <none>        80/TCP    29s
[root@master1 efk]# kubectl exec -it nginx-8444fcb447-fcg5r -- bash	进入到nginx的容器里
root@nginx-8444fcb447-fcg5r:/# echo 'StorageClass used' > /usr/share/nginx/html/index.html	创建一个nginx的测试页面
root@nginx-8444fcb447-fcg5r:/# exit
exit
[root@master1 efk]# curl 10.101.12.184		访问nginx的集群IP
StorageClass used
[root@master1 efk]#

三十五、三台都做快照,快照名称:pvc-sc  后开机

三十六、在nfs服务器上查看测试页面,所有pod内容都放在nfs服务器上

[root@nfs ~]# cat /data/v1/default-pvc-sc-pvc-c74653fc-2575-4e69-acd5-1a891d72e8d8/index.html		在nfs服务器上查看测试页面,所有pod内容都放在nfs服务器上
StorageClass used
[root@nfs ~]# 

三十七、在node1、node2上传elasticsearch-7-12-1.tar.gz、fluentd-containerd.tar.gz、kibana-7-12-1.tar.gz压缩包(elasticsearch 和kibana版本需一致,elasticsearch-7-12-1和 kibana-7-12-1)

[root@node1、node2 ~]# ctr -n k8s.io images import elasticsearch-7-12-1.tar.gz	在所有工作节点导入elasticsearch-7-12-1.tar.gz的镜像
Unpacking docker.io/library/elasticsearch:7.12.1 (sha256:a3c1c0ad93c9c48c8e60cf94c5e600279f568cdd2d5307c0297e570eb0cbe783)...done
[root@node1、node2  ~]# ctr -n k8s.io images import fluentd-containerd.tar.gz	在所有工作节点导入fluentd-containerd.tar.gz的镜像
unpackingdocker.io/fluent/fluentd-kubernetes-daemonset:v1.16-debian-elasticsearch7-1(sha256:ab85c3288e71416963b4                                         3d5fb713b77821ce11c025eb2834cfada5c7e5754918)...done
[root@node1、node2  ~]# ctr -n k8s.io images import kibana-7-12-1.tar.gz		在所有工作节点导入kibana-7-12-1.tar.gz的镜像
unpacking docker.io/library/kibana:7.12.1 (sha256:163a637ca1ab65838f9aba81a1d32e1058e9d32a28cdf5dd253ad3137b0c04fb)...done
[root@node1 、node2 ~]#  ctr -n k8s.io images import fluentd-containerd.tar.gz
unpacking docker.io/fluent/fluentd-kubernetes-daemonset:v1.16-debian-elasticsearch7-1 (sha256:ab85c3288e71416963b43d5fb713b77821ce11c025eb2834cfada5c7e5754918)...done
[root@node1、node2  ~]#

三十八、在master1上上传fluentd-containerd.tar.gz的镜像

[root@master1 efk~]#  ctr -n k8s.io images import fluentd-containerd.tar.gz		在所有工作节点导入fluentd-containerd.tar.gz的镜像
unpacking docker.io/fluent/fluentd-kubernetes-daemonset:v1.16-debian-elasticsearch7-1 (sha256:ab85c3288e71416963b43d5fb713b77821ce11c025eb2834cfada5c7e5754918)...done

三十九、master1上创建日志配置文件

[root@master1 efk~]# vim kube-logging.yaml		创建日志配置文件
kind: Namespace		类型为名称空间		为了分类
apiVersion: v1			api版本为V1
metadata:			源数据的名称为:kube-logging 
  name: kube-logging
[root@master1 efk]# kubectl apply -f kube-logging.yaml		应用日志配置文件
namespace/kube-logging created

四十、查看名称空间

[root@master1 efk]# kubectl get ns		查看名称空间
NAME              STATUS   AGE
default           Active   20d
kube-logging      Active   6m32s	看这个
kube-node-lease   Active   20d
kube-public       Active   20d
kube-system       Active   20d
[root@master1 efk]# 

四十一、创建elasticsearch的配置文件

[root@master1 efk]# vim elasticsearch_svc.yaml	创建elasticsearch的配置文件
kind: Service		类型服务
apiVersion: v1		api版本为V1
metadata:		源数据的名称为:elasticsearch
  name: elasticsearch	名称空间:elasticsearch
  namespace: kube-logging	
  labels:			卷标:kube-logging
    app: elasticsearch
spec:			定义资源对象
  selector:		调度器
    app: elasticsearch	
  clusterIP: None	集群IP为无
  ports:			端口组
    - port: 9200	端口9200
      name: rest	名称:rest 外部端口
    - port: 9300	端口9300
      name: inter-node	内部节点端口
[root@master1 efk]# kubectl apply -f elasticsearch_svc.yaml 应用elasticsearch的配置文件
service/elasticsearch created
[root@master1 efk]# kubectl get services -n kube-logging	查看日志命名空间的服务
NAME            TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)             AGE
elasticsearch   ClusterIP   None         <none>        9200/TCP,9300/TCP   12s
[root@master1 efk]# 

四十二、创建es elasticsearch类的配置文件

[root@master1 efk]# vim es_class.yaml		创建es elasticsearch类的配置文件
apiVersion: storage.k8s.io/v1		api版本V1
kind: StorageClass				类别存储类
metadata:					元数据名称:do-block-storage
  name: do-block-storage
provisioner: example.com/nfs	提供商:example.com/nfs,不一致连接nfs不上
~   
[root@master1 efk]# kubectl apply -f es_class.yaml 	应用es elasticsearch的配置文件
storageclass.storage.k8s.io/do-block-storage created

四十三、master1上传es elasticsearch的配置文件

四十四、编辑es elasticsearch的配置文件

[root@master1 efk]# vim elasticsearch-statefulset.yaml 	编辑es elasticsearch的配置文件
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: es-cluster
  namespace: kube-logging
spec:
  serviceName: elasticsearch
  replicas: 3
  selector:
    matchLabels:
      app: elasticsearch
  template:
    metadata:
      labels:
        app: elasticsearch
    spec:
      containers:
      - name: elasticsearch
        image: elasticsearch:7.12.1
        imagePullPolicy: IfNotPresent
        resources:
            limits:
              cpu: 1000m
            requests:
              cpu: 100m
        ports:
        - containerPort: 9200
          name: rest
          protocol: TCP
        - containerPort: 9300
          name: inter-node
          protocol: TCP
        volumeMounts:
        - name: data
          mountPath: /usr/share/elasticsearch/data
        env:
          - name: cluster.name
            value: k8s-logs
          - name: node.name
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: discovery.seed_hosts
            value: "es-cluster-0.elasticsearch,es-cluster-1.elasticsearch,es-cluster-2.elasticsearch"
          - name: cluster.initial_master_nodes
            value: "es-cluster-0,es-cluster-1,es-cluster-2"
          - name: ES_JAVA_OPTS
            value: "-Xms512m -Xmx512m"
      initContainers:
      - name: fix-permissions
        image: busybox
        imagePullPolicy: IfNotPresent
        command: ["sh", "-c", "chown -R 1000:1000 /usr/share/elasticsearch/data"]
        securityContext:
          privileged: true
        volumeMounts:
        - name: data
          mountPath: /usr/share/elasticsearch/data
      - name: increase-vm-max-map
        image: busybox
        imagePullPolicy: IfNotPresent
        command: ["sysctl", "-w", "vm.max_map_count=262144"]
        securityContext:
          privileged: true
      - name: increase-fd-ulimit
        image: busybox
        imagePullPolicy: IfNotPresent
        command: ["sh", "-c", "ulimit -n 65536"]
        securityContext:
          privileged: true
  volumeClaimTemplates:
  - metadata:
      name: data
      labels:
        app: elasticsearch
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: do-block-storage
      resources:
        requests:
          storage: 10Gi

四十五、master1应用es elasticsearch的配置文件

[root@master1 efk]# kubectl apply -f elasticsearch-statefulset.yaml 应用es elasticsearch的配置文件
statefulset.apps/es-cluster configured
[root@master1 efk]# kubectl get pods -n kube-logging		查看命名空间日志pod信息	
NAME           READY   STATUS    RESTARTS   AGE
es-cluster-0   1/1     Running   0          8m39s
es-cluster-1   1/1     Running   0          4m20s
es-cluster-2   1/1     Running   0          4m14s
[root@master1 efk]# 
[root@master1 ~]# kubectl get svc -n kube-logging		查看日志命名空间的服务
NAME            TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)             AGE
elasticsearch   ClusterIP   None         <none>        9200/TCP,9300/TCP   60m

四十六、node1、node2上上传Busybox压缩包,并导入镜像

[root@node1 \node2~]# ls
anaconda-ks.cfg
busybox-1-28.tar.gz
busybox.tar.gz
calico.tar.gz
elasticsearch-7-12-1.tar.gz
fluentd-containerd.tar.gz
kibana-7-12-1.tar.gz
nfs-subdir-external-provisioner.tar
nginx-1.21.6.tar
[root@node1 \node2~]# ctr -n k8s.io images import busybox.tar.gz

四十七、master1上上传fluentd-containerd.tar.gz压缩包,并导入镜像

[root@master1 efk]# ls
elasticsearch-7-12-1.tar.gz     es_class.yaml        nfs-storageclass.yaml  serviceaccount.yaml fluentd-containerd.tar.gz 
elasticsearch-statefulset.yaml  kube-logging.yaml    nginx.yaml
elasticsearch_svc.yaml          nfs-deployment.yaml  pvc-sc.yaml
[root@master1 efk]# ctr -n k8s.io images import fluentd-containerd.tar.gz 
unpacking docker.io/fluent/fluentd-kubernetes-daemonset:v1.16-debian-elasticsearch7-1 (sha256:ab85c3288e71416963b43d5fb713b77821ce11c025eb2834cfada5c7e5754918)...done

四十八、运行日志命名空间pod的配置文件

[root@master1 efk]# kubectl get pods -n kube-logging.yaml 
No resources found in kube-logging.yaml namespace.

四十九、查看日志命名空间的pod信息

[root@master1 efk]# kubectl get pods -n kube-logging  查看日志命名空间的pod信息
NAME           READY   STATUS    RESTARTS      AGE
es-cluster-0   1/1     Running   1 (16m ago)   3h44m
es-cluster-1   1/1     Running   1 (16m ago)   3h39m
es-cluster-2   1/1     Running   1 (16m ago)   3h39m

四台关机做快照,快照名称:ES,后

先开nfs185,后开master1,node1,node2

五十、查看日志命名空间的服务

[root@master1 ~]# kubectl get svc -n kube-logging
NAME            TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)             AGE
elasticsearch   ClusterIP   None         <none>        9200/TCP,9300/TCP   54m
[root@master1 ~]# cd efk/

五十一、master1节点上传Kibana配置文件,并应用
kibana.yaml
的配置文件

[root@master1 efk]# ls
elasticsearch-7-12-1.tar.gz     es_class.yaml      nfs-deployment.yaml    pvc-sc.yaml
elasticsearch-statefulset.yaml  kibana.yaml        nfs-storageclass.yaml  serviceaccount.yaml
elasticsearch_svc.yaml          kube-logging.yaml  nginx.yaml
[root@master1 efk]# kubectl apply -f kibana.yaml 应用kibana.yaml的配置文件
service/kibana created
deployment.apps/kibana created

五十二、查看日志命名空间的服务

[root@master1 efk]# kubectl get pods -n kube-logging		查看日志命名空间的服务
NAME                      READY   STATUS    RESTARTS      AGE
es-cluster-0              1/1     Running   0             37m
es-cluster-1              1/1     Running   1 (19m ago)   32m
es-cluster-2              1/1     Running   0             32m
kibana-74694f4c7b-6hmpg   1/1     Running   0             17s
[root@master1 efk]#

五十三、查看日志命名空间的服务

[root@master1 efk]# kubectl get svc -n kube-logging                        查看日志命名空间的服务
NAME            TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)             AGE
elasticsearch   ClusterIP   None            <none>        9200/TCP,9300/TCP   71m
kibana   kibana集群IP   ClusterIP   10.110.26.154   <none>        5601/TCP            95s

五十四、编辑在日志命名空间下的kibana服务

[root@master1 efk]# kubectl edit svc kibana -n kube-logging		编辑在日志命名空间下的 kibana服务
type: NodePort		类型由集群IP改为为NodePort

五十五、master1查看日志命名空间的服务

[root@master1 efk]# kubectl get svc -n kube-logging	查看日志命名空间的服务
\NAME            TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)             AGE
elasticsearch   ClusterIP   None            <none>        9200/TCP,9300/TCP   74m
kibana          NodePort  (变成NodePort)  10.110.26.154   <none>        5601:32262/TCP      4m18s
[root@master1 efk]#

五十六、应用fluentd配置文件

[root@master1 ~]# cd efk/
[root@master1 efk]# kubectl apply -f kibana.yaml 	应用fluentd配置文件
service/kibana created
deployment.apps/kibana created

五十七、查看日志命名空间的pod信息、查看日志命名空间的服务

[root@master1 efk]# kubectl get pods -n kube-logging		查看日志命名空间的pod信息
NAME                      READY   STATUS    RESTARTS   AGE
es-cluster-0              1/1     Running   0          3m10s
es-cluster-1              1/1     Running   0          3m6s
es-cluster-2              1/1     Running   0          3m2s
kibana-74694f4c7b-vnswm   1/1     Running   0          31s
[root@master1 efk]# kubectl get svc -n kube-logging             查看日志命名空间的服务
NAME            TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
elasticsearch   ClusterIP   None             <none>        9200/TCP,9300/TCP   4h26m
kibana          ClusterIP   10.103.239.224   <none>        5601/TCP            78s

五十八、编辑日志命名空间的服务

[root@master1 efk]# kubectl edit svc kibana -n kube-logging		编辑日志命名空间的服务
service/kibana edited
[root@master1 efk]# kubectl get svc -n kube-logging              查看日志命名空间的服务
NAME            TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
elasticsearch   ClusterIP   None             <none>        9200/TCP,9300/TCP   4h28m
kibana          NodePort    10.103.239.224   <none>        5601:30113/TCP      3m28s
[root@master1 efk]# 

五十九、测试

六十、master1上上传fluent配置文件

查看fluent配置文件

[root@master1 efk]# cat fluentd.yaml 
apiVersion: v1
kind: ServiceAccount
metadata:
  name: fluentd
  namespace: kube-logging
  labels:
    app: fluentd
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: fluentd
  labels:
    app: fluentd
rules:
- apiGroups:
  - ""
  resources:
  - pods
  - namespaces
  verbs:
  - get
  - list
  - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: fluentd
roleRef:
  kind: ClusterRole
  name: fluentd
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
  name: fluentd
  namespace: kube-logging
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluentd
  namespace: kube-logging
  labels:
    app: fluentd
spec:
  selector:
    matchLabels:
      app: fluentd
  template:
    metadata:
      labels:
        app: fluentd
    spec:
      serviceAccount: fluentd
      serviceAccountName: fluentd
      tolerations:
      - key: node-role.kubernetes.io/control-plane
        effect: NoSchedule
      containers:
      - name: fluentd
        image: docker.io/fluent/fluentd-kubernetes-daemonset:v1.16-debian-elasticsearch7-1
        imagePullPolicy: IfNotPresent
        env:
          - name:  FLUENT_ELASTICSEARCH_HOST
            value: "elasticsearch.kube-logging.svc.cluster.local"
          - name:  FLUENT_ELASTICSEARCH_PORT
            value: "9200"
          - name: FLUENT_ELASTICSEARCH_SCHEME
            value: "http"
          - name: FLUENTD_SYSTEMD_CONF
            value: disable
          - name: FLUENT_CONTAINER_TAIL_PARSER_TYPE
            value: "cri"
          - name: FLUENT_CONTAINER_TAIL_PARSER_TIME_FORMAT
            value: "%Y-%m-%dT%H:%M:%S.%L%z"
        resources:
          limits:
            memory: 512Mi
          requests:
            cpu: 100m
            memory: 200Mi
        volumeMounts:
        - name: varlog
          mountPath: /var/log
        - name: containers
          mountPath: /var/log/containers
          readOnly: true
      terminationGracePeriodSeconds: 30
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: containers
        hostPath:
          path: /var/log/containers
[root@master1 efk]# kubectl apply -f fluentd.yaml 应用fluentd.yaml配置文件
serviceaccount/fluentd created
clusterrole.rbac.authorization.k8s.io/fluentd created
clusterrolebinding.rbac.authorization.k8s.io/fluentd created
daemonset.apps/fluentd created

六十一、查看日志命名空间的pod信息

[root@master1 efk]# kubectl get pods -n kube-logging   查看日志命名空间的pod信息
NAME                      READY   STATUS    RESTARTS   AGE
es-cluster-0              1/1     Running   0          64m
es-cluster-1              1/1     Running   0          64m
es-cluster-2              1/1     Running   0          64m
fluentd-j5qqr             1/1     Running   0          28s
fluentd-xtqpl             1/1     Running   0          28s
fluentd-zqtj6             1/1     Running   0          28s
kibana-74694f4c7b-vnswm   1/1     Running   0          61m
[root@master1 efk]# 

六十二、真实机上测试

暂无评论

发送评论 编辑评论


				
|´・ω・)ノ
ヾ(≧∇≦*)ゝ
(☆ω☆)
(╯‵□′)╯︵┴─┴
 ̄﹃ ̄
(/ω\)
∠( ᐛ 」∠)_
(๑•̀ㅁ•́ฅ)
→_→
୧(๑•̀⌄•́๑)૭
٩(ˊᗜˋ*)و
(ノ°ο°)ノ
(´இ皿இ`)
⌇●﹏●⌇
(ฅ´ω`ฅ)
(╯°A°)╯︵○○○
φ( ̄∇ ̄o)
ヾ(´・ ・`。)ノ"
( ง ᵒ̌皿ᵒ̌)ง⁼³₌₃
(ó﹏ò。)
Σ(っ °Д °;)っ
( ,,´・ω・)ノ"(´っω・`。)
╮(╯▽╰)╭
o(*////▽////*)q
>﹏<
( ๑´•ω•) "(ㆆᴗㆆ)
😂
😀
😅
😊
🙂
🙃
😌
😍
😘
😜
😝
😏
😒
🙄
😳
😡
😔
😫
😱
😭
💩
👻
🙌
🖕
👍
👫
👬
👭
🌚
🌝
🙈
💊
😶
🙏
🍦
🍉
😣
Source: github.com/k4yt3x/flowerhd
颜文字
Emoji
小恐龙
花!
上一篇
下一篇