## 运维痛处 k8s集群在线上跑了一段时间，可是我发现我对集群内部的变化没有办法把控的很清楚，好比某个pod被从新调度了、某个node节点上的imagegc失败了、某个hpa被触发了等等，而这些都是能够经过events拿到的，可是events并非永久存储的，它包含了集群各类资源的状态变化，因此咱们能够经过收集分析events来了解整个集群内部的变化。 ## kubernetes-event-exporter产品 [该存储库](https://github.com/opsgenie/kubernetes-event-exporter) 该工具允许将经常错过的 Kubernetes 事件导出到各种输出，以便将它们用于可观察性或警报目的。你不会相信你错过了资源的变化。 ## 安装 kubernetes-event-exporter > 提请创建好 `kube-mon` 命名空间 1. 相关的rbac权限 ```yaml --- apiVersion: v1 kind: ServiceAccount metadata: namespace: kube-mon name: event-exporter --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: event-exporter roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: view subjects: - kind: ServiceAccount namespace: kube-mon name: event-exporter ``` 2. events配置文件 ```yaml apiVersion: v1 kind: ConfigMap metadata: name: event-exporter-cfg namespace: kube-mon data: config.yaml: | logLevel: error logFormat: json route: routes: - match: - receiver: "dump" receivers: - name: "dump" ## kafka配置 kafka: clientId: "kubernetes" topic: "kube-events" brokers: - "192.168.31.235:9092" ## elasticsearch配置 ## 必须要有index、indexFormat参数 #elasticsearch: # hosts: # - http://192.168.31.235:9200 # index: kube-events # indexFormat: "kube-events-{2006-01-02}" # useEventID: true ## 直接输出到容器日志 # stdout: {} ``` 更多的配置内容请查看 [官方文档](https://github.com/opsgenie/kubernetes-event-exporter#configuration) 3. events资源清单 ```yaml apiVersion: apps/v1 kind: Deployment metadata: name: event-exporter namespace: kube-mon spec: replicas: 1 template: metadata: labels: app: event-exporter version: v1 spec: serviceAccountName: event-exporter containers: - name: event-exporter image: ghcr.io/opsgenie/kubernetes-event-exporter:v0.11 imagePullPolicy: IfNotPresent args: - -conf=/data/config.yaml volumeMounts: - mountPath: /data name: cfg volumes: - name: cfg configMap: name: event-exporter-cfg selector: matchLabels: app: event-exporter version: v1 ``` 4. 创建以上的清单 ```shell $ kubectl apply -f 00-roles.yaml serviceaccount/event-exporter created clusterrolebinding.rbac.authorization.k8s.io/event-exporter created $ kubectl apply -f 01-config.yaml configmap/event-exporter-cfg created $ kubectl apply -f 02-deployment.yaml deployment.apps/event-exporter created ``` ## 验证 ```shell $ kubectl -n kube-mon get deploy event-exporter NAME READY UP-TO-DATE AVAILABLE AGE event-exporter 1/1 1 1 85s $ kafka-get-offsets.sh --bootstrap-server 192.168.31.235:9092 --topic kube-events kube-events:0:43 ``` > 手动删除一个pod。查看topic为 `kube-events` 的返回结果，返回结果为 `kube-events:0:43`，分别是topic名称、partitionID、partitionID的偏移量。查看任意partitionID的偏移量是否为0。 > - 0 --> 代表没有上传到Kafka > - 非0 --> 代表有数据上传到Kafka，说明整个流程正常