[TOC] # 必要条件 - Kubernetes 必须配置为使用 CNI（请参阅 [网络插件要求](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/#network-plugin-requirements)） - Linux kernel >= 4.9.17，升级内核 [参考步骤](../../other/upgrade_kernel.md) - 卸载其他cni插件，请参阅 [卸载calico网络插件](../calico/uninstall.md) - 有关系统要求的更多详细信息，请参阅 [系统要求](https://docs.cilium.io/en/stable/operations/system_requirements/#admin-system-reqs) # 与kubernets兼容性 | cilium version | k8s Version | | :-: | :-: | | v1.11 | 1.16, 1.17, 1.18, 1.19, 1.20, 1.21, 1.22, 1.23 | | v1.12 | 1.16, 1.17, 1.18, 1.19, 1.20, 1.21, 1.22, 1.23, 1.24 | | v1.13 | 1.16, 1.17, 1.18, 1.19, 1.20, 1.21, 1.22, 1.23, 1.24, 1.25, 1.26 | | v1.14 | 1.19, 1.20, 1.21, 1.22, 1.23, 1.24, 1.25, 1.26, 1.27 | | v1.15 | 1.26, 1.27, 1.28, 1.29 | >[info] 参考：https://docs.cilium.io/en/v1.13/network/kubernetes/requirements/ # 安装cilium ## cilium cli方式 cilium CLI命令安装 ```shell CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/main/stable.txt) if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; elif [ "$(uname -m)" = "x86_64" ]; then CLI_ARCH=amd64; fi curl -L -O https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin rm -f cilium-linux-${CLI_ARCH}.tar.gz ``` 安装cilium ```shell $ cilium install --version 1.13.12 ℹ️ Using Cilium version 1.13.12 Auto-detected cluster name: kubernetes Auto-detected kube-proxy has been installed ``` >[info] 查看 cilium 所有版本 `cilium install --list-versions` 验证cilium ```shell $ cilium status --wait /¯¯\ /¯¯\__/¯¯\ Cilium: OK \__/¯¯\__/ Operator: OK /¯¯\__/¯¯\ Envoy DaemonSet: disabled (using embedded mode) \__/¯¯\__/ Hubble Relay: disabled \__/ ClusterMesh: disabled Deployment cilium-operator Desired: 1, Ready: 1/1, Available: 1/1 DaemonSet cilium Desired: 3, Ready: 3/3, Available: 3/3 Containers: cilium Running: 3 cilium-operator Running: 1 Cluster Pods: 0/7 managed by Cilium Helm chart version: 1.13.12 Image versions cilium-operator quay.io/cilium/operator-generic:v1.13.12@sha256:f83734bbe270f961d545c7929152785507ce04a05d818ebc9776941723736d02: 1 cilium quay.io/cilium/cilium:v1.13.12@sha256:d99204aa7b3b7bd2c9ab47fd398cc9f40290799bc0c7a4386c8dc5c1780cd3d3: 3 ``` ## helm方式【推荐】 ```shell $ helm repo add cilium https://helm.cilium.io/ "cilium" has been added to your repositories $ helm install cilium cilium/cilium --version 1.13.12 \ --namespace kube-system \ --set operator.replicas=1 \ --set k8sServiceHost=192.168.32.100 \ --set k8sServicePort=6443 NAME: cilium LAST DEPLOYED: Fri Mar 22 10:35:58 2024 NAMESPACE: kube-system STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: You have successfully installed Cilium with Hubble. Your release version is 1.13.12. For any further help, visit https://docs.cilium.io/en/v1.13/gettinghelp ``` 验证cilium ```shell $ kubectl -n kube-system get pod | grep cilium cilium-h5tb5 1/1 Running 0 23s cilium-n6sjg 1/1 Running 0 23s cilium-operator-6b465dcf6d-q7dtm 1/1 Running 0 23s cilium-vfl4d 1/1 Running 0 23s ``` # cilium开启hubble功能 Hubble 是 Cilium 的可观察层，可用于获取 Kubernetes 集群的网络和安全层的集群范围内的可见性 ## cilium cli方式 开启Hubble功能 ```shell $ cilium hubble enable --ui ``` >[info] 启用 Hubble 需要在运行 Cilium 的所有节点上打开 TCP 端口 4244。 这是 hubble-relay 正确运行所必需的。 运行 cilium status 以验证 Hubble 是否已启用并正在运行 ```shell $ cilium status /¯¯\ /¯¯\__/¯¯\ Cilium: OK \__/¯¯\__/ Operator: OK /¯¯\__/¯¯\ Envoy DaemonSet: disabled (using embedded mode) \__/¯¯\__/ Hubble Relay: OK \__/ ClusterMesh: disabled Deployment hubble-relay Desired: 1, Ready: 1/1, Available: 1/1 Deployment cilium-operator Desired: 1, Ready: 1/1, Available: 1/1 Deployment hubble-ui Desired: 1, Ready: 1/1, Available: 1/1 DaemonSet cilium Desired: 3, Ready: 3/3, Available: 3/3 Containers: cilium Running: 3 hubble-relay Running: 1 cilium-operator Running: 1 hubble-ui Running: 1 Cluster Pods: 7/7 managed by Cilium Helm chart version: 1.13.12 Image versions cilium quay.io/cilium/cilium:v1.13.12@sha256:d99204aa7b3b7bd2c9ab47fd398cc9f40290799bc0c7a4386c8dc5c1780cd3d3: 3 hubble-relay quay.io/cilium/hubble-relay:v1.13.12@sha256:01b23ea40bcd81145dde6bfcbfc4d542749d08c2a1c6348954c85123a8d2b1fc: 1 cilium-operator quay.io/cilium/operator-generic:v1.13.12@sha256:f83734bbe270f961d545c7929152785507ce04a05d818ebc9776941723736d02: 1 hubble-ui quay.io/cilium/hubble-ui:v0.13.0@sha256:7d663dc16538dd6e29061abd1047013a645e6e69c115e008bee9ea9fef9a6666: 1 hubble-ui quay.io/cilium/hubble-ui-backend:v0.13.0@sha256:1e7657d997c5a48253bb8dc91ecee75b63018d16ff5e5797e5af367336bc8803: 1 ``` ## helm方式 修改安装cilium参数 ```shell # 获取当前版本号 cilium_version=$(helm -n kube-system ls | awk '/cilium/ {print $NF}') echo $cilium_version # 备份上一次安装的参数 helm -n kube-system get values cilium > cilium_custom.yaml sed -i '1d' cilium_custom.yaml # 添加安装Hubble参数 cat <<EOF | tee -a cilium_custom.yaml >> /dev/null hubble: relay: enabled: true ui: enabled: true EOF # 修改Hubble参数 helm -n kube-system upgrade cilium cilium/cilium --version $cilium_version -f cilium_custom.yaml ``` 回滚版本 ```shell # 查看所有版本 $ helm -n kube-system history cilium # 回滚到上一个版本 $ helm -n kube-system rollback cilium Rollback was a success! Happy Helming! # 回滚到指定版本 $ helm -n kube-system rollback cilium 9 Rollback was a success! Happy Helming! ``` # 安装hubble cli ```shell HUBBLE_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/hubble/master/stable.txt) if [ "$(uname -m)" = "aarch64" ]; then HUBBLE_ARCH=arm64; elif [ "$(uname -m)" = "x86_64" ]; then HUBBLE_ARCH=amd64; fi curl -L -O https://github.com/cilium/hubble/releases/download/$HUBBLE_VERSION/hubble-linux-${HUBBLE_ARCH}.tar.gz sudo tar xzvfC hubble-linux-${HUBBLE_ARCH}.tar.gz /usr/local/bin rm -f hubble-linux-${HUBBLE_ARCH}.tar.gz ``` ## 验证 Hubble API 访问 1. 暴露hubble-relay服务 >[info] 默认hubble-relay服务是没有暴露端口的。hubble命令需要访问hubble-relay服务，所以需要把服务暴露出来 ```shell $ cilium hubble port-forward & [1] 31717 $ kubectl port-forward svc/hubble-relay -n kube-system 4245:80 & [1] 31425 ``` 2. 验证是否可以通过安装的 CLI 访问 Hubble API ```shell $ hubble status Healthcheck (via localhost:4245): Ok Current/Max Flows: 12,285/12,285 (100.00%) Flows/s: 89.72 Connected Nodes: 3/3 ``` 3. 命令行查看网络调用流 ```shell $ hubble observe ``` ## 验证 Hubble UI 访问 ```shell # 只暴露127.0.0.1地址 $ cilium hubble ui # 监听所有网卡地址 $ kubectl port-forward -n kube-system svc/hubble-ui --address 0.0.0.0 12000:80 ```