[TOC]
资源配额，通过 ResourceQuota 对象来定义，对每个命名空间的资源消耗总量提供限制。 它可以限制命名空间中某种类型的对象的总数目上限，也可以限制命令空间中的 Pod 可以使用的计算资源的总上限。
## 启用资源配额
资源配额的支持在很多 Kubernetes 版本中是默认启用的。 当 API 服务器 的命令行标志 --enable-admission-plugins= 中包含 ResourceQuota 时， 资源配额会被启用。
```shell
grep enable-admission-plugins /data/k8s/conf/kube-apiserver.conf
```
## 限制资源配置
```yaml
apiVersion: v1
kind: ResourceQuota
metadata:
name: quota
namespace: test
spec:
hard:
requests.cpu: "4"
requests.memory: "1Gi"
requests.storage: "100Gi"
pods: "4"
```
> **说明：**
> 如果所使用的是 CRI 容器运行时，容器日志会被计入临时存储配额。 这可能会导致存储配额耗尽的 Pods 被意外地驱逐出节点。 参考日志架构 了解详细信息。
## 生效资源限制
```shell
$ kubectl apply -f test-namespaces.yml
resourcequota/quota create
```
## 验证
### 验证内存
创建一个 `1Gi` 的deployment清单文件
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: client
name: client
spec:
replicas: 1
selector:
matchLabels:
app: client
strategy: {}
template:
metadata:
labels:
app: client
spec:
containers:
- image: busybox:1.24.1
name: busybox
args:
- sh
- -c
- "sleep 3600"
resources:
requests:
memory: 2Gi
```
创建容器
```shell
$ kubectl apply -f client.yml -n test
deployment.apps/client created
$ kubectl -n test get deployments.apps
NAME READY UP-TO-DATE AVAILABLE AGE
client 0/1 0 0 38s
$ kubectl -n test describe deployments.apps client
Name: client
Namespace: test
CreationTimestamp: Wed, 09 Feb 2022 11:21:25 +0800
Labels: app=client
Annotations: deployment.kubernetes.io/revision: 1
Selector: app=client
Replicas: 1 desired | 0 updated | 0 total | 0 available | 1 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels: app=client
Containers:
busybox:
Image: busybox:1.24.1
Port: <none>
Host Port: <none>
Args:
sh
-c
sleep 3600
Requests:
cpu: 50m
memory: 2Gi
Environment: <none>
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
Progressing True NewReplicaSetCreated
Available False MinimumReplicasUnavailable
ReplicaFailure True FailedCreate
OldReplicaSets: <none>
NewReplicaSet: client-9d57dfdf6 (0/1 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 20s deployment-controller Scaled up replica set client-9d57dfdf6 to 1
$ kubectl -n test describe rs client-9d57dfdf6
Name: client-9d57dfdf6
Namespace: test
Selector: app=client,pod-template-hash=9d57dfdf6
Labels: app=client
pod-template-hash=9d57dfdf6
Annotations: deployment.kubernetes.io/desired-replicas: 1
deployment.kubernetes.io/max-replicas: 2
deployment.kubernetes.io/revision: 1
Controlled By: Deployment/client
Replicas: 0 current / 1 desired
Pods Status: 0 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
Labels: app=client
pod-template-hash=9d57dfdf6
Containers:
busybox:
Image: busybox:1.24.1
Port: <none>
Host Port: <none>
Args:
sh
-c
sleep 3600
Requests:
cpu: 50m
memory: 2Gi
Environment: <none>
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
ReplicaFailure True FailedCreate
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedCreate 29s replicaset-controller Error creating: pods "client-9d57dfdf6-9x74f" is forbidden: exceeded quota: quota, requested: requests.memory=2Gi, used: requests.memory=1Gi, limited: requests.memory=1Gi
Warning FailedCreate 29s replicaset-controller Error creating: pods "client-9d57dfdf6-62kxp" is forbidden: exceeded quota: quota, requested: requests.memory=2Gi, used: requests.memory=1Gi, limited: requests.memory=1Gi
Warning FailedCreate 29s replicaset-controller Error creating: pods "client-9d57dfdf6-nx9bs" is forbidden: exceeded quota: quota, requested: requests.memory=2Gi, used: requests.memory=1Gi, limited: requests.memory=1Gi
Warning FailedCreate 29s replicaset-controller Error creating: pods "client-9d57dfdf6-zs9jl" is forbidden: exceeded quota: quota, requested: requests.memory=2Gi, used: requests.memory=1Gi, limited: requests.memory=1Gi
Warning FailedCreate 28s replicaset-controller Error creating: pods "client-9d57dfdf6-llrdj" is forbidden: exceeded quota: quota, requested: requests.memory=2Gi, used: requests.memory=1Gi, limited: requests.memory=1Gi
Warning FailedCreate 28s replicaset-controller Error creating: pods "client-9d57dfdf6-25qrk" is forbidden: exceeded quota: quota, requested: requests.memory=2Gi, used: requests.memory=1Gi, limited: requests.memory=1Gi
Warning FailedCreate 28s replicaset-controller Error creating: pods "client-9d57dfdf6-2tlxl" is forbidden: exceeded quota: quota, requested: requests.memory=2Gi, used: requests.memory=1Gi, limited: requests.memory=1Gi
Warning FailedCreate 28s replicaset-controller Error creating: pods "client-9d57dfdf6-fdl4j" is forbidden: exceeded quota: quota, requested: requests.memory=2Gi, used: requests.memory=1Gi, limited: requests.memory=1Gi
Warning FailedCreate 27s replicaset-controller Error creating: pods "client-9d57dfdf6-hjfnf" is forbidden: exceeded quota: quota, requested: requests.memory=2Gi, used: requests.memory=1Gi, limited: requests.memory=1Gi
Warning FailedCreate 8s (x4 over 26s) replicaset-controller (combined from similar events): Error creating: pods "client-9d57dfdf6-5xkj7" is forbidden: exceeded quota: quota, requested: requests.memory=2Gi, used: requests.memory=0, limited: requests.memory=1Gi
```
### 验证pod数量
创建一个5副本的deployment清单文件
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: web
name: web
spec:
replicas: 5
selector:
matchLabels:
app: web
strategy: {}
template:
metadata:
labels:
app: web
spec:
containers:
- image: nginx
name: nginx
resources:
requests:
memory: 100m
cpu: 50m
```
创建容器
```shell
$ kubectl apply -f web.yml -n test
deployment.apps/web created
$ kubectl -n test get pod
NAME READY STATUS RESTARTS AGE
web-584b96b57-24pk5 1/1 Running 0 43s
web-584b96b57-czr6q 1/1 Running 0 43s
web-584b96b57-m9hkv 1/1 Running 0 43s
web-584b96b57-szg9x 1/1 Running 0 43s
$ kubectl -n test get deployments.apps
NAME READY UP-TO-DATE AVAILABLE AGE
web 4/5 4 4 30s
$ kubectl -n test describe rs web-584b96b57
Name: web-584b96b57
Namespace: test
Selector: app=web,pod-template-hash=584b96b57
Labels: app=web
pod-template-hash=584b96b57
Annotations: deployment.kubernetes.io/desired-replicas: 5
deployment.kubernetes.io/max-replicas: 7
deployment.kubernetes.io/revision: 1
Controlled By: Deployment/web
Replicas: 4 current / 5 desired
Pods Status: 4 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
Labels: app=web
pod-template-hash=584b96b57
Containers:
nginx:
Image: nginx
Port: <none>
Host Port: <none>
Requests:
cpu: 50m
memory: 100m
Environment: <none>
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
ReplicaFailure True FailedCreate
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SuccessfulCreate 70s replicaset-controller Created pod: web-584b96b57-m9hkv
Normal SuccessfulCreate 70s replicaset-controller Created pod: web-584b96b57-24pk5
Normal SuccessfulCreate 70s replicaset-controller Created pod: web-584b96b57-szg9x
Warning FailedCreate 70s replicaset-controller Error creating: pods "web-584b96b57-4ttxz" is forbidden: exceeded quota: quota, requested: pods=1, used: pods=4, limited: pods=4
Normal SuccessfulCreate 70s replicaset-controller Created pod: web-584b96b57-czr6q
Warning FailedCreate 70s replicaset-controller Error creating: pods "web-584b96b57-jv9mp" is forbidden: exceeded quota: quota, requested: pods=1, used: pods=4, limited: pods=4
Warning FailedCreate 70s replicaset-controller Error creating: pods "web-584b96b57-7vsjh" is forbidden: exceeded quota: quota, requested: pods=1, used: pods=4, limited: pods=4
Warning FailedCreate 70s replicaset-controller Error creating: pods "web-584b96b57-7pbxc" is forbidden: exceeded quota: quota, requested: pods=1, used: pods=4, limited: pods=4
Warning FailedCreate 70s replicaset-controller Error creating: pods "web-584b96b57-sdlgw" is forbidden: exceeded quota: quota, requested: pods=1, used: pods=4, limited: pods=4
Warning FailedCreate 70s replicaset-controller Error creating: pods "web-584b96b57-ksjzx" is forbidden: exceeded quota: quota, requested: pods=1, used: pods=4, limited: pods=4
Warning FailedCreate 70s replicaset-controller Error creating: pods "web-584b96b57-gqk28" is forbidden: exceeded quota: quota, requested: pods=1, used: pods=4, limited: pods=4
Warning FailedCreate 70s replicaset-controller Error creating: pods "web-584b96b57-spczj" is forbidden: exceeded quota: quota, requested: pods=1, used: pods=4, limited: pods=4
Warning FailedCreate 70s replicaset-controller Error creating: pods "web-584b96b57-8kzvt" is forbidden: exceeded quota: quota, requested: pods=1, used: pods=4, limited: pods=4
Warning FailedCreate 20s (x12 over 69s) replicaset-controller (combined from similar events): Error creating: pods "web-584b96b57-rjkxh" is forbidden: exceeded quota: quota, requested: pods=1, used: pods=4, limited: pods=4
```
### 其他验证
请自行测试，这里就不进行一一验证。
> 说明：
> 1、创建deployment必须设置 `requests.memory` 和 `requests.cpu`，否则创建pod失败
> 2、查看deployment创建成功，但是pod没有创建出来。那得看 `rs` 创建的情况
## 参考文章
https://kubernetes.io/zh/docs/concepts/policy/resource-quotas/
- 前言
- 架构
- 部署
- kubeadm部署
- kubeadm扩容节点
- 二进制安装基础组件
- 添加master节点
- 添加工作节点
- 选装插件安装
- Kubernetes使用
- k8s与dockerfile启动参数
- hostPort与hostNetwork异同
- 应用上下线最佳实践
- 进入容器命名空间
- 主机与pod之间拷贝
- events排序问题
- k8s会话保持
- 容器root特权
- CNI插件
- calico
- calicoctl安装
- calico网络通信
- calico更改pod地址范围
- 新增节点网卡名不一致
- 修改calico模式
- calico数据存储迁移
- 启用 kubectl 来管理 Calico
- calico卸载
- cilium
- cilium架构
- cilium/hubble安装
- cilium网络路由
- IP地址管理(IPAM)
- Cilium替换KubeProxy
- NodePort运行DSR模式
- IP地址伪装
- ingress使用
- nginx-ingress
- ingress安装
- ingress高可用
- helm方式安装
- 基本使用
- Rewrite配置
- tls安全路由
- ingress发布管理
- 代理k8s集群外的web应用
- ingress自定义日志
- ingress记录真实IP地址
- 自定义参数
- traefik-ingress
- traefik名词概念
- traefik安装
- traefik初次使用
- traefik路由(IngressRoute)
- traefik中间件(middlewares)
- traefik记录真实IP地址
- cert-manager
- 安装教程
- 颁布者CA
- 创建证书
- 外部存储
- 对接NFS
- 对接ceph-rbd
- 对接cephfs
- 监控平台
- Prometheus
- Prometheus安装
- grafana安装
- Prometheus配置文件
- node_exporter安装
- kube-state-metrics安装
- Prometheus黑盒监控
- Prometheus告警
- grafana仪表盘设置
- 常用监控配置文件
- thanos
- Prometheus
- Sidecar组件
- Store Gateway组件
- Querier组件
- Compactor组件
- Prometheus监控项
- grafana
- Querier对接grafana
- alertmanager
- Prometheus对接alertmanager
- 日志中心
- filebeat安装
- kafka安装
- logstash安装
- elasticsearch安装
- elasticsearch索引生命周期管理
- kibana安装
- event事件收集
- 资源预留
- 节点资源预留
- imagefs与nodefs验证
- 资源预留 vs 驱逐 vs OOM
- scheduler调度原理
- Helm
- Helm安装
- Helm基本使用
- 安全
- apiserver审计日志
- RBAC鉴权
- namespace资源限制
- 加密Secret数据
- 服务网格
- 备份恢复
- Velero安装
- 备份与恢复
- 常用维护操作
- container runtime
- 拉取私有仓库镜像配置
- 拉取公网镜像加速配置
- runtime网络代理
- overlay2目录占用过大
- 更改Docker的数据目录
- Harbor
- 重置Harbor密码
- 问题处理
- 关闭或开启Harbor的认证
- 固定harbor的IP地址范围
- ETCD
- ETCD扩缩容
- ETCD常用命令
- ETCD数据空间压缩清理
- ingress
- ingress-nginx header配置
- kubernetes
- 验证yaml合法性
- 切换KubeProxy模式
- 容器解析域名
- 删除节点
- 修改镜像仓库
- 修改node名称
- 升级k8s集群
- 切换容器运行时
- apiserver接口
- 其他
- 升级内核
- k8s组件性能分析
- ETCD
- calico
- calico健康检查失败
- Harbor
- harbor同步失败
- Kubernetes
- 资源Terminating状态
- 启动容器报错