Limit Range & Quota

Limit Range & Quota

LimitRange is an API object that limits resource usage per container or Pod in a Namespace It uses three relevant options:

type: specifies whether it applies to Pods or containers defaultRequest: the default resources the application will request default: the maximum resources the application can use

apiVersion: v1 
kind: LimitRange 
metadata:
  name: mem-limit-range
  namespace: limited
spec:
  limits:
  - default:
      memory: 125Mi
    defaultRequest:
      memory: 100Mi
    max: 
      memory: 500Mi 
    type: Container

This will setup a memory limit on containers

I will create a specific deployment and i will set up high memory usage to exceed limit range and see what happen

apiVersion: apps/v1
kind: Deployment
metadata:
  name: busybox-deploy
  namespace: limited
spec:
  selector:
      matchLabels:
        app: busybox-pod
  template:
    metadata:
      labels:
        app: busybox-pod
    spec:
      containers:
      - name: busybox
        image: busybox
        command:
        - sh
        - -c
        - sleep 3600
        resources:
          requests:
            memory: "700Mi"
            cpu: "500m"
          limits:
            memory: "700Mi"
            cpu: "500m"

Let’s see what happend

The pod is not created, and if i check the replicaset (since it is the close on to the pod)

touk@k8smaster:~/Documents/k8s/ResourceQuota&LimitRange$ k describe replicaset.apps/busybox-deploy-794845987  -n limited
Name:           busybox-deploy-794845987
Namespace:      limited
Selector:       app=busybox-pod,pod-template-hash=794845987
Labels:         app=busybox-pod
                pod-template-hash=794845987
Annotations:    deployment.kubernetes.io/desired-replicas: 1
                deployment.kubernetes.io/max-replicas: 2
                deployment.kubernetes.io/revision: 1
Controlled By:  Deployment/busybox-deploy
Replicas:       0 current / 1 desired
Pods Status:    0 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
  Labels:  app=busybox-pod
           pod-template-hash=794845987
  Containers:
   busybox:
    Image:      busybox
    Port:       <none>
    Host Port:  <none>
    Command:
      sh
      -c
      sleep 3600
    Limits:
      cpu:     500m
      memory:  700Mi
    Requests:
      cpu:        500m
      memory:     700Mi
  
  Type             Status  Reason
  ----             ------  ------
  ReplicaFailure   True    FailedCreate
Events:
  Type     Reason        Age               From                   Message
  ----     ------        ----              ----                   -------
  Warning  FailedCreate  86s               replicaset-controller  Error creating: pods "busybox-deploy-794845987-sl449" is forbidden: maximum memory usage per Container is 500Mi, but limit is 700Mi
  Warning  FailedCreate  86s               replicaset-controller  Error creating: pods "busybox-deploy-794845987-hbdfn" is forbidden: maximum memory usage per Container is 500Mi, but limit is 700Mi
  Warning  FailedCreate  86s               replicaset-controller  Error creating: pods "busybox-deploy-794845987-n97gq" is forbidden: maximum memory usage per Container is 500Mi, but limit is 700Mi
  Warning  FailedCreate  86s               replicaset-controller  Error creating: pods "busybox-deploy-794845987-bc7s4" is forbidden: maximum memory usage per Container is 500Mi, but limit is 700Mi
  Warning  FailedCreate  86s               replicaset-controller  Error creating: pods "busybox-deploy-794845987-sbhtx" is forbidden: maximum memory usage per Container is 500Mi, but limit is 700Mi
  Warning  FailedCreate  86s               replicaset-controller  Error creating: pods "busybox-deploy-794845987-t55kl" is forbidden: maximum memory usage per Container is 500Mi, but limit is 700Mi
  Warning  FailedCreate  86s               replicaset-controller  Error creating: pods "busybox-deploy-794845987-9dvxh" is forbidden: maximum memory usage per Container is 500Mi, but limit is 700Mi
  Warning  FailedCreate  86s               replicaset-controller  Error creating: pods "busybox-deploy-794845987-jcgmg" is forbidden: maximum memory usage per Container is 500Mi, but limit is 700Mi
  Warning  FailedCreate  85s               replicaset-controller  Error creating: pods "busybox-deploy-794845987-bpmj4" is forbidden: maximum memory usage per Container is 500Mi, but limit is 700Mi
  Warning  FailedCreate  4s (x6 over 84s)  replicaset-controller  (combined from similar events): Error creating: pods "busybox-deploy-794845987-f9xh5" is forbidden: maximum memory usage per Container is 500Mi, but limit is 700Mi

The ReplicSet failed to create the pod becuse it’s demands are higher than what the limit range set

Quota

Quota is an API object that limits total resources available in a Namespace If a Namespace is configured with Quota, applications in that Namespace must be configured with resource settings in pod.spec.containers.resources

Where the goal of the LimitRange is to set default restrictions for each application running in a Namespace, the goal of Quota is to define maximum resources that can be consumed within a Namespace by all

I will define a resource quota manifest file that wil be bound to a new name space called ‘team’

apiVersion: v1 
kind: ResourceQuota
metadata:
  name: team-quota
  namespace: team
spec:
  hard:
    pods: "10"
    requests.memory: "500Mi"

This will limit resources of memory and cpu on the team namespcae

I will create a deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: busybox-deploy
  namespace: team
spec:
  selector:
      matchLabels:
        app: busybox-pod
  template:
    metadata:
      labels:
        app: busybox-pod
    spec:
      containers:
      - name: busybox
        image: busybox
        command:
        - sh
        - -c
        - sleep 3600
        resources:
          requests:
            memory: "100Mi"

So when i try to scale it up to 7 replicas, it wont, because each pod is requesting 100Mi from memory and 5 pods will be the maximum and that’s why i have only 5 pods

touk@k8smaster:~/Documents/k8s/ResourceQuota&LimitRange$ kubectl scale deployment busybox-deploy --replicas=7 -n team
deployment.apps/busybox-deploy scaled
touk@k8smaster:~/Documents/k8s/ResourceQuota&LimitRange$ kubectl get pods -n team
NAME                              READY   STATUS              RESTARTS   AGE
busybox-deploy-648b8fbdb6-772jl   1/1     Running             0          6m22s
busybox-deploy-648b8fbdb6-9spvr   0/1     ContainerCreating   0          2s
busybox-deploy-648b8fbdb6-bdqst   0/1     ContainerCreating   0          2s
busybox-deploy-648b8fbdb6-fs9wn   0/1     ContainerCreating   0          2s
busybox-deploy-648b8fbdb6-mft2v   0/1     ContainerCreating   0          2s
touk@k8smaster:~/Documents/k8s/ResourceQuota&LimitRange$ k get resourcequota -n team
NAME         AGE   REQUEST                                    LIMIT
team-quota   17m   pods: 5/10, requests.memory: 500Mi/500Mi   
touk@k8smaster:~/Documents/k8s/ResourceQuota&LimitRange$ kubectl get pods -n team
NAME                              READY   STATUS    RESTARTS   AGE
busybox-deploy-648b8fbdb6-772jl   1/1     Running   0          6m34s
busybox-deploy-648b8fbdb6-9spvr   1/1     Running   0          14s
busybox-deploy-648b8fbdb6-bdqst   1/1     Running   0          14s
busybox-deploy-648b8fbdb6-fs9wn   1/1     Running   0          14s
busybox-deploy-648b8fbdb6-mft2v   1/1     Running   0          14s