[ kubernetes  linux  opensource  ]

Running kubernetes custom scheduler

Kubernetes cluster have a default scheduler kube-scheduler. If the default scheduler does not suits our requirement we can also create our own scheduler. In the post we will discus how to create multiple scheduler and schedule pods based on different scheduler.


I am using the Virtualbox(running in Ubuntu 18.04 physical machine) for this entire setup . The physical machine is Dell inspiron laptop with 12GB RAM , Intel® Core™ i7-6500U CPU @ 2.50GHz × 4 and 512GB SSD hardisk.

Step 1: Copy the default scheduler yaml and modify

Copy the default scheduler yaml from master node and modify the name.

In this below example i am naming the custom schedule are my-scheduler. Add a ServiceAccount and ClusterRoleBinding to the yaml file as given below.

The key changes made are

vikki@kubernetes1:~$ sudo cp /etc/kubernetes/manifests/kube-scheduler.yaml custom-scheduler.yaml
vikki@kubernetes1:~$ sudo chmod 777 custom-scheduler.yaml 
vikki@kubernetes1:~$ vim custom-scheduler.yaml 
Step 2: Create the custom scheduler
vikki@kubernetes1:~$ kubectl create -f custom-scheduler.yaml 
serviceaccount/my-scheduler created
clusterrolebinding.rbac.authorization.k8s.io/my-scheduler-as-kube-scheduler created
pod/my-scheduler created
vikki@kubernetes1:~$ kubectl get pods -n=kube-system 
calico-kube-controllers-55754f75c-fgs8g 1/1 Running 7 22d
calico-node-4h72l 1/1 Running 7 22d
calico-node-ld84s 1/1 Running 7 22d
calico-node-lrfz9 1/1 Running 2 31h
calico-node-ws576 1/1 Running 1 27h
coredns-5644d7b6d9-2g6rs 1/1 Running 7 22d
coredns-5644d7b6d9-ccxsg 1/1 Running 7 22d
etcd-kubernetes1 1/1 Running 8 22d
kube-apiserver-kubernetes1 1/1 Running 8 22d
kube-controller-manager-kubernetes1 1/1 Running 8 22d
kube-proxy-6xd8l 1/1 Running 2 31h
kube-proxy-96q5x 1/1 Running 7 22d
kube-proxy-njl6r 1/1 Running 7 22d
kube-proxy-whlhw 1/1 Running 1 27h
kube-scheduler-kubernetes1 1/1 Running 8 22d
my-scheduler 1/1 Running 0 6s

Now we can see a new custom scheduler my-scheduler is running along with defautl scheduler kube-scheduler-kubernetes1

Step 3: Edit the custerrole for kube-scheduler

If RBAC is enabled on your cluster, you must update the system:kube-scheduler cluster role. Edit the clusterrole for kube-scheduler and modify.

Below are the key changes made in orignal file

vikki@kubernetes1:~$ kubectl edit clusterrole system:kube-scheduler
Step 4: Create pods with different types of scheduler

Create a pod without explicitly mentioning any scheuler name. This pod should be scheduled by default scheduler kube-scheduler

vikki@kubernetes1:~$ vim pod_default_scheduler.yaml

Create a pod by explicitly mentioning custom scheuler name my-scheduler.

vikki@kubernetes1:~$ vim pod_custom_scheduler.yaml 
vikki@kubernetes1:~$ kubectl create -f pod_default_scheduler.yaml 
pod/nginx-pod-default-scheduler created
vikki@kubernetes1:~$ kubectl create -f pod_custom_scheduler.yaml 
pod/nginx-pod-custom-scheduler created
vikki@kubernetes1:~$ kubectl get pods nginx-pod-custom-scheduler nginx-pod-default-scheduler
nginx-pod-custom-scheduler 1/1 Running 0 9m9s
nginx-pod-default-scheduler 1/1 Running 0 9m13s

Now we can see both the pods are created and running

vikki@kubernetes1:~$ kubectl get events -o wide |grep nginx-pod-custom-scheduler |head -1
<unknown> Normal Scheduled pod/nginx-pod-custom-scheduler my-scheduler Successfully assigned default/nginx-pod-custom-scheduler to kubernetes3 <unknown> 0 nginx-pod-custom-scheduler.15da76383c2c2859

vikki@kubernetes1:~$ kubectl get events -o wide |grep nginx-pod-default-scheduler |head -1
<unknown> Normal Scheduled pod/nginx-pod-default-scheduler default-scheduler Successfully assigned default/nginx-pod-default-scheduler to kubernetes3 <unknown> 0 nginx-pod-default-scheduler.15da76372f4444f7

From the events, we can see pod nginx-pod-custom-scheduler is scheduled by my-scheduler and nginx-pod-default-scheduler by  default-scheduler.

Share on:

Discussion and feedback