How to safely drain the k8s node!
Of course, we may have to drain the node occasionally for patching and maintenance
Use the command Kubectl drain <nodename>
An interesting fact is, You can use this “kubectl drain” command on a single node only but what if you executed it parallel in multiple terminals(Maybe accidentally or for fun).
Of course, concurrent execution will happen and all nodes will be drained. Now HIGH AVAILABILITY is gone (Yeah you drained all nodes completely right !! ) and you are in a danger zone.
How do you ensure high availability all the time irrespective of the situation?
Yes, #PodDisruptionBudget is the solution.
You can create a deployment of kind #PodDisruptionBudget where You will have an attribute #minAvailable. You can
mention the minimum number of pods to be running all the time by setting the value to this minAvailable attribute in PodDisruptionBudget deployment file. So even you tried draining all the nodes via concurrent
execution, that will still obey PodDisruptionBudget and keep the minimum number of pods running.
Another question is, What if the eviction is struck during the process? Oh, I am not making this post any longer. Let’s discuss that another time.
Anyway, Have you had any hands-on with pod maintenance? Any interesting scenarios that you handled during the pod maintenance?
