Multiple kubectl scale command high CPU usage

Question

I'm trying to run a 'kubectl scale' command on all the deployments in a namespace (around 30). I'm creating a list of all deployments, and then run one by one (with xargs / foreach) with a 'kubectl scale' command.

Few technical details:

• AWS EKS cluster, version 1.20.
• kubectl client vesrion 1.23.5
• kubectl server version 1.20.11-eks-f17b81

When I'm doing it, I can see a massive CPU usage (in many cases its also causing the app to crash). I'm currently running it in a kubernetes pod with limits of CPU: 1200m, MEM:2Gi. (however, it's also happening when I run it locally on my computer).

To show the problem, I run the scale, and at the same time running both "kubectl get deployments" and "kubectl top pod" to show the status of the namespace scaling and the resources usage of the pod running the scales. In this example I'm scaling all deployments replicas from 0 to 1.

As you can see here, there is a massive increase of the CPU usage after a few scale commands (in this case around 15, there are times that even with less).

After it finished running the scale commands on all deployments, and some of them finished the scale (the pod created successfully), the CPU usage started to decrease.

And even before all pods were created, the CPU returned to its normal

So the question is why the scale command causing such a massive usage of the cpu? This command is using (as far as I understand) the kubernetes API, so its not something that actually runs on it, so why its getting so high CPU usage?

I originally thought that the CPU usage is related with some wait of the scale command to actual finish (for the pods to be recreated) and that its continue to do some stuff on the background until its happening. However - the decrease of the CPU usage before all pods were created disproved it.

Thanks, Afik