Reducing GKE Pricing using Multiple Node Pools

 

In this post we'll review a method of reducing the GKE compute cost using multiple node pool.

The GKE (kubernetes cluster on Google Cloud Platform) is using compute instances. Each compute instance is a VM with specific CPU and memory properties. See this table for each VM type cost details.

When running pods on GKE, it is a good practice to allocate in advanced the CPU and memory resources, for example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      configid: my-container
  template:
    metadata:
      labels:
        configid: my-container        
    spec:
      containers:
        - name: container1
          image: my-image
          resources:
            requests:
              cpu: "2"
              memory: 2Gi

The example deployment is using 2 CPUs, and 2G RAM. The deployment pods will run on the GKE VMs, and consume its resources. By nature some of the deployments tend to be higher CPU consumers, while other deployments tend to be higher memory consumers. 

As we want to reduce costs, we can use VMs whose CPU is cheaper for the high CPU consumers deployments, and use VMs whose RAM is cheaper for the high RAM consumers deployments.

For example, checkout the e2-highcpu-8, and the e2-highmem-8 in the instances table. We can create 2 node pools, one for CPU consumers, and second for memory consumers:

gcloud container node-pools create pool-mem --cluster=mycluster --num-nodes=2 --machine-type=e2-highmem-8
gcloud container node-pools create pool-cpu --cluster=mycluster --num-nodes=2 --machine-type=e2-highcpu-8

Let's also auto-scale these pools:

gcloud container clusters update mycluster --enable-autoscaling --min-nodes 2 --max-nodes 15 --zone us-central1-c --node-pool pool-mem
gcloud container clusters update mycluster --enable-autoscaling --min-nodes 2 --max-nodes 8 --zone us-central1-c --node-pool pool-cpu

To assign a deployment to a pool, we add the node selector to its spec:

spec:
  nodeSelector:
    cloud.google.com/gke-nodepool: "pool-cpu"
  containers:
    ...

Final Note

We have used the method described in a specific project, and reduce GKE costs by 30%. We've used the memory pool for the redis pods. This is especially relevant as the redis is a single CPU process, and hence requires a lot of RAM, and only 1 CPU. Other pods, that were very high on CPU, used a very small amount of memory, and hence we assigned these to the CPU pool.