Multi Metrics Scaling Using KEDA and Prometheus

 

We've review KEDA usage with prometheus in this post. However, in real life things get complicated.

How do we handle scaling based on multiple metrics? KEDA does not provide support for this, and the documentation for such task is missing.

Let review an example: We have a deployment with multiple pods that handle some granular tasks. We want the scale the replica pods by the following metrics:

• CPU is over 80%
  or
• Memory is over 80%
  or
  
• Tasks rate per second per pod is over 100

First, we need to understand the requirements:

When do we want to scale up?

We want to scale up if any of these metrics is over the thresholds in any pod.

For example:

pod1 CPU=90%, Memory=50%, Tasks rate=20.

pod2 CPU=10%, Memory=50%, Tasks rate=20.

We should scale in this state even we have only a single metric above the threshold.

How do we achieve this?

The trick is to implement a new Prometheus metric with our application.

We create a code in our application that calculate the following metric:

scale_metric=max(memory_ratio, cpu_ratio, tasks_ratio)

Where

memory_ratio = used_memory_percentage / 80%

cpu_ratio = used_cpu_percentage / 80%

tasks_ratio = tasks_per_second / 100

Next we configure KEDA scaling by the max of this metric for all the pods:

triggers:
  - type: kafka
    metadata:
      serverAddress: {{ .Values.keda.prometheusServerUrl }}
      metricName: scale_metric
      threshold: '1'
      query: max(scale_metric)

Final Note

While this solution requires actual coding, and not just configuration, it provides a solid scaling based on all required features of our business logic.