KEDA scale-to-zero issues

Gav Sturm 20 Reputation points
2025-04-29T15:58:15.2566667+00:00

KEDA HTTP Add-on (v0.10.0) Issues in AKS

Problem Description

We're experiencing issues with the KEDA HTTP Add-on (v0.10.0) in our AKS cluster. The main problem manifests in two ways:

  1. When routing requests through the KEDA HTTP interceptor, we consistently get 502 Bad Gateway errors after about 22 seconds. The interesting part is that direct connections to our target pods work fine, but the interceptor seems to be dropping the connection.
  2. Despite configuring our HTTPScaledObject with min: 0, the HPA that KEDA creates always sets minReplicas: 1. This prevents us from achieving true scale-to-zero functionality.

Environment

We're running this on an AKS cluster with GPU nodes, using KEDA v2.17.0 with the HTTP Add-on v0.10.0. Our setup includes an Nginx Ingress Controller and a GPU-based deployment that exposes port 5000.

Current Setup

Our request path looks like this:


Ingress -> ExternalName Service -> KEDA Interceptor Service -> Target Deployment

We've configured the HTTPScaledObject with min: 0 and max: 5, and set various timeout values (900s for interceptor and Nginx) to handle long-running requests.

What We've Found

When investigating the 502 errors, we discovered that while Nginx can successfully connect to the Interceptor Service, the requests never make it to our target pods. The interceptor pods don't show any logs of receiving these requests, and our target pods remain untouched. However, if we bypass the interceptor and connect directly to our target pods, everything works as expected.

For the scale-to-zero issue, we've confirmed that the HTTPScaledObject is correctly configured with min: 0, but the HPA that KEDA creates always sets minReplicas: 1. This seems to be a consistent behavior regardless of our configuration.

Questions

  1. Is the 22-second timeout a known limitation or bug in the KEDA HTTP Add-on?
  2. Why does the request path break down between Nginx and the Interceptor Service?
  3. Is there a specific configuration needed to enable proper request routing through the Interceptor?
  4. Why does the HPA ignore the min: 0 setting from the HTTPScaledObject?
  5. Are there any known issues with the KEDA HTTP Add-on v0.10.0 that could explain these behaviors?
Azure Kubernetes Service (AKS)
Azure Kubernetes Service (AKS)
An Azure service that provides serverless Kubernetes, an integrated continuous integration and continuous delivery experience, and enterprise-grade security and governance.
2,385 questions
{count} votes

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.