Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ScaledObject downscales deployment to 0 replicas outside specified timeframe in cron trigger even when cpu trigger should keep it running #6057

Open
JuniorJPDJ opened this issue Aug 9, 2024 · 2 comments · May be fixed by #6174
Labels
bug Something isn't working

Comments

@JuniorJPDJ
Copy link

Report

On time XX:33 it starts, when cpu is being used it scales up, but then even if more than 1 replicas are here on XX:55 its just scaling it all down to 0 replicas.

The ScaledObject:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: <REDACTED>
  namespace: <REDACTED>
spec:
  advanced:
    horizontalPodAutoscalerConfig:
      behavior:
        scaleDown:
          policies:
          - periodSeconds: 15
            type: Percent
            value: 100 
          stabilizationWindowSeconds: 300 
    restoreToOriginalReplicaCount: true
    scalingModifiers: {}
  cooldownPeriod: 10
  maxReplicaCount: 3
  minReplicaCount: 0
  pollingInterval: 30
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: <REDACTED>
  triggers:
  - metadata:
      value: "70"
    metricType: Utilization
    type: cpu
  - metadata:
      desiredReplicas: "1"
      end: 55 * * * *
      start: 33 * * * *
      timezone: Etc/UTC
    type: cron

Expected Behavior

After reading this: https://keda.sh/docs/2.13/reference/faq/#using-multiple-triggers-for-the-same-scale-target
It seems like the deployment shouldn't scale down until the CPU trigger is keeping it up.
It should only downscale to 0 when there is low CPU usage and we are not inside timeframe defined for cron trigger.

Actual Behavior

After timeframe of cron trigger is over, instantly deployment scales down to 0.

Steps to Reproduce the Problem

  1. Setup the SO like mine or similar.
  2. Run some CPU-intensive task (like sha1sum /dev/zero) inside a pod and wait it to scale up.
  3. Wait for timeframe of cron trigger to be over

Logs from KEDA operator

No response

KEDA Version

2.13.1

Kubernetes Version

1.29

Platform

Amazon Web Services

Scaler Details

cpu, cron

Anything else?

No response

@JuniorJPDJ JuniorJPDJ added the bug Something isn't working label Aug 9, 2024
@JuniorJPDJ
Copy link
Author

This is similar or duplicate to #5620 which is closed due to inactivity

@kunwooy
Copy link
Contributor

kunwooy commented Sep 13, 2024

This issue is very difficult to solve with the current implementation of CPU scaler. The biggest problem is that resource-metrics based scalers (like CPU and memory) do not have activationThreshold field which is crucial for determining whether the deployment should be scaled to zero or not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: To Triage
2 participants