Pod Disruption Budgets (PDBs) enforce availability guarantees during voluntary disruptions, but aggressive HPA scaling can conflict with these constraints. Here’s how to align them in production.

Context and Problem

Aggressive HPA scaling (e.g., rapid scale-up/down based on metrics) can collide with PDBs that limit pod evictions. Without careful tuning, PDBs may block necessary scaling actions, causing resource starvation or availability risks.

Actionable Workflow

1. Set minAvailable based on observed workload behavior:
   • Start with minAvailable: 1 for critical workloads, adjust upward if scale-down latency is acceptable.
   • Use activeRequests to balance scaling speed and availability.
2. Monitor readiness and scaling events:
   • Check kubectl describe pdb <name> for violation counts and constraints.
   • Watch HPA events with kubectl describe hpa <name> to detect scaling blocks.
3. Adjust PDBs dynamically:
   • For stateful workloads, increase minAvailable during peak hours.
   • Use vertical scaling or pod priorities as complementary strategies.

Policy Example

apiVersion: policy/v1  
kind: PodDisruptionBudget  
metadata:  
  name: my-app-pdb  
spec:  
  selector:  
    matchLabels:  
      app: my-app  
  minAvailable: 2  
  # Optional: limit voluntary pod evictions per minute  
  maxUnavailable: 1  

Tooling

• Check PDB status:

  kubectl get pdb --watch  
  kubectl describe pdb my-app-pdb  
  

• Monitor HPA scaling decisions:

  kubectl describe hpa my-app-hpa  
  kubectl logs -f <hp-controller-pod>  
  

• Metrics: Use Prometheus to track kube_pod_status_ready and kube_hpa_desired_replicas.

Tradeoffs and Caveats

• Resource overprovisioning: High minAvailable values can lead to unused resources during low demand.
• Readiness probe sensitivity: Overly strict probes may falsely trigger PDB violations.
• Voluntary disruption limits: PDBs don’t protect against node failures or cluster-wide issues.

Troubleshooting

• PDB blocking HPA scale-down:
  • Check kubectl get events --sort-by=.metadata.creationTimestamp for PodDisruptionBudget events.
  • Temporarily reduce minAvailable if scale-down is critical.
• Misconfigured selectors: Ensure PDB selectors align with deployment labels.
• HPA stuck in “ScalingActive”: Verify no external scale-down inhibitors (e.g., cluster autoscaler pauses).

Conclusion

Align PDBs with HPA by starting conservative, monitoring closely, and adjusting based on real-world behavior. Prioritize observability to catch conflicts early and avoid over-reliance on static configurations.

Source thread: How are you handling pod disruption budgets in clusters with aggressive HPA scaling?