Kubernetes uses Services and Ingress to abstract and manage load balancing, enabling scalable traffic distribution without manual infrastructure coordination.

The Problem: Traffic Distribution at Scale

In production, applications require traffic to be distributed across multiple pods reliably. Manual LB management doesn’t scale with dynamic pod lifecycles, network policies, and multi-team environments. Kubernetes abstracts this complexity through declarative resources.

How Kubernetes Handles Load Balancing

Core Components

1. Services: Abstracts pods behind a stable IP and DNS name, using kube-proxy to manage endpoint mappings.
2. Ingress: Manages L7 routing (HTTP/HTTPS) via controllers (e.g., nginx-ingress, Traefik) that translate rules into LB configurations.
3. Cloud Provider LBs: Integrates with external LBs (e.g., AWS ALB, GCP GLB) for external traffic.

Workflow: Deploying a Load-Balanced App

1. Deploy Pods with Labels:

   metadata:
     labels:
       app: myapp
   

2. Create a Service:

   apiVersion: v1
   kind: Service
   metadata:
     name: myapp-service
   spec:
     selector:
       app: myapp
     ports:
       - protocol: TCP
         port: 80
         targetPort: 8080
     type: LoadBalancer  # Cloud provider creates external LB
   

3. Verify Endpoints:

   kubectl get endpoints myapp-service
   # Should list healthy pod IPs
   

4. (Optional) Configure Ingress for L7 Routing:

   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     name: myapp-ingress
   spec:
     rules:
     - http:
         paths:
         - path: /api
           pathType: Prefix
           backend:
             service:
               name: myapp-service
               port:
                 number: 80
   

Policy Example: Enforcing LB Best Practices

Use admission controllers or OPA Gatekeeper to enforce:

• Services must not use type: LoadBalancer without cloud provider context (prevents resource creation failures).
• Ingress rules must specify hostname or path to avoid unintended routing.

Tooling

• kubectl: Debug services/endpoints (kubectl describe service myapp-service).
• Ingress Controllers: nginx-ingress, Traefik, or cloud-specific controllers (AWS ALB Ingress Controller).
• Cloud Provider CLI: Check external LB status (e.g., aws elbv2 describe-load-balancers).
• OpenShift Router: Built-in Ingress controller with TLS termination and metrics.

Tradeoffs and Caveats

• L4 vs L7:
  • L4 (Services) is simpler and lower latency but lacks path-based routing.
  • L7 (Ingress) adds flexibility but introduces controller-specific complexity.
• Cloud LB Costs: External LBs (e.g., AWS ALB) can become expensive at scale; consider shared Ingress controllers.
• Portability: Cloud provider LB integrations tie you to their platform.

Troubleshooting Common Issues

1. No External IP for LoadBalancer Service:
   • Check cloud provider quotas or permissions.
   • Verify type: LoadBalancer is supported in your environment.
2. Endpoints Not Populating:
   • Ensure pod labels match Service selector.
   • Check pod readiness probes (unhealthy pods won’t be added to endpoints).
3. Ingress 404/502 Errors:
   • Validate Ingress rule paths and backend service names.
   • Check ingress controller logs (kubectl logs -l app=nginx-ingress).
4. DNS Failures for External LBs:
   • Confirm DNS provider has propagated records.
   • Test with dig or nslookup outside the cluster.

Prevention: Observability and Testing

• Monitor: Track service latency, error rates, and endpoint counts (Prometheus + Grafana).
• Test: Use curl or k6 to simulate traffic and validate routing.
• Chaos Test: Kill pods or disrupt LBs to verify resilience.

In production, load balancing in Kubernetes works best when you treat it as a collaboration between declarative resources and infrastructure tooling—stay close to the metal with monitoring and testing, and avoid over-engineering abstractions that don’t map to your team’s operational capacity.

Source thread: How Is Load Balancing Really Used in Production with Kubernetes?