A step-by-step approach to safely migrate from Ingress to Gateway API using DNS switching, canary deployments, and thorough testing.

Workflow

1. Audit Existing Ingress Configurations

   • List all Ingress resources and annotations:

     kubectl get ingress --all-namespaces -o wide  
     kubectl describe ingress <name> --namespace <ns>  
     

   • Document TLS secrets, rewrite rules, timeouts, and authentication settings.

2. Convert Ingress to Gateway API Resources

   • Use ingress2gateway for initial translation:

     ingress2gateway convert -f <ingress.yaml> -o gateway/  
     

   • Treat output as a draft: Manually adjust TLS listener configurations, route priorities, and header modifications.

3. Deploy Gateway Resources Alongside Existing Ingress

   • Apply converted resources to the cluster:

     kubectl apply -f gateway/  
     

   • Ensure GatewayClass and HTTPRoutes reference the same backends as the original Ingress.

4. Test with Bare IPs and Routing Criteria

   • Bypass DNS and test directly using pod IPs or service IPs:

     curl -v --resolve <host-header> <gateway-ip>  
     

   • Validate path-based routing and query parameters using Host headers.

5. Canary Deployment by Route

   • Gradually shift traffic using weighted routing (e.g., Istio VirtualServices):

     # Example canary route split  
     http:  
       - match:  
           uri:  
             prefix: /v1  
         route:  
           - destination:  
               host: old-service  
               port:  
                 number: 80  
       - match:  
           uri:  
             prefix: /v2  
         route:  
           - destination:  
               host: new-service  
               port:  
                 number: 80  
     

6. DNS Switch and Monitor

   • Update DNS records to point to the new Gateway IP.
   • Monitor old Ingress traffic:

     kubectl logs -l app=old-ingress-controller --tail=50  
     

7. Decommission Old Ingress

   • Delete Ingress resources and associated secrets after traffic drops to zero.

Policy Example

Enforce Gateway API usage post-migration with an admission webhook:

apiVersion: admission.k8s.io/v1  
kind: ValidatingWebhookConfiguration  
metadata:  
  name: block-ingress-creation  
webhooks:  
  - name: block-ingress.hook.example  
    rules:  
      - apiGroups: ["networking.k8s.io"]  
        apiVersions: ["v1"]  
        operations: ["CREATE"]  
        resources: ["ingresses"]  
    failurePolicy: Fail  
    clientConfig:  
      service:  
        name: policy-webhook  
        namespace: kube-system  

Tooling

• ingress2gateway: Initial resource conversion (not production-ready).
• kubectl: For apply/describe/get operations.
• Network policies: Restrict traffic during testing.
• Prometheus/Grafana: Monitor traffic shifts and error rates.

Tradeoffs

• Manual effort: ingress2gateway misses annotation-driven behavior (e.g., NGINX timeouts, rewrites).
• Time investment: Canary deployments reduce risk but prolong migration.

Troubleshooting

• TLS errors: Verify secret names and mounting in Gateway TLS listeners.
• Missing headers: Check if Gateway API supports required header modifications (e.g., X-Forwarded-*).
• Route conflicts: Use kubectl get httproutes --all-namespaces to debug overlapping paths.
• Traffic drop: Confirm DNS propagation with dig or nslookup.

This approach minimizes downtime and avoids “big bang” failures. Start small, test ruthlessly, and phase out legacy resources only after validation.

Source thread: Anyone working on Ingress-to-Gateway API migration? Would love to connect and learn