Application Gateway Ingress Controller helm chart not working as expected with Azure CNI Overlay.

Hi Demian Sciessere,

Welcome to Microsoft Q&A Platform.

To successfully configure the Application Gateway Ingress Controller (AGIC) with an overlay CNI in Azure Kubernetes Service (AKS), there are a few key considerations to make sure your setup works correctly.

Steps to Troubleshoot and Configure AGIC with Overlay CNI:

 1.Network Configuration:

• Ensure that your AKS cluster and Application Gateway (AppGW) are correctly peered, which you have mentioned is in place.
• Verify that the overlay CNI is set up correctly, especially with respect to routing and IP address availability.

2.Service Annotations:

• Use specific annotations in your Kubernetes service to inform AGIC how to handle traffic routing. Make sure that your services expose the proper NodePort for the backend Pods. Example annotations include: service.beta.kubernetes.io/azure-load-balancer-mode: "overlay"

3.Backend Pools Configuration:

• Since you're having issues with the Pod IPs being unreachable due to overlay networking, consider keeping the backend pools aligned with Node IPs instead:
• ⁠Use the node-ip:node-port approach, but create a custom Helm value that avoids AGIC from overriding this configuration.
• Refer to this Helm value when installing AGIC:  

# Custom-values.yaml
backendPools:
  - name: <backend-pool-name>
    backendIPConfiguration:
      - ipAddress: <node-ip>
      - port: <node-port>

4.Health Probes:

• Make sure that health probes are configured to match your service’s NodePorts; set them correctly to ensure the health checks are returning a successful status when using NodePorts.
• Verify your configuration parameters for health probes in AGIC. You may want to modify the health probe in your Application Gateway to check against NodePorts instead of pod IPs:
  • Use Azure CLI or AZ commands to configure:  

az network application-gateway probe create --resource-group <resource-group> --gateway-name <appgw-name> --name <probe-name> --protocol http --host <node-ip> --path /health --interval 30 --timeout 30 --threshold 2 --enable-http2

5.AGIC Version:

• Since you’re using version 1.9.2, ensure that this version does provide the compatibility you need with overlay networking. If not, consider upgrading to a later version, if possible, as there might be patches or fixes for known issues related to your setup.

Please refer : https://learn.microsoft.com/en-us/azure/aks/configure-azure-cni?tabs=configure-networking-powershell
https://learn.microsoft.com/en-us/azure/aks/azure-cni-overview
https://learn.microsoft.com/en-us/azure/application-gateway/ingress-controller-overview#container-networking-and-agic

Hello @Satish Mada , thank you for your response.

About point 2:

I don't know which annotations in particular do you mean. We have configured AGIC according to it's annotations which are very limited and no reference to Overlay, not backend configurations:
https://github.com/Azure/application-gateway-kubernetes-ingress/blob/master/docs/annotations.md

3:
Where can I get that configuration from ?
Looking at the Helm Chart, there is no configuration at all that mentions the backendpools are you suggested:

https://github.com/Azure/application-gateway-kubernetes-ingress/blob/master/helm/ingress-azure/values.yaml

4:

It is impossible on doing what you say as AGIC removes the manual configuration. That's why we ask how to configure using AGIC

5:

1.9.2 is the last version:

https://github.com/Azure/application-gateway-kubernetes-ingress/releases

Thank you