4.1.5 Get client IP from NGINX Ingress Controller

When using the Nginx Ingress Controller, it is necessary to check the real Client IP for monitoring, etc. in the application in the Pod. However, since the client’s request goes through the OCI Load Balancer used by the Nginx Ingress Controller, the IP of the Load Balancer is recorded, not the IP of the actual client. Here, we will learn how to get the real Client IP.

• References

  • Preserving The Client IP Address

  • How to Read a Remote IP Address Behind Ingress and Load Balancer in OKE

Check the application log in the default installation state

1. Install NGINX Ingress Controller in default state.

   • 4.1.1 Install NGINX Ingress Controller

2. Install the test app.

   • 4.1.2 Path-based test app installation

3. Check the Ingress address and connect.

   $ kubectl get ingress
   NAME                 CLASS    HOSTS   ADDRESS          PORTS   AGE
   ingress-path-basic   <none>   *       146.56.xxx.xxx   80      21m
   

4. Access the test app with a browser.

   Ex) http://146.56.xxx.xxx/blue

5. Check the application log.

   In the example, the client address is 10.0.0.20.21.

   • 10.0.20.x is the band of oke-svclbsubnet when the OKE cluster is created in Quick Create mode, and is the private IP of the Load Balancer linked with the Nginx Ingress Controller.

   $ kubectl logs -lapp=nginx-blue -f
   10.244.1.3 - - [02/Sep/2022:12:35:39 +0000] "GET /blue HTTP/1.1" 200 7272 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36" "10.0.20.21"
   

6. As you can see from the test, in the default Nginx Ingress Controller installation state, you can see that the Client IP taken from the application pod is not the actual client IP.

Setting related to get client IP

1. Download the NGINX Ingress Controller installation file.

   wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.3/deploy/static/provider/cloud/deploy.yaml
   

2. Make the following settings to preserve the Client IP.

   • Set the Load Balancer type to Network Load Balancer.

     oci.oraclecloud.com/load-balancer-type: "nlb"
     

   • Set Client IP preservation of Network Load Balancer.

     oci-network-load-balancer.oraclecloud.com/is-preserve-source: "true"
     

   • Designate externalTrafficPolicy as Local for Service.

   • Example of changing the downloaded NGINX Ingress Controller installation file

     ...
     ---
     apiVersion: v1
     kind: Service
     metadata:
       ...
       name: ingress-nginx-controller
       namespace: ingress-nginx
       annotations:
         oci.oraclecloud.com/load-balancer-type: "nlb"
         oci-network-load-balancer.oraclecloud.com/is-preserve-source: "true"
     spec:
       externalTrafficPolicy: Local
     ...
     

3. Deploy the NGINX Ingress Controller with the changed configuration file.

   kubectl apply -f deploy.yaml
   

   • If there is an existing NGINX Ingress Controller, delete it and redeploy it. Also, if you want to use the public IP of the existing Nginx Ingress Controller as it is, refer to the following to set the loadBalancerIP value.

     • Specifying Reserved Public IP Addresses

       apiVersion: v1
       kind: Service
       metadata:
         ...
         annotations:
           oci.oraclecloud.com/load-balancer-type: "nlb"
       spec:
         loadBalancerIP: 146.56.xx.xxx
         type: LoadBalancer
         ...
       

4. Set up the Security Rule.

   • Add the following rule to the Security List of Service Subnet so that the Load Balancer can access the Worker Node (10.0.10.0/24).

     • Security List name: Ex) oke-svclbseclist-quick-~~~~

     • Egress Rule to be added

       Stateless	Destination	IP Protocol	Source Port Range	Destination Port Range	Type and Code	Allows	Description
       No	10.0.10.0/24	TCP	All	30000-32767		TCP traffic for ports: 30000-32767

• Add the following rule to the Security List of Worker Node Subnet so that requests coming through Load Balancer from Worker Node (10.0.10.0/24) can be accessed by client IP.

  • Security List name: Ex) oke-nodeseclist-quick-~~~~

  • Ingress Rule to be added

    Stateless	Source	IP Protocol	Source Port Range	Destination Port Range	Type and Code	Allows	Description
    No	0.0.0.0/0	TCP	All	30000-32767		TCP traffic for ports: 30000-32767

5. Check your public IP through the relevant site in your web browser.

   

6. Access the test app with a browser.

   Ex) http://146.56.xxx.xxx/blue

   

7. Check the application log.

   In the example, the client address is 220.117.xxx.x, and you can see that the public IP of my work PC that I checked earlier is displayed.

   $ kubectl logs -lapp=nginx-blue -f
   10.244.1.4 - - [02/Sep/2022:14:03:35 +0000] "GET /blue HTTP/1.1" 200 7272 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36" "220.117.xxx.x"
   

8. If you make the relevant settings to preserve the Client IP according to the OCI document like the test result, you can check the actual Client IP from the application pod.