4.4.1 Deploying microservice apps without Service Mesh

This is a review of the contents of the Istio document below to find out what problems may occur when using a microservice app and why a Service Mesh like Istio is needed.

• https://istio.io/latest/docs/examples/microservices-istio/bookinfo-kubernetes/
• https://istio.io/latest/docs/examples/microservices-istio/add-new-microservice-version/

Deploy the test microservice app (Bookinfo)

Test app

• Consists of 4 microservice apps: Product, Review, Details, and Ratings.

• First of all, the Reviews service only distributes v1 here.

• Check the distribution and results in the state that istio is not set up yet.

  

1. Deploy the app

   kubectl apply -l version!=v2,version!=v3 -f https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/platform/kube/bookinfo.yaml
   

2. Scale

   kubectl scale deployments --all --replicas 3
   

3. Open service for external access

   It is opened as a Load Balancer type for test convenience.

   kubectl patch svc productpage -p '{"spec": {"type": "LoadBalancer"}}'
   

4. Check deployment status

   You can see that each app has 3 Pods, and each Pod consists of 1 Container.

   oke_admin@cloudshell:~ (ap-seoul-1)$ kubectl get pod
   NAME                              READY   STATUS    RESTARTS   AGE
   details-v1-79f774bdb9-29z48       1/1     Running   0          21s
   details-v1-79f774bdb9-d5lfq       1/1     Running   0          32s
   details-v1-79f774bdb9-sqp9p       1/1     Running   0          20s
   productpage-v1-6b746f74dc-drgbw   1/1     Running   0          21s
   productpage-v1-6b746f74dc-pc2h2   1/1     Running   0          32s
   productpage-v1-6b746f74dc-qzkqm   1/1     Running   0          20s
   ratings-v1-b6994bb9-6xlq7         1/1     Running   0          32s
   ratings-v1-b6994bb9-j2l78         1/1     Running   0          20s
   ratings-v1-b6994bb9-w748t         1/1     Running   0          20s
   reviews-v1-545db77b95-hg7j8       1/1     Running   0          32s
   reviews-v1-545db77b95-l42v4       1/1     Running   0          20s
   reviews-v1-545db77b95-lgl89       1/1     Running   0          20s
   

5. Check service

   oke_admin@cloudshell:~ (ap-seoul-1)$ kubectl get svc
   NAME          TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)          AGE
   details       ClusterIP      10.96.175.211   <none>          9080/TCP         67s
   kubernetes    ClusterIP      10.96.0.1       <none>          443/TCP          16d
   productpage   LoadBalancer   10.96.79.55     146.56.142.95   9080:32649/TCP   66s
   ratings       ClusterIP      10.96.105.212   <none>          9080/TCP         66s
   reviews       ClusterIP      10.96.197.86    <none>          9080/TCP         66s
   

6. Access the main page

   • http://{productpage external ip}:9080/productpage
   • You can see that each service is called and a normal page is displayed.
   • In Review v1, rating information is not visible because it is not linked with the rating service yet.

   

7. Check the product page calling code

   • Source file: https://github.com/istio/istio/blob/release-1.12/samples/bookinfo/src/productpage/productpage.py
   • If you look at the review call address, you can see that it is http://{reviewsHostname}.{servicesDomain}:9080/reviews.

   

   • Actual call log

     If you look at the actual log, you can see that http://reviews:9080/reviews was called normally, and reviews were normally called as the address in the same namespace through the reviews Service of ClusterIP type at the time of deployment.

     oke_admin@cloudshell:~ (ap-seoul-1)$ kubectl logs -l app=productpage -f
     ...
     DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): reviews:9080
     send: b'GET /reviews/0 HTTP/1.1\r\nHost: reviews:9080\r\nuser-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\nX-B3-TraceId: cd0101ed97c6b43f\r\nX-B3-SpanId: cd0101ed97c6b43f\r\nX-B3-Sampled: 1\r\n\r\n'
     reply: 'HTTP/1.1 200 OK\r\n'
     header: X-Powered-By: Servlet/3.1
     header: Content-Type: application/json
     header: Date: Mon, 20 Dec 2021 07:41:55 GMT
     header: Content-Language: en-US
     header: Content-Length: 295
     DEBUG:urllib3.connectionpool:http://reviews:9080 "GET /reviews/0 HTTP/1.1" 200 295
     INFO:werkzeug:10.244.1.128 - - [20/Dec/2021 07:41:55] "GET /productpage HTTP/1.1" 200 
     

Reviews New version release

1. Check the Reviews Service

   We are distributing to Pods labeled app=reviews as shown in Selector. It is being distributed as Endpoints of the three Pods of Reviews v1, which are currently distributed.

   oke_admin@cloudshell:~ (ap-seoul-1)$ kubectl describe svc reviews
   Name:              reviews
   Namespace:         default
   Labels:            app=reviews
                      service=reviews
   Annotations:       <none>
   Selector:          app=reviews
   Type:              ClusterIP
   IP Family Policy:  SingleStack
   IP Families:       IPv4
   IP:                10.96.197.86
   IPs:               10.96.197.86
   Port:              http  9080/TCP
   TargetPort:        9080/TCP
   Endpoints:         10.244.2.42:9080,10.244.2.44:9080,10.244.2.45:9080
   Session Affinity:  None
   Events:            <none>
   

2. If you develop and distribute v2, a new version of the Reviews app in this state, you will probably add app=review to service routing after deploying and verifying normal operation. After that, we will actually operate to service some requests and restore them when there is a problem, or if there is no problem, we will reduce the requests to v1 and gradually increase the requests to v2 and transfer them to the actual v2.

Reviews v2 distribution

1. Deploy the Reviews v2 version.

   Deploy by relabeling app: reviews_test to exclude it from routing.

   curl -s https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/platform/kube/bookinfo.yaml | sed 's/app: reviews/app: reviews_test/' | kubectl apply -l app=reviews_test,version=v2 -f -
   

2. Check the deployment result

   oke_admin@cloudshell:~ (ap-seoul-1)$ kubectl get pod -o wide
   NAME                              READY   STATUS    RESTARTS   AGE   IP             NODE          NOMINATED NODE   READINESS GATES
   ...
   reviews-v1-545db77b95-hg7j8       1/1     Running   0          76m   10.244.2.42    10.0.10.9     <none>           <none>
   reviews-v1-545db77b95-l42v4       1/1     Running   0          76m   10.244.2.45    10.0.10.9     <none>           <none>
   reviews-v1-545db77b95-lgl89       1/1     Running   0          76m   10.244.2.44    10.0.10.9     <none>           <none>
   reviews-v2-7cd8b85558-c2jmw       1/1     Running   0          30s   10.244.2.47    10.0.10.9     <none>           <none>
   

3. Test Reviews v2 Pod

   It would be nice to test with the Reviews v2 Pod in the productpage Pod, but since there is no curl for testing in the productpage Pod, I deploy a separate Pod for the client and test it.

   • Deploy test client pods

     kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.12/samples/sleep/sleep.yaml
     

   • test

     REVIEWS_V2_POD_IP=$(kubectl get pod -l app=reviews_test,version=v2 -o jsonpath='{.items[0].status.podIP}')
     echo $REVIEWS_V2_POD_IP
     kubectl exec $(kubectl get pod -l app=sleep -o jsonpath='{.items[0].metadata.name}') -- curl -sS "$REVIEWS_V2_POD_IP:9080/reviews/7"
     

4. Test results

   Confirm that the v2 version works well as shown below.

   oke_admin@cloudshell:~ (ap-seoul-1)$ REVIEWS_V2_POD_IP=$(kubectl get pod -l app=reviews_test,version=v2 -o jsonpath='{.items[0].status.podIP}')
   oke_admin@cloudshell:~ (ap-seoul-1)$ echo $REVIEWS_V2_POD_IP
   10.244.2.47
   oke_admin@cloudshell:~ (ap-seoul-1)$ kubectl exec $(kubectl get pod -l app=sleep -o jsonpath='{.items[0].metadata.name}') -- curl -sS "$REVIEWS_V2_POD_IP:9080/reviews/7" {"id": "7","reviews": [{  "reviewer": "Reviewer1",  "text": "An extremely entertaining play by Shakespeare. The slapstick humour is refreshing!", "rating": {"stars": 5, "color": "black"}},{  "reviewer": "Reviewer2",  "text": "Absolutely fun and entertaining. The play lacks thematic depth when compared to other plays by Shakespeare.", "rating": {"stars": 4, "color": "black"}}]}
   

Added to service routing in Reviews v2

1. Add the label app=reviews to the Review v2 Pod labeled version=v2.

   oke_admin@cloudshell:~ (ap-seoul-1)$ kubectl label pod -l version=v2 app=reviews --overwrite
   pod/reviews-v2-7cd8b85558-c2jmw labeled
   

2. Recheck Reviews Service

   As shown below, v2 has been added and you can see that Endpoints has changed from the previous 3 to 4.

   oke_admin@cloudshell:~ (ap-seoul-1)$ kubectl describe svc reviews
   Name:              reviews
   ...
   Selector:          app=reviews
   ...
   Endpoints:         10.244.2.42:9080,10.244.2.44:9080,10.244.2.45:9080 + 1 more...
   

3. Access the main page

   • http://{productpage external ip}:9080/productpage

   • When the page is refreshed, 1 out of 4 is routed to the Review v2 Pod, and you can see the star rating as shown in the picture below.

     

Reviews v2 Redistributing after checking the problem

1. If a problem occurs during service, delete it as follows.

   • Since we changed the label of the existing Pod for service routing above, the Deployment of app=reviews_test still remains in addition to the changed Pod (app=reviews). Delete both. (Actually, you shouldn’t do this. However, the label of the Pod Template defined in the Deployment cannot be changed…)

   kubectl delete deployment reviews-v2
   kubectl delete pod -l app=reviews,version=v2
   

2. Redistribution after troubleshooting

   In the final distribution, the number of v2 will be increased and v1 will be deleted after distribution as shown below. In a real world situation, there will also be a requirement to gradually increase v2 and decrease v1.

   kubectl apply -l app=reviews,version=v2 -f https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/platform/kube/bookinfo.yaml
   kubectl scale deployment reviews-v2 --replicas=3
   kubectl delete deployment reviews-v1
   

How to respond to actual distribution

In order to ensure stable deployment when updating microservices in a real environment, various methods have been proposed for testing after the new version is deployed and for gradual migration to the new version. As a representative method, the most used method is largely based on Service Mesh and Netflix’s Hystrix.

• Service Mesh: The method to use the service mesh represented by Istio is to add a module as a sidecar in the Service Mesh stage. A sidecar container is added to the pod in addition to the application container, and service routing and additional functions are provided using the sidecar. The advantage is that no extra work is required in the application code.
• Application code implementation: As in the example of Netflix’s Hystrix, functions required for service routing, security, monitoring, etc. are provided by the library module. Because the library is used, the amount of coding itself is not large, and annotations are used. The Spring framework has evolved into Spring Cloud to provide functions.