TheKoguryo's Tech Blog

Version 2023.06.19

Warning

This content has been generated by machine translation. The translations are automated and have not undergone human review or validation.

4.4.2 Applying Service Mesh - Istio to microservices apps

Installation is possible with istioctl, Helm, or manual. Here, we will install the istioctl standard described as an example in the OKE document and check that it is applied to the service according to the Istio document.

Install Istio

  1. Connect to Cloud Shell or work environment.

  2. Download Istio

    curl -L https://istio.io/downloadIstio | ISTIO_VERSION=1.12.1 TARGET_ARCH=x86_64 sh -

  3. Go to the Istio route

    cd istio-1.12.1

  4. Add to PATH environment variable

    export PATH=$PWD/bin:$PATH

  5. Run pre-verification

    istioctl x precheck

  6. Installation

    istioctl install --set profile=demo

  7. Scale

    Scale beyond 2 to account for upgrades that are set to pod’s disruption budget default of 1.

    kubectl scale --replicas=2 deployment -n istio-system istio-egressgateway
kubectl scale --replicas=2 deployment -n istio-system istio-ingressgateway
kubectl scale --replicas=2 deployment -n istio-system istiod

Istio Enable

productpage service Istio Enable

  1. Redistributing by adding Istio sidecar

    Add sidecar to productpage app only with istioctl kube-inject command.

    curl -s https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/platform/kube/bookinfo.yaml | istioctl kube-inject -f - | sed 's/replicas: 1/replicas: 3/g' | kubectl apply -l app=productpage,version=v1 -f -

  2. Check the results

    You can see that the istio-proxy container has been added in the product Pod.

    oke_admin@cloudshell:~ (ap-seoul-1)$ kubectl get pod
NAME                            READY   STATUS    RESTARTS   AGE
details-v1-79f774bdb9-29z48     1/1     Running   0          89m
details-v1-79f774bdb9-d5lfq     1/1     Running   0          89m
details-v1-79f774bdb9-sqp9p     1/1     Running   0          89m
productpage-v1-c6885474-6g754   2/2     Running   0          16s
productpage-v1-c6885474-r9l8l   2/2     Running   0          19s
productpage-v1-c6885474-sz9mf   2/2     Running   0          14s
ratings-v1-b6994bb9-6xlq7       1/1     Running   0          89m
ratings-v1-b6994bb9-j2l78       1/1     Running   0          89m
ratings-v1-b6994bb9-w748t       1/1     Running   0          89m
reviews-v2-7bf8c9648f-4rh4z     1/1     Running   0          7m16s
reviews-v2-7bf8c9648f-6vhlq     1/1     Running   0          7m18s
reviews-v2-7bf8c9648f-8zjcw     1/1     Running   0          7m16s
sleep-557747455f-jxk5z          1/1     Running   0          12m
oke_admin@cloudshell:~ (ap-seoul-1)$ kubectl describe pod productpage-v1-c6885474-6g75
Name:         productpage-v1-c6885474-6g754
Namespace:    default
...
Containers:
  productpage:
    Container ID:   cri-o://95f68b2fcb5e8168879e1a5703bf520a94729664cc410edbe0d3255113344149
    Image:          docker.io/istio/examples-bookinfo-productpage-v1:1.16.2
    ...
  istio-proxy:
    Container ID:  cri-o://74f2707caa99b8b12b3e293a129f74234c5ac1ab36729f70045de640eeb2b4b4
    Image:         docker.io/istio/proxyv2:1.12.1    
    ...

  3. Access the productpage page as before.

    • http://{productpage external ip}:9080/productpage

  4. Check the istio-proxy log

    You can see that the web page is accessed as before and additionally goes through istio-proxy.

    oke_admin@cloudshell:~ (ap-seoul-1)$ kubectl logs -l app=productpage -c istio-proxy -f
...
[2021-12-20T09:16:54.826Z] "GET /details/0 HTTP/1.1" 200 - via_upstream - "-" 0 178 2 2 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" "adeb5dbb-261c-9982-9d66-523cc3890e2b" "details:9080" "10.244.2.43:9080" outbound|9080||details.default.svc.cluster.local 10.244.2.149:41122 10.96.175.211:9080 10.244.2.149:55324 - default
[2021-12-20T09:16:54.842Z] "GET /reviews/0 HTTP/1.1" 200 - via_upstream - "-" 0 379 591 591 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" "adeb5dbb-261c-9982-9d66-523cc3890e2b" "reviews:9080" "10.244.2.51:9080" outbound|9080||reviews.default.svc.cluster.local 10.244.2.149:45512 10.96.197.86:9080 10.244.2.149:35752 - default
[2021-12-20T09:16:54.820Z] "GET /productpage HTTP/1.1" 200 - via_upstream - "-" 0 5183 616 615 "10.179.87.76" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" "adeb5dbb-261c-9982-9d66-523cc3890e2b" "146.56.142.95:9080" "10.244.2.149:9080" inbound|9080|| 127.0.0.6:41867 10.244.2.149:9080 10.179.87.76:0 - default

Full Service Istio Enable

  1. Enable Istio for Review v2 and other services with the following command.

    curl -s https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/platform/kube/bookinfo.yaml | istioctl kube-inject -f - | kubectl apply -l app!=reviews -f -
curl -s https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/platform/kube/bookinfo.yaml | istioctl kube-inject -f - | kubectl apply -l app=reviews,version=v2 -f -

  2. Execution result

    If Istio-proxy is added and you see 2 containers per Pod, bookinfo.yaml is reapplied to note that replica=1.

    oke_admin@cloudshell:~ (ap-seoul-1)$ kubectl get pod
NAME                            READY   STATUS    RESTARTS   AGE
details-v1-5c6d488c49-ltw8r     2/2     Running   0          69s
productpage-v1-c6885474-6g754   2/2     Running   0          15h
ratings-v1-7dfc8d4bbf-p8n2f     2/2     Running   0          68s
reviews-v2-5ff75858c7-zhn57     2/2     Running   0          64s
sleep-557747455f-jxk5z          1/1     Running   0          16h

  3. Note

    If you label istio-injection as shown below, pods deployed in that namespace will automatically have istio sidecar added.

    kubectl label namespace <NAMESPACE_NAME> istio-injection=enabled

Set up Istio Ingress Gateway

Earlier in another chapter, we used the nginx ingress controller as the Ingress Controller. To control microservices, configure Istio’s Ingress Gateway for services.

  1. Create an Istio Ingress Gateway for the example book-info with the contents below.

    kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: bookinfo-gateway
spec:
  selector:
    istio: ingressgateway # use Istio default gateway implementation
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: bookinfo
spec:
  hosts:
  - "*"
  gateways:
  - bookinfo-gateway
  http:
  - match:
    - uri:
        exact: /productpage
    - uri:
        exact: /login
    - uri:
        exact: /logout
    - uri:
        prefix: /static
    route:
    - destination:
        host: productpage
        port:
          number: 9080
EOF

  2. Check the IP for access through the Istio Ingress Gateway.

    oke_admin@cloudshell:~ (ap-seoul-1)$ kubectl get svc -n istio-system
NAME                   TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)                                                                      AGE
istio-egressgateway    ClusterIP      10.96.59.144   <none>          80/TCP,443/TCP                                                               66m
istio-ingressgateway   LoadBalancer   10.96.150.51   146.56.186.52   15021:32250/TCP,80:32707/TCP,443:32572/TCP,31400:32532/TCP,15443:31522/TCP   66m
istiod                 ClusterIP      10.96.188.42   <none>          15010/TCP,15012/TCP,443/TCP,15014/TCP                                        66m

  3. Check the connection

    • Example, http://146.56.186.52:80/productpage

    image-20211221112639038

Metric monitoring

  1. Install Prometheus & Grafana

    kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.12/samples/addons/prometheus.yaml
kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.12/samples/addons/grafana.yaml

  2. External access to the installed Grafana dashboard

    • Option #1:
      • You can connect through a local proxy with the istioctl dashboard grafana command. Cloud Shell does not allow public IP and external access, so if external access is required, relevant settings are required.
    • Option #2:

  3. Set up external access by referring to the official documentation.

  4. Example of external connection setting

    • Domain settings for settings

      • Check the Istio Ingress Gateway’s External IP and set it in the DNS server or in the client’s /etc/hosts file. Enter the domain address you want to use.

        export INGRESS_DOMAIN="istio.thekoguryo.ml"

    • Open external connection

    cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: grafana-gateway
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http-grafana
      protocol: HTTP
    hosts:
    - "grafana.${INGRESS_DOMAIN}"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: grafana-vs
  namespace: istio-system
spec:
  hosts:
  - "grafana.${INGRESS_DOMAIN}"
  gateways:
  - grafana-gateway
  http:
  - route:
    - destination:
        host: grafana
        port:
          number: 3000
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: grafana
  namespace: istio-system
spec:
  host: grafana
  trafficPolicy:
    tls:
      mode: DISABLE
---
EOF

  5. Test data generation

    • Generate test data by connecting through Istio Ingress Gateway of productpage.
      • Example test app page, http://146.56.186.52:80/productpage

  6. Access Grafana

    image-20211221123432063

    • You can check the processing information of each service by looking at the Grafana dashboard.

    image-20211221123914692

Distributed Tracing

A microservice processes a client’s request through a decentralized service and then responds to the client. The Bookinfo app used as an example also provides the client with information collected by the productpage service from the backend service. In such a distributed environment, there are various tools for tracing call information between services for a client’s request. Istio leverages Envoy’s distributed tracing capabilities to leverage tools such as Jaeger, Zipkin, and LightStep. Here we will see how to do it via Zipkin. For other tools, please refer to the related page.

Zipkin

  1. Install Zipkin

    kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.12/samples/addons/extras/zipkin.yaml

  2. Access the installed Zipkin page from outside

  3. Example of external connection setting

    • Domain settings for settings

      export INGRESS_DOMAIN="istio.thekoguryo.ml"

    • Open external connection

    cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: tracing-gateway
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http-tracing
      protocol: HTTP
    hosts:
    - "tracing.${INGRESS_DOMAIN}"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: tracing-vs
  namespace: istio-system
spec:
  hosts:
  - "tracing.${INGRESS_DOMAIN}"
  gateways:
  - tracing-gateway
  http:
  - route:
    - destination:
        host: tracing
        port:
          number: 80
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: tracing
  namespace: istio-system
spec:
  host: tracing
  trafficPolicy:
    tls:
      mode: DISABLE
---
EOF

  4. Test data generation

    • Generate test data by connecting through Istio Ingress Gateway of productpage.
      • Example test app page, http://146.56.186.52:80/productpage

  5. Access the Zipkin page.

  6. Service trace test

    • On the Find a trace screen, click the red plus sign to set the service to search and run the query.

    image-20211221132104691

    • Check the search results and click SHOW on the right for the item you want to track.

      image-20211221133039612

    • You can see the call relationship between services for the request, the time, and tag information for each service.

      image-20211221133331040

  7. Trace by Trace ID

    • For each request, it can be traced through Span ID (Trace ID). You can check it in the log.

      image-20211221132634370

    • If you search directly through the Trace ID search box at the top right of the screen, you can check the request immediately.

      image-20211221134349660

service mesh visualization

A microservice processes a client’s request through a decentralized service and then responds to the client. The Bookinfo app used as an example also provides the client with information collected by the productpage service from the backend service. In such a distributed environment, Kiali provides visualization of call information between services for client requests.

  1. Install Kiali

    kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.12/samples/addons/kiali.yaml

  2. External access to installed Kiali page

  3. Example of external connection setting

    • Domain settings for settings

      export INGRESS_DOMAIN="istio.thekoguryo.ml"

    • Open external connection

    cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: kiali-gateway
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http-kiali
      protocol: HTTP
    hosts:
    - "kiali.${INGRESS_DOMAIN}"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: kiali-vs
  namespace: istio-system
spec:
  hosts:
  - "kiali.${INGRESS_DOMAIN}"
  gateways:
  - kiali-gateway
  http:
  - route:
    - destination:
        host: kiali
        port:
          number: 20001
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: kiali
  namespace: istio-system
spec:
  host: kiali
  trafficPolicy:
    tls:
      mode: DISABLE
---
EOF

  4. Test data generation

    • Generate test data by connecting through Istio Ingress Gateway of productpage.
      • Example test app page, http://146.56.186.52:80/productpage

  5. Access Kiali Dashboard.

  6. If you click Graph in the navigation menu, you can visualize the call information between services.

    image-20211221141550099

Reviews New version release

Reviews v3 distribution

  1. Deploy the Reviews v3 version.

    Deploy v3 assuming version update with Istio enabled.

    curl -s https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/platform/kube/bookinfo.yaml | istioctl kube-inject -f - | kubectl apply -l app=reviews,version=v3 -f -

  2. Check the deployment result

    oke_admin@cloudshell:~ (ap-seoul-1)$ kubectl get pod
NAME                            READY   STATUS    RESTARTS   AGE
...
reviews-v2-5ff75858c7-hwrgv     2/2     Running   0          5h15m
reviews-v3-5484f6cbd6-685b6     2/2     Running   0          3m8s
...

  3. Test by connecting through Istio Ingress Gateway in productpage.

    • Example test app page, http://146.56.186.52:80/productpage
    • Reviews v2 and Reviews v3 are routed once, as each Pod is not set according to the distribution rule. You can see that Reviews v2 (black star) and Reviews v3 (red star) are displayed at once.

    image-20211221161551911

Weighted Routing

Let’s look at a scenario where only some requests based on weight are routed for validation before the new version is fully serviced.

  1. Check the version label of the deployed Reviews Pod. Version=v2 and version=v2 are assigned respectively.

    oke_admin@cloudshell:~ (ap-seoul-1)$ kubectl get pod -L version
NAME                            READY   STATUS    RESTARTS   AGE     VERSION
details-v1-5c6d488c49-m5t7j     2/2     Running   0          5h28m   v1
productpage-v1-c6885474-qkpx9   2/2     Running   0          5h28m   v1
ratings-v1-7dfc8d4bbf-97p6z     2/2     Running   0          5h28m   v1
reviews-v2-5ff75858c7-hwrgv     2/2     Running   0          5h28m   v2
reviews-v3-5484f6cbd6-685b6     2/2     Running   0          15m     v3
sleep-557747455f-jxk5z          1/1     Running   0          22h

  2. Deploy the below settings required by Istio for routing by label.

    • DestinationRule: Filter by labels for service endpoints. In the test example, only the Pod, not the Service Type, has a version label. It seems to be the label of the pod
    • VirtualService: Distributed based on weight for the defined destination. The sum of the weights should be 100. 90 times out of 100 times, 10 times, it is guessed by probability, not by exact distribution.
    kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
    - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v2
      weight: 90
    - destination:
        host: reviews
        subset: v3
      weight: 10
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: reviews
spec:
  host: reviews   
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2
  - name: v3
    labels:
      version: v3
EOF

  3. Repeat the test as shown below.

    counter=1; \
while [ $counter -lt 20 ]; \
do
   curl -s http://146.56.186.52/productpage | grep color | head -1;
   counter=$(( $counter + 1 )); \
done

  4. Test results

    You can see that 18 out of 20 are black asterisks and 2 red asterisks. Chances are it won’t be accurate.

    oke_admin@cloudshell:~ (ap-seoul-1)$ counter=1; \
> while [ $counter -lt 20 ]; \
> do
>    curl -s http://146.56.186.52/productpage | grep color | head -1;
>    counter=$(( $counter + 1 )); \
> done
        <font color="black">
        <font color="black">
        <font color="black">
        <font color="red">
        <font color="black">
        <font color="black">
        <font color="black">
        <font color="black">
        <font color="black">
        <font color="black">
        <font color="black">
        <font color="black">
        <font color="black">
        <font color="black">
        <font color="black">
        <font color="black">
        <font color="red">
        <font color="black">
        <font color="black">

  5. Check the call flow to Kiali

    As you can see in the picture below, you can see that it is distributed according to the weights set in Reviews v2 and v3.

    image-20211221171751998



As an individual, this article was written with my personal time. There may be errors in the content of the article, and the opinions in the article are personal opinions.

Last updated on 21 Dec 2021