6.4 Deploying Applications to Multicluster

Deploy the Helidon sample application

We have seen earlier that Java applications developed using the Helidon framework are deployed in Verrazzano using the Open Application Model. I’ve verified that I’m deploying against a single Kubernetes cluster.

This time, let’s check the deployment in a multi-cluster environment. Verrazzano is deployed against three clusters.

• Admin Cluster
  • admin-cluster
• Managed Cluster
  • managed-cluster-1
  • managed-cluster-2

Application Deployment

1. Create namespace

   • Based on single cluster (reference)

     In the previous example, you had to create a namespace to deploy and set a label in advance.

     kubectl create namespace hello-helidon
     kubectl label namespace hello-helidon verrazzano-managed=true istio-injection=enabled
     

   • Create a namespace for multi-cluster (i.e. create a project)

     • VerrazzanoProject type is provided to manage namespaces across multi-clusters where applications are deployed. Executing the following command on the Admin Cluster will create a corresponding namespace in the Admin Cluster and the target Managed Cluster, which will be managed by verrazzano.
     • Deploy the settings below to admin-cluster.

   apiVersion: clusters.verrazzano.io/v1alpha1
   kind: VerrazzanoProject
   metadata:
     name: hello-helidon
     namespace: verrazzano-mc
   spec:
     template:
       namespaces:
         - metadata:
             name: hello-helidon
     placement:
       clusters:
         - name: managed-cluster-1
         - name: managed-cluster-2      
   

   • Check the result

     When you create a VerrazzanoProject, a namespace is created for the target clusters as shown below and a label is automatically assigned.

   [opc@bastion-host ~ ( )]$ kubectl config use-context admin-cluster
   Switched to context "admin-cluster".
   [opc@bastion-host ~ (⎈ |admin-cluster:default)]$ kubectl get ns hello-helidon --show-labels
   NAME            STATUS   AGE     LABELS
   hello-helidon   Active   4m50s   istio-injection=enabled,verrazzano-managed=true
   [opc@bastion-host ~ (⎈ |admin-cluster:default)]$ kubectl config use-context managed-cluster-1
   Switched to context "managed-cluster-1".
   [opc@bastion-host ~ (⎈ |managed-cluster-1:default)]$ kubectl get ns hello-helidon --show-labels
   NAME            STATUS   AGE     LABELS
   hello-helidon   Active   4m50s   istio-injection=enabled,verrazzano-managed=true
   [opc@bastion-host ~ (⎈ |managed-cluster-1:default)]$ kubectl config use-context managed-cluster-2
   Switched to context "managed-cluster-2".
   [opc@bastion-host ~ (⎈ |managed-cluster-2:default)]$ kubectl get ns hello-helidon --show-labels
   NAME            STATUS   AGE    LABELS
   hello-helidon   Active   5m3s   istio-injection=enabled,verrazzano-managed=true
   

2. Component distribution

   There is no difference between single cluster and multi cluster. The config file is the same as before, deploying to admin-cluster instead.

   kubectl apply -f https://raw.githubusercontent.com/verrazzano/verrazzano/v1.1.0/examples/hello-helidon/hello-helidon-comp.yaml
   

3. Application Configuration Deployment

   • YAML example for multi-cluster deployment

     Multi-cluster deployments use the MultiClusterApplicationConfiguration type. You can see that the part defined in the existing ApplicationConfiguration has gone into the sub-spec of MultiClusterApplicationConfiguration. Additionally, placement syntax for multi-cluster deployments is added.

     apiVersion: clusters.verrazzano.io/v1alpha1
     kind: MultiClusterApplicationConfiguration
     metadata:
       name: hello-helidon-appconf
       namespace: hello-helidon
     spec:
       template:
         metadata:
           annotations:
             version: v1.0.0
             description: "Hello Helidon application"
         spec:
           components:
             - componentName: hello-helidon-component
               traits:
                 ...
     
       placement:
         clusters:
           - name: managed-cluster-1
     

   • Modify and deploy Application Configuration for multi-cluster deployment according to each placement.

     wget https://raw.githubusercontent.com/verrazzano/verrazzano/v1.1.0/examples/multicluster/hello-helidon/mc-hello-helidon-app.yaml
     # Modify placement
     kubectl apply -f mc-hello-helidon-app.yaml
     

     • If necessary, compare directly with the file below used in the previous single cluster deployment.

       https://raw.githubusercontent.com/verrazzano/verrazzano/v1.1.0/examples/hello-helidon/hello-helidon-app.yaml
       

4. Distribution result

   When deploying components and applications to admin-cluster, you can see that they are deployed to the cluster designated as placement, as shown in the result below.

   [opc@bastion-host ~ (⎈ |admin-cluster:default)]$ kubectl apply -f https://raw.githubusercontent.com/verrazzano/verrazzano/v1.1.0/examples/hello-helidon/hello-helidon-comp.yaml
   component.core.oam.dev/hello-helidon-component created
   [opc@bastion-host ~ (⎈ |admin-cluster:default)]$ kubectl apply -f mc-hello-helidon-app.yaml
   multiclusterapplicationconfiguration.clusters.verrazzano.io/hello-helidon-appconf created
   [opc@bastion-host ~ (⎈ |admin-cluster:default)]$ kubectl get multiclusterapplicationconfigurations,component,deployment -n hello-helidon
   NAME                                                                                AGE
   multiclusterapplicationconfiguration.clusters.verrazzano.io/hello-helidon-appconf   11m
   
   NAME                                             WORKLOAD-KIND               AGE
   component.core.oam.dev/hello-helidon-component   VerrazzanoHelidonWorkload   17m
   
   [opc@bastion-host ~ (⎈ |admin-cluster:default)]$ kubectl config use-context managed-cluster-1
   Switched to context "managed-cluster-1".
   [opc@bastion-host ~ (⎈ |managed-cluster-1:default)]$ kubectl get multiclusterapplicationconfigurations,component,deployment -n hello-helidon
   NAME                                                                                AGE
   multiclusterapplicationconfiguration.clusters.verrazzano.io/hello-helidon-appconf   11m
   
   NAME                                             WORKLOAD-KIND               AGE
   component.core.oam.dev/hello-helidon-component   VerrazzanoHelidonWorkload   11m
   
   NAME                                       READY   UP-TO-DATE   AVAILABLE   AGE
   deployment.apps/hello-helidon-deployment   1/1     1            1           11m
   
   [opc@bastion-host ~ (⎈ |managed-cluster-1:default)]$ kubectl config use-context managed-cluster-2
   Switched to context "managed-cluster-2".
   [opc@bastion-host ~ (⎈ |managed-cluster-2:default)]$ kubectl get multiclusterapplicationconfigurations,component,deployment -n hello-helidon
   No resources found in hello-helidon namespace.
   

5. Check the registered ingress DNS through istio of the deployed app

   kubectl get gateway hello-helidon-hello-helidon-appconf-gw \
       -n hello-helidon \
       -o jsonpath='{.spec.servers[0].hosts[0]}'
   

6. Execution example

   $ kubectl get gateway hello-helidon-hello-helidon-appconf-gw \
   >     -n hello-helidon \
   >     -o jsonpath='{.spec.servers[0].hosts[0]}'
   hello-helidon-appconf.hello-helidon.mc-1.thekoguryo.ml
   

7. Application testing

   You can see that it is called normally with the resolved https://{ingress DNS address}/greet address.

   

monitoring

Verrazzano console

1. Log in to the Verrazzano Console.

   Example) https://verrazzano.myenv.thekoguryo.ml

   • Accessed user: verrazzano
   • Password: Password initialized during installation

2. In the Resources section on the left, you can check the deployed app information in the Application and Component sections.

   

Log Monitoring (Elasticsearch / Kibana)

1. Click the Kibana link in the Verrazzano console. SSO is configured, so additional login is not required.

2. Create an index pattern with the verrazzano-namespace-hello-helidon namespace in the same way as for a single cluster and deployment.

3. You can see logs collected for the created index pattern and all logs collected on the target namespace. admin-cluster collects logs from all registered Managed Clusters. Since the previously deployed app was only deployed on managed-cluster-1, you can see that the actual collected logs are also the same.

   

Monitoring metrics (Prometheus / Grafana)

1. Click the Grafana link in the Verrazzano console. SSO is configured, so additional login is not required.

2. Select the Helidon dashboard among the default dashboards.

3. You can check the status of previously deployed apps on the Helidon dashboard. If you can see the metrics collected on all clusters, it will look based on the data collected because it is on the actual managed-cluster-1.

   

Change the deployment of the Helidon example application

Change the application deployment location

1. The cluster location can be changed through the placement item. Create a placement file to change.

   # patch.yaml
   spec:
     placement:
       clusters:
         - name: managed-cluster-2
   

2. Change the deployment location

   kubectl patch MultiClusterApplicationConfiguration hello-helidon-appconf \
       -n hello-helidon \
       --type merge \
       --patch "$(cat patch.yaml)"
   

3. Check distribution result

   You can see that the app previously deployed to managed-cluster-1 has been relocated to managed-cluster-2.

   [opc@bastion-host ~ (⎈ |admin-cluster:default)]$ kubectl patch MultiClusterApplicationConfiguration hello-helidon-appconf \
   >     -n hello-helidon \
   >     --type merge \
   >     --patch "$(cat patch.yaml)"
   multiclusterapplicationconfiguration.clusters.verrazzano.io/hello-helidon-appconf patched
   [opc@bastion-host ~ (⎈ |admin-cluster:default)]$ kubectl config use-context managed-cluster-1
   Switched to context "managed-cluster-1".
   [opc@bastion-host ~ (⎈ |managed-cluster-1:default)]$ kubectl get multiclusterapplicationconfigurations,component,deployment -n hello-helidon
   No resources found in hello-helidon namespace.
   [opc@bastion-host ~ (⎈ |managed-cluster-1:default)]$ kubectl config use-context managed-cluster-2
   Switched to context "managed-cluster-2".
   [opc@bastion-host ~ (⎈ |managed-cluster-2:default)]$ kubectl get multiclusterapplicationconfigurations,component,deployment -n hello-helidon
   NAME                                                                                AGE
   multiclusterapplicationconfiguration.clusters.verrazzano.io/hello-helidon-appconf   3m25s
   
   NAME                                             WORKLOAD-KIND               AGE
   component.core.oam.dev/hello-helidon-component   VerrazzanoHelidonWorkload   2m26s
   
   NAME                                       READY   UP-TO-DATE   AVAILABLE   AGE
   deployment.apps/hello-helidon-deployment   1/1     1            1           2m22s