TheKoguryo's Tech Blog

Version 2023.07.06

Warning

This content has been generated by machine translation. The translations are automated and have not undergone human review or validation.

5.4 Using Service Operator for OCI resource management in OKE

OCI Service Operator for Kubernetes (OSOK) is a tool that helps you manage OCI resources through the Kubernetes API. You can understand the Autonomous Database service as allowing you to create, delete, etc. instances through the Kubernetes API, kubectl. It is built on the open source Operator Framework used by Kubernetes. Related reference sites are as follows.

The OCI services currently supported as of v1.1.0 are as follows.

  • Autonomous Database service
  • MySQL Database service
  • Streaming service
  • Service Mesh service

Install OCI Service Operator for Kubernetes on OKE Cluster

It is installed according to the product installation document. For details, refer to the document below.

Install Operator SDK

Install according to the official installation documentation.

Example of an installation command based on Cloud Shell

  • Check if operator-sdk cli operates normally by installing it with the command below.

    echo Download the release binary
export ARCH=$(case $(uname -m) in x86_64) echo -n amd64 ;; aarch64) echo -n arm64 ;; *) echo -n $(uname -m) ;; esac)
export OS=$(uname | awk '{print tolower($0)}')
export OPERATOR_SDK_DL_URL=https://github.com/operator-framework/operator-sdk/releases/download/v1.19.1
curl -LO ${OPERATOR_SDK_DL_URL}/operator-sdk_${OS}_${ARCH}

echo Verify the downloaded binary
gpg --keyserver keyserver.ubuntu.com --recv-keys 052996E2A20B5C7E
curl -LO ${OPERATOR_SDK_DL_URL}/checksums.txt
curl -LO ${OPERATOR_SDK_DL_URL}/checksums.txt.asc
gpg -u "Operator SDK (release) <cncf-operator-sdk@cncf.io>" --verify checksums.txt.asc
grep operator-sdk_${OS}_${ARCH} checksums.txt | sha256sum -c -

echo Install the release binary in your PATH
mkdir -p ~/.local/bin
chmod +x operator-sdk_${OS}_${ARCH} && mv operator-sdk_${OS}_${ARCH} ~/.local/bin/operator-sdk

operator-sdk version

Install Operator Lifecycle Manager (OLM)

Install and check the OLM resource in the current OKE cluster with the command below.

operator-sdk olm install --version 0.20.0
operator-sdk olm status

Install OCI Service Operator for Kubernetes

Set Permissions

OCI Service Operator for Kubernetes requires permission to work with OCI services. If OSOK is installed in OKE inside OCI, you can grant permission through Instance Principal. If OSOK is installed in an environment outside of OCI, or if you want to grant permissions to a user group that OSOK will use, you can grant permissions via User Principal. Normally we’ll set the permissions in OKE, so we’ll use the Instance Principal here.

Instance Principal
Create Dynamic Group for OKE Worker Node
  1. Log in to the OCI console and go to Identity & Security > Identity > Compartments and copy the OCID of the Compartment where the OKE cluster is located.
  2. Go to the Dynamic Group menu on the left and create a Dynamic Group with the following rules using the copied OCID.
    • Name: ex) oke-labs-dynamic-group
    • Rule: instance.compartment.id = '<compartment-ocid>'
Create Policy for Dynamic Group

  1. Go to the Policies menu on the left and grant the following rights to the created Dynamic Group.

    Allow dynamic-group <DYNAMICGROUP_NAME> to manage <OCI_SERVICE_1> in compartment <COMPARTMENT_NAME>
...

  2. Examples

    • Policy Name: oke-labs-osok-policy

    • COMPARTMENT_NAME: oke-labs

    • DYNAMICGROUP_NAME: oke-labs-dynamic-group

    • Autonomous Database Service

      Allow dynamic-group oke-labs-dynamic-group to manage autonomous-database-family in compartment oke-labs

    • MySQL DB System Service

      Allow dynamic-group oke-labs-dynamic-group to manage mysql-family in compartment oke-labs
Allow dynamic-group oke-labs-dynamic-group to {SUBNET_READ, SUBNET_ATTACH, SUBNET_DETACH, VCN_READ, COMPARTMENT_INSPECT} in compartment oke-labs
Allow dynamic-group oke-labs-dynamic-group to use tag-namespaces in compartment oke-labs

    • Service Mesh Service

      Allow dynamic-group <your-dynamic-group-name> to manage service-mesh-family in compartment <your-compartment-name>
Allow dynamic-group <your-dynamic-group-name> to use metrics in compartment <your-compartment-name>
Allow dynamic-group <your-dynamic-group-name> to use log-content in compartment <your-compartment-name>
Allow dynamic-group <your-dynamic-group-name> to use tag-namespaces in compartment <your-compartment-name>
Allow dynamic-group <your-dynamic-group-name> to manage leaf-certificates in compartment <your-compartment-name>
Allow dynamic-group <your-dynamic-group-name> to manage leaf-certificate-family in compartment <your-compartment-name>
Allow dynamic-group <your-dynamic-group-name> to use certificate-authority-delegate in compartment <your-compartment-name>
Allow dynamic-group <your-dynamic-group-name> to use certificate-authority-family in compartment <your-compartment-name>
Allow dynamic-group <your-dynamic-group-name> to inspect vault in compartment <your-compartment-name>
Allow dynamic-group <your-dynamic-group-name> to {MESH_PROXY_DETAILS_READ} in tenancy
Allow dynamic-group <your-dynamic-group-name> to manage certificate-associations in compartment <your-compartment-name>
Allow dynamic-group <your-dynamic-group-name> to manage certificate-authority-associations in compartment <your-compartment-name>
Allow any-user to manage keys in compartment <your-compartment-name> where any {request.principal.type='certificate', request.principal.type='certificateauthority'}
Allow any-user to manage object-family in compartment <your-compartment-name> where any {request.principal.type='certificate', request.principal.type='certificateauthority'}
Enable User Principal

OCI Service Operator for Kubernetes (OSOK) deployment

  1. Create a namespace where OSOK will be deployed.

    kubectl create ns oci-service-operator-system

  2. OSOK Operator installation

    docker pull iad.ocir.io/oracle/oci-service-operator-bundle:1.1.0
operator-sdk run bundle iad.ocir.io/oracle/oci-service-operator-bundle:1.1.0 -n oci-service-operator-system --timeout 5m

  3. After OSOK installation, you can see that CustomResource for OCI service has been added.

    winter@cloudshell:~ (ap-chuncheon-1)$ kubectl api-resources | grep oci.oracle.com
autonomousdatabases                            oci.oracle.com/v1beta1                 true         AutonomousDatabases
mysqldbsystems                                 oci.oracle.com/v1beta1                 true         MySqlDbSystem
streams                                        oci.oracle.com/v1beta1                 true         Stream
accesspolicies                                 servicemesh.oci.oracle.com/v1beta1     true         AccessPolicy
ingressgatewaydeployments                      servicemesh.oci.oracle.com/v1beta1     true         IngressGatewayDeployment
ingressgatewayroutetables                      servicemesh.oci.oracle.com/v1beta1     true         IngressGatewayRouteTable
ingressgateways                                servicemesh.oci.oracle.com/v1beta1     true         IngressGateway
meshes                                         servicemesh.oci.oracle.com/v1beta1     true         Mesh
virtualdeploymentbindings                      servicemesh.oci.oracle.com/v1beta1     true         VirtualDeploymentBinding
virtualdeployments                             servicemesh.oci.oracle.com/v1beta1     true         VirtualDeployment
virtualserviceroutetables                      servicemesh.oci.oracle.com/v1beta1     true         VirtualServiceRouteTable
virtualservices                                servicemesh.oci.oracle.com/v1beta1     true         VirtualService

Manage Autonomous Database Service with OSOK

References

Autonomous Database (ADB) Binding

This is a case of binding the ADB instance created in the OCI console as a resource in Kubernetes.

  1. Log in to the OCI console and create an instance of Autonomous Database to bind to.

  2. Check and write the required specification yaml for binding.

    • spec.id: Enter the OCID of the existing ADB to bind to.
    • walletName: Enter the name of the kubernetes secret where the wallet will be stored after binding
    • walletPassword.secret.secretName: The secret name where the password to be used for the wallet is stored, and a secret is created in advance before binding.
      • The wallet download password should contain at least 1 number or special character
    apiVersion: oci.oracle.com/v1beta1
kind: AutonomousDatabases
metadata:
  name: <CR_OBJECT_NAME>
spec:
  id: <AUTONOMOUS_DATABASE_OCID>
  wallet:
    walletName: <WALLET_SECRET_NAME>
    walletPassword:
      secret:
        secretName: <WALLET_PASSWORD_SECRET_NAME>

  3. Execution example

    • Create walletPassword

      • The wallet download password should contain at least 1 number or special character
      kubectl create secret generic ociadb-wallet-password-secret --from-literal=walletPassword='xxxxxxxxxxxx'

    • Execution

      cat <<EOF > autonomousdatabases-bind.yaml
apiVersion: oci.oracle.com/v1beta1
kind: AutonomousDatabases
metadata:
  name: ociadb
spec:
  id: ocid1.autonomousdatabase.oc1.ap-chuncheon-1.an4w4ljrbgdxvpqamlmtqcttftib5w724cu4yfcsfzxru3prohr65637rlbq
  wallet:
    walletName: ociadb-wallet-secret
    walletPassword:
      secret:
        secretName: ociadb-wallet-password-secret
EOF
kubectl apply -f autonomousdatabases-bind.yaml

  4. Check the results

    Check if the binding was successful without any errors using the kubectl describe command.

    winter@cloudshell:~ (ap-chuncheon-1)$ kubectl get autonomousdatabases
NAME     DBWORKLOAD   STATUS   AGE
ociadb                Active   15s
winter@cloudshell:~ (ap-chuncheon-1)$ kubectl describe autonomousdatabases ociadb
Name:         ociadb
...
Kind:         AutonomousDatabases
...
Status:
  Status:
    Conditions:
      Last Transition Time:  2022-04-28T05:00:57Z
      Message:               AutonomousDatabase Bound success
...
Events:
  Type    Reason   Age                From                 Message
  ----    ------   ----               ----                 -------
  Normal  Success  27s                AutonomousDatabases  Finalizer is added to the object
  Normal  Success  25s (x2 over 25s)  AutonomousDatabases  Create or Update of resource succeeded

  5. Check your wallet

    As a result of binding, the ociadb-wallet-secret secret is created, and if you look at the contents, you can see that the files in the wallet are in Base64 encoded form. You can mount the secret in the application container and use it when connecting to ADB.

    winter@cloudshell:~ (ap-chuncheon-1)$ kubectl get secret
NAME                              TYPE                                  DATA   AGE
...
ociadb-wallet-password-secret   Opaque                                1      4m27s
ociadb-wallet-secret            Opaque                                8      2m8
winter@cloudshell:~ (ap-chuncheon-1)$ kubectl get secret ociadb-wallet-secret -o yaml
apiVersion: v1
data:
  README: V2FsbGV0IE...
  cwallet.sso: ofhONgAAAA...
  ewallet.p12: MIIZ/AIBAz...
  keystore.jks: /u3+7QAAA...
  ojdbc.properties: IyBDb2...
  sqlnet.ora: V0FMTE...
  tnsnames.ora: b2NpYWR...
  truststore.jks: /u3+7QAAAA...
kind: Secret
metadata:
...
  name: ociadb-wallet-secret
...
type: Opaque

Autonomous Database (ADB) Provisioning

  1. Check and fill out the required specification yaml for Provisioning.

    • spec.compartmentId: Enter the OCID of the compartment where the ADB to be created will be located
    • walletName: Enter the name of the kubernetes secret where the wallet will be stored after binding
    • walletPassword.secret.secretName: The secret name where the password to be used for the wallet is stored, and a secret is created in advance before binding.
    • For the rest of the items, enter the desired values the same as those entered when creating ADB in the OCI console - [Refer to item specification] (https://github.com/oracle/oci-service-operator/blob/main/docs/adb.md #autonomous-database-specification-parameters)
    apiVersion: oci.oracle.com/v1beta1
kind: AutonomousDatabases
metadata:
  name: <CR_OBJECT_NAME>
spec:
  compartmentId: <COMPARTMENT_OCID>
  displayName: <DISPLAY_NAME>
  dbName: <DB_NAME>
  dbWorkload: <OLTP/DW>
  isDedicated: <false/true>
  dbVersion: <ORABLE_DB_VERSION>
  dataStorageSizeInTBs: <SIZE_IN_TBs>
  cpuCoreCount: <COUNT>
  adminPassword:
    secret:
      secretName: <ADMIN_PASSWORD_SECRET_NAME>
  isAutoScalingEnabled: <true/false>
  isFreeTier: <false/true>
  licenseModel: <BRING_YOUR_OWN_LICENSE/LICENSE_INCLUDEE>
  wallet:
    walletName: <WALLET_SECRET_NAME>
    walletPassword:
      secret:
        secretName: <WALLET_PASSWORD_SECRET_NAME>
  freeformTags:
    <KEY1>: <VALUE1>
  definedTags:
    <TAGNAMESPACE1>:
      <KEY1>: <VALUE1>

  2. Execution example

    • Create adminPassword and walletPassword

    • The wallet download password should contain at least 1 number or special character

    kubectl create secret generic ociadb-by-osok-admin-password-secret --from-literal=password='xxxxxxxxxxxx'
kubectl create secret generic ociadb-by-osok-wallet-password-secret --from-literal=walletPassword='xxxxxxxxxxxx'

    • Execution

      cat <<EOF > autonomousdatabases-provision.yaml
apiVersion: oci.oracle.com/v1beta1
kind: AutonomousDatabases
metadata:
  name: ociadbbyosok
spec:
  compartmentId: ocid1.compartment.oc1..aaaaaaaaa2jcbfqjyz24y4hbbqurdxjegmsp6eqhzq4r2gni5bocoh2axb4a
  displayName: OCIADBbyOSOK
  dbName: ociadbbyosok
  dbWorkload: OLTP
  isDedicated: false
  dbVersion: 19c
  dataStorageSizeInTBs: 1
  cpuCoreCount: 1
  adminPassword:
    secret:
      secretName: ociadb-by-osok-admin-password-secret
  isAutoScalingEnabled: false
  isFreeTier: false
  licenseModel: LICENSE_INCLUDED
  wallet:
    walletName: ociadb-by-osok-wallet-secret
    walletPassword:
      secret:
        secretName: ociadb-by-osok-wallet-password-secret
EOF
kubectl apply -f autonomousdatabases-provision.yaml

  3. Result

    Check if the binding was successful without any errors using the kubectl describe command.

    winter@cloudshell:~ (ap-chuncheon-1)$ kubectl get autonomousdatabases
NAME           DBWORKLOAD   STATUS   AGE
...
ociadbbyosok   OLTP         Active   3m4s
winter@cloudshell:~ (ap-chuncheon-1)$ kubectl describe autonomousdatabases ociadbbyosok
Name:         ociadbbyosok
...
Kind:         AutonomousDatabases
...
Status:
  Status:
    Conditions:
      Last Transition Time:  2022-04-28T05:08:06Z
      Message:               AutonomousDatabase Provisioning
      Status:                True
...
Events:
  Type    Reason   Age                From                 Message
  ----    ------   ----               ----                 -------
  Normal  Success  3m10s              AutonomousDatabases  Finalizer is added to the object
  Normal  Success  58s (x2 over 58s)  AutonomousDatabases  Create or Update of resource succeeded

  4. Check your wallet

    As a result of binding, the ociadb-wallet-secret secret is created, and if you look at the contents, you can see that the files in the wallet are in Base64 encoded form. You can mount the secret in the application container and use it when connecting to ADB.

    winter@cloudshell:~ (ap-chuncheon-1)$ kubectl get secret
NAME                                    TYPE                                  DATA   AGE
...
ociadb-by-osok-admin-password-secret    Opaque                                1      7m22s
ociadb-by-osok-wallet-password-secret   Opaque                                1      7m11s
ociadb-by-osok-wallet-secret            Opaque                                8      3m13s
winter@cloudshell:~ (ap-chuncheon-1)$ kubectl get secret ociadb-by-osok-wallet-secret -o yaml
apiVersion: v1
data:
  README: V2FsbGV0IE...
  cwallet.sso: ofhONgAAAA...
  ewallet.p12: MIIZ/AIBAz...
  keystore.jks: /u3+7QAAA...
  ojdbc.properties: IyBDb2...
  sqlnet.ora: V0FMTE...
  tnsnames.ora: b2NpYWR...
  truststore.jks: /u3+7QAAAA...
kind: Secret
metadata:
...
  name: ociadb-by-osok-wallet-secret
...
type: Opaque

Autonomous Database (ADB) Update

Refer to the example of OSOK GitHub document in Update support for Autonomous Database provided by OCI API.

  • Oracle Cloud Infrastructure API Reference and Endpoints / UpdateAutonomousDatabaseDetails Reference

  • Based on GitHub documentation

    apiVersion: oci.oracle.com/v1beta1
kind: AutonomousDatabases
metadata:
  name: <CR_OBJECT_NAME>
spec:
  id: <AUTONOMOUS_DATABASE_OCID>
  displayName: <DISPLAY_NAME>
  dbName: <DB_NAME>
  dbWorkload: <OLTP/DW>
  isDedicated: <false/true>
  dbVersion: <ORABLE_DB_VERSION>
  dataStorageSizeInTBs: <SIZE_IN_TBs>
  cpuCoreCount: <COUNT>
  adminPassword:
    secret:
      secretName: <ADMIN_PASSWORD_SECRET_NAME>
  isAutoScalingEnabled: <true/false>
  isFreeTier: <false/true>
  licenseModel: <BRING_YOUR_OWN_LICENSE/LICENSE_INCLUDEE>
  wallet:
    walletName: <WALLET_SECRET_NAME>
    walletPassword:
      secret:
        secretName: <WALLET_PASSWORD_SECRET_NAME>
  freeformTags:
    <KEY1>: <VALUE1>
  definedTags:
    <TAGNAMESPACE1>:
      <KEY1>: <VALUE1>

Binding Case

Add updates to existing YAML or deployed YAML to reflect them.

  • Example of storage increase

    Deploy by adding dataStorageSizeInTBs entry to the above autonomousdatabases-bind.yaml file.

    apiVersion: oci.oracle.com/v1beta1
kind: AutonomousDatabases
metadata:
  name: ociadb
spec:
  id: ocid1.autonomousdatabase.oc1.ap-chuncheon-1.an4w4ljrbgdxvpqamlmtqcttftib5w724cu4yfcsfzxru3prohr65637rlbq
  wallet:
    walletName: ociadb-wallet-secret
    walletPassword:
      secret:
        secretName: ociadb-wallet-password-secret
  dataStorageSizeInTBs: 2

Provisioning Case

  • Example of storage increase

    Add the OCID of ADB created in the above autonomousdatabases-provision.yaml file to spec.id. And change the dataStorageSizeInTBs value.

    apiVersion: oci.oracle.com/v1beta1
kind: AutonomousDatabases
metadata:
  name: ociadbbyosok
spec:
  id: ocid1.autonomousdatabase.oc1.ap-chuncheon-1.an4w4ljrbgdxvpqa4ix636vselpdrig6i2peeczd2fgkevel34bph6feb46a
  compartmentId: ocid1.compartment.oc1..aaaaaaaafz2jdbc6csuogmfpmpce7t3cianevguuzbtjgaitqxf7yosvzkca
  displayName: OCIADBbyOSOK
  dbName: ociadbbyosok
  dbWorkload: OLTP
  isDedicated: false
  dbVersion: 19c
  dataStorageSizeInTBs: 2
  cpuCoreCount: 1
  adminPassword:
    secret:
      secretName: ociadb-by-osok-admin-password-secret
  isAutoScalingEnabled: false
  isFreeTier: false
  licenseModel: LICENSE_INCLUDED
  wallet:
    walletName: ociadb-by-osok-wallet-secret
    walletPassword:
      secret:
        secretName: ociadb-by-osok-wallet-password-secret

  • Update execution result

    image-20211223142835787

    winter@cloudshell:~ (ap-chuncheon-1)$ kubectl describe autonomousdatabases ociadbbyosok
Name:         ociadbbyosok
...
Status:
  Status:
    Conditions:
      Last Transition Time:  2022-04-28T05:08:06Z
      Message:               AutonomousDatabase Provisioning
      Status:                True
      Type:                  Provisioning
      Last Transition Time:  2022-04-28T05:10:15Z
      Message:               AutonomousDatabase OCIADBbyOSOK is Active
      Status:                True
      Type:                  Active
      Last Transition Time:  2022-04-28T05:25:21Z
      Message:               AutonomousDatabase Update success
      Status:                True
      Type:                  Active
    Ocid:                    ocid1.autonomousdatabase.oc1.ap-chuncheon-1.an4w4ljrbgdxvpqa4ix636vselpdrig6i2peeczd2fgkevel34bph6feb46a
...

Autonomous Database (ADB) Delete

As the Delete function is not provided separately based on the current version, deleting the autonomousdatabases resource in the OKE cluster with the kubectl delete command does not delete the actual ADB instance from OCI.



As an individual, this article was written with my personal time. There may be errors in the content of the article, and the opinions in the article are personal opinions.

Last updated on 28 Apr 2022