OASE Agent on Kubernetes - Online

Introduction

This document aims to explain how to install the Exastro OASE Agent, which is used to link with external services when using OASE, on Kubernetes.

Features

This document contains information on how to install Exastro OASE Agent with high availability and service level.
For a more simple installation for testing and temporary usage, we recommend the Docker Compose version of the OASE Agent.

Prerequisites

  • Exastro IT Automation

    Both the Exastro OASE Agent and the Exastro IT Automation must be on the same version in order to operate the Exastro OASE Agent.

  • Client requirements

    The following describes confirmed compatible client application as well as their versions.
    Table 35 Client requirements

    Application

    Version

    Helm

    v3.9.x

    kubectl

    1.23

  • Deploy environment

    The following describes confirmed compatible operation systems as well as their versions.
    Table 36 Hardware requirements (minimum requirements)

    Resource type

    Required resource

    CPU

    2 Cores (3.0 GHz, x86_64)

    Memory

    4GB

    Storage (Container image size)

    10GB

    Kubernetes (Container image size)

    1.23 or later

  • Communication Protocols

    • The user must be able to access the target fetch server from the OASE Agent.

    • The user must be able to connect to Docker Hub in order to acquire the container image from the container environment.

    Warning

    If deploying to an environment constructed in Helm chart (Kubernetes) version, make sure that the environment has enough specs to meet additional minimum specs of the OASE Agent.

Preparation

Register Helm repository

The Exastro OASE Agent exists on the same helm repository as the Exastro system.

  • Exastro OASE Agent

Repository

https://exastro-suite.github.io/exastro-helm/
Listing 131 Cmmand
1# Register Exastro system's Helm repository.
2helm repo add exastro https://exastro-suite.github.io/exastro-helm/ --namespace exastro
3# Update repository information
4helm repo update

Fetch default setting values

The following command outputs the values.yaml default values. This makes it easier to manage the input parameters.
Listing 132 Command
helm show values exastro/exastro-agent > exastro-agent.yaml
exastro-agent.yaml
  1# Default values for exastro-agent.
  2# This is a YAML-formatted file.
  3# Declare variables to be passed into your templates.
  4global:
  5  agentGlobalDefinition:
  6    name: agent-global
  7    enabled: true
  8    image:
  9      registry: "docker.io"
 10      organization: exastro
 11      package: exastro-it-automation
 12
 13ita-ag-oase:
 14  agents:
 15    - image:
 16        repository: ""
 17        # Overrides the image tag whose default is the chart appVersion.
 18        tag: ""
 19        pullPolicy: IfNotPresent
 20      extraEnv:
 21        TZ: Asia/Tokyo
 22        DEFAULT_LANGUAGE: ja
 23        LANGUAGE: "en"
 24        ITERATION: "500"
 25        EXECUTE_INTERVAL: "10"
 26        LOG_LEVEL: INFO
 27        AGENT_NAME: "oase-agent"
 28        EXASTRO_URL: "http://platform-auth:8000"
 29        EXASTRO_ORGANIZATION_ID: "org001"
 30        EXASTRO_WORKSPACE_ID: "ws01"
 31        # ROLES: "_ws_admin"
 32        EVENT_COLLECTION_SETTINGS_NAMES: "id0001"
 33      secret:
 34        EXASTRO_REFRESH_TOKEN: "exastro_refresh_token"
 35        # EXASTRO_USERNAME: "admin"
 36        # EXASTRO_PASSWORD: "sample-password"
 37      resources: {}
 38        # requests:
 39        #   memory: "64Mi"
 40        #   cpu: "250m"
 41        # limits:
 42        #   memory: "64Mi"
 43        #   cpu: "250m"
 44
 45  imagePullSecrets: []
 46  nameOverride: ""
 47  fullnameOverride: ""
 48
 49  initContainerImage:
 50    repository: "registry.access.redhat.com/ubi8/ubi-init"
 51    pullPolicy: IfNotPresent
 52    # Overrides the image tag whose default is the chart appVersion.
 53    tag: ""
 54
 55  serviceAccount:
 56    # Specifies whether a service account should be created
 57    create: false
 58    # Annotations to add to the service account
 59    annotations: {}
 60    # The name of the service account to use.
 61    # If not set and create is true, a name is generated using the fullname template
 62    name: ""
 63
 64  persistence:
 65    enabled: true
 66    reinstall: false
 67    accessMode: ReadWriteMany
 68    size: 10Gi
 69    volumeType: hostPath # e.g.) hostPath or AKS
 70    storageClass: "-" # e.g.) azurefile or - (None)
 71    # matchLabels:
 72    #   release: "stable"
 73    # matchExpressions:
 74    #   - {key: environment, operator: In, values: [dev]}
 75    mountPath:
 76      storage: /storage
 77      homeDir: /home/app_user
 78      pid:
 79        path: /var/run_app_user/httpd/pid
 80        subPath: httpd-pid
 81      socket:
 82        path: /var/run_app_user/httpd/socket
 83        subPath: httpd-socket
 84      tmp: /tmp
 85
 86  podAnnotations: {}
 87
 88  podSecurityContext: {}
 89    # fsGroup: 2000
 90
 91  securityContext:
 92    allowPrivilegeEscalation: false
 93    readOnlyRootFilesystem: true
 94    runAsUser: 1000
 95    runAsGroup: 1000
 96    runAsNonRoot: true
 97
 98  service: {}
 99
100  ingress:
101    enabled: false
102    className: ""
103    annotations: {}
104      # kubernetes.io/ingress.class: nginx
105      # kubernetes.io/tls-acme: "true"
106    hosts:
107      - host: chart-example.local
108        paths:
109          - path: /
110            pathType: ImplementationSpecific
111    tls: []
112  #  - secretName: chart-example-tls
113    #    hosts:
114    #      - chart-example.local
115
116  nodeSelector: {}
117
118  tolerations: []
119
120  affinity: {}
The following steps will configure required parameters to exastro-agent.yaml.

OASE Agent settings

OASE Agentを立ち上げる際の代表的な設定方法について紹介します。
In the following example, the persistent volume is configured to hostPath.

  • Simple architecture

  • Multiple agents (Same Pod)

  • Multiple agents (Different Pod)

Parameter

See the following in order to see the usable parameters.
Table 37 List of Values in ita-ag-oase

Parameters

Description

Change

Default Value / Available Options

ita-ag-oase.agents.image.repository

Container Image Repository Name

Disabled

""

ita-ag-oase.agents.image.tag

Container Image Tag

Disabled

""

ita-ag-oase.agents.image.pullPolicy

Image Pull Policy

Enabled
IfNotPresent (Default): Pull only if the container image is not present
Always: Always pull
None: Do not pull

ita-ag-oase.agents.extraEnv.TZ

Default Language Used by the OASE Agent System

Enabled

Asia/Tokyo

ita-ag-oase.agents.extraEnv.DEFAULT_LANGUAGE

Language Used by the OASE Agent System

Enabled

ja

ita-ag-oase.agents.extraEnv.LANGUAGE

OASE Agent Version

Enabled

en

ita-ag-oase.agents.extraEnv.ITERATION

Number of retries before the OASE Agent initializes the settings (Maximum: 120, Minimum: 10)

Enabled

500

ita-ag-oase.agents.extraEnv.EXECUTE_INTERVAL

Interval (in seconds) for the OASE Agent (ita-ag-oase) to fetch events from the target (Minimum: 3 seconds)

Enabled

10

ita-ag-oase.agents.extraEnv.LOG_LEVEL

OASE Agent Log Level

Enabled

INFO

ita-ag-oase.eagents.xtraEnv.AGENT_NAME

Name of the OASE Agent to be started

Enabled

oase-agent

ita-ag-oase.agents.extraEnv.EXASTRO_URL

Exastro IT Automation Service URL

Enabled

http://platform-auth:8000

ita-ag-oase.agents.extraEnv.EXASTRO_ORGANIZATION_ID

Organization ID created in Exastro IT Automation

Required

org001

ita-ag-oase.agents.extraEnv.EXASTRO_WORKSPACE_ID

Workspace ID created in Exastro IT Automation

Required

ws01

ita-ag-oase.agents.extraEnv.EVENT_COLLECTION_SETTINGS_NAMES
Event Collection Setting Name created in OASE Management → Event Collection of Exastro IT Automation
Multiple entries can be specified, separated by commas

Required

id0001

ita-ag-oase.agents.secret.EXASTRO_REFRESH_TOKEN
Refresh token obtained from the Exastro system management screen
The user’s role must have maintenance permissions for the OASE → Events → Event History menu.

Enabled

None

ita-ag-oase.agents.secret.EXASTRO_USERNAME
Username created in Exastro IT Automation
The user’s role must have permission to maintain the OASE → Events → Event History menu.
When not using EXASTRO_REFRESH_TOKEN (not recommended)

Enabled

admin

ita-ag-oase.agents.secret.EXASTRO_PASSWORD
Password created in Exastro IT Automation
If not using EXASTRO_REFRESH_TOKEN (not recommended)

Enabled

sample-password

ita-ag-oase.agents.resources.requests.memory

Memory request

Enabled

“64Mi”

ita-ag-oase.agents.resources.requests.cpu

CPU request

Enabled

“250m”

ita-ag-oase.agents.resources.limits.memory

Memory limit

Enabled

“64Mi”

ita-ag-oase.agents.resources.limits.cpu

CPU limit

Enabled

“250m”

ita-ag-oase.nameOverride
Exastro OASE Agent Definition Name
Required when deploying multiple agents within the same cluster

Enabled

""

ita-ag-oase.persistence.enabled

Deployment status of the OASE Agent container

Enabled
true (Default): Deploy the OASE Agent container.
false : Do not deploy the OASE Agent container.

ita-ag-oase.persistence.reinstall

Whether to initialize the data volume during reinstallation

Disabled
true : Initialize (delete) the data
false (Default): Do not initialize (delete) the data

ita-ag-oase.persistence.accessMode

Specify the access mode of the persistent volume.

Disabled

“ReadWriteMany”

ita-ag-oase.persistence.size

Disk capacity of the persistent volume

Enabled (When data persistence is enabled)

“10Gi”

ita-ag-oase.persistence.volumeType

Volume type of the persistent volume

Enabled (Currently disabled)
hostPath (Default): Store data on Kubernetes cluster nodes (deprecated)
AKS: Use AKS storage class

ita-ag-oase.persistence.storageClass

Specify the storage class to use for the persistent volume

Enabled (When data persistence is enabled)
- (Default): Do not specify a storage class.
storage class name: Specify the storage class name provided by the cloud provider or similar.

exastro-platform.mariadb.persistence.matchLabels.name

Specify the name of the persistent volume to use

Enabled(When data persistence is enabled)

""

OASE Agent parameter setting example

The following is a sample of Agent settings.

  • Features

Simple architecture with 1 container for 1 pod.

  • Setting example

  1. OASE Agent settings

    Configure the OASE Agent.
    Listing 133 exastro-agent.yaml
    13ita-ag-oase:
14  agents:
15    - image:
16        repository: ""
17        # Overrides the image tag whose default is the chart appVersion.
18        tag: "alpha.4ccca4.20240124-110529"
19        pullPolicy: IfNotPresent
20      extraEnv:
21        TZ: Asia/Tokyo
22        DEFAULT_LANGUAGE: ja
23        LANGUAGE: "en"
24        ITERATION: "500"
25        EXECUTE_INTERVAL: "10"
26        LOG_LEVEL: INFO
27        AGENT_NAME: "oase-agent"
28-       EXASTRO_URL: "http://platform-auth:8000"
29+       EXASTRO_URL: "http://your-exastro-url"                                   # Exastro IT Automation Service URL
30-       EXASTRO_ORGANIZATION_ID: "org001"
31+       EXASTRO_ORGANIZATION_ID: "your-organization-id"                          # Organization created in Exastro IT Automation
32-       EXASTRO_WORKSPACE_ID: "ws01"
33+       EXASTRO_WORKSPACE_ID: "your-workspace-id"                                # WorkspaceID created in Exastro IT Automation
34        # ROLES: "_ws_admin"
35-       EVENT_COLLECTION_SETTINGS_NAMES: "id0001"
36+       EVENT_COLLECTION_SETTINGS_NAMES: "your-event-collection-settigs-names"   # The Event collection setting name created in Exastro IT Automation's OASE Management event collection.
37      secret:
38-       EXASTRO_REFRESH_TOKEN: "exastro_refresh_token"
39+       EXASTRO_REFRESH_TOKEN: "your_exastro_refresh_token"                      # Refresh token fetched from the Exastro System management page.
40        # EXASTRO_USERNAME: "admin"
41        # EXASTRO_PASSWORD: "sample-password"
42+       # EXASTRO_USERNAME: "your-ita-user-name"                # Username created in Exastro IT Automation.(Remember to uncomment if using this)
43+       # EXASTRO_PASSWORD: "your-ita-user-password"            # Password created in Exastro IT Automation.(Remember to uncomment if using this)

Persistent volume settings

In order to persist data in a database(if the container is within a cluster), or files, the user will need to configure a persistent volume.
For more information regarding persistent volumes, see Persistent volumes - Kubernetes を参照してください。

  • Features

This method uses storage on the Kubernetes node. There is no need to provide seperate storage, but we recommend that this method is only used for testing and developing.

Danger

While persisting data is possible, data might be deleted if compute nodes are changed. We strongly recommend against using this method to persist data in production.
Note that if AKS clusters created with Azure are stopped, the AKS cluster’s node will be released. This means that all saved information will be deleted.

  • Example

The example below uses hostPath.
Listing 141 pv-ita-ag-oase.yaml
 1# pv-ita-ag-oase.yaml
 2apiVersion: v1
 3kind: PersistentVolume
 4metadata:
 5  name: pv-ita-ag-oase
 6spec:
 7  claimRef:
 8    name: pvc-ita-ag-oase
 9    namespace: exastro
10  capacity:
11    storage: 10Gi
12  accessModes:
13    - ReadWriteMany
14  persistentVolumeReclaimPolicy: Retain
15  hostPath:
16    path: /var/data/exastro-suite/exastro-agent/ita-ag-oase
17    type: DirectoryOrCreate

Install

Note

If the installation fails, follow Uninstall and try reinstalling.

Create Persistent volumes

Apply the manifest file created in Persistent volume settings and create persistent volume.
# pv-ita-ag-oase.yaml
kubectl apply -f pv-ita-ag-oase.yaml

# Run the following for Multiple Agents (Different Pod)
# pv-ita-ag-oase-1.yaml
kubectl apply -f pv-ita-ag-oase-1.yaml

# pv-ita-ag-oase-2.yaml
kubectl apply -f pv-ita-ag-oase-2.yaml

# Confirm
kubectl get pv

NAME              CAPACITY   ACCESS MODES   RECLAIM POLICY    STATUS      CLAIM     STORAGECLASS   REASON   AGE
pv-ita-ag-oase     10Gi       RWX            Retain           Available                                     6s

NAME              CAPACITY   ACCESS MODES   RECLAIM POLICY    STATUS      CLAIM     STORAGECLASS   REASON   AGE
pv-ita-ag-oase-1   10Gi       RWX            Retain           Available                                     5s
pv-ita-ag-oase-2   10Gi       RWX            Retain           Available                                     6s

Install

See the exastro-helm site <https://github.com/exastro-suite/exastro-helm> for more information regarding the Helm and Application versions.

  1. Use Helm command to install on Kubernetes environment.

    Listing 144 Command
    helm install exastro-agent exastro/exastro-agent \
  --namespace exastro --create-namespace \
  --values exastro-agent.yaml
    Listing 145 Output results
    NAME: exastro-agent
LAST DEPLOYED: Wed Feb 14 14:36:27 2024
NAMESPACE: exastro
STATUS: deployed
REVISION: 1
TEST SUITE: None

  2. Confirm Installation status

    Listing 146 Command
    # Fetch Pod  list
kubectl get po --namespace exastro

| If running normally, the status will say "Running".
| ※The user might have to wait a couple of minutes before the status changes to "Running".
    Listing 147 Output results
     NAME                             READY   STATUS    RESTARTS   AGE
 ita-ag-oase-66cb7669c6-m2q8c     1/1     Running   0          16m

Update

This section describes how to update the OASE Agent.

Update preparation

Warning

We recommend that back up the data before updating.
Check the version before updating.
Listing 154 Command
1# Check Repository information
2helm search repo exastro
Listing 155 Run results
1helm search repo exastro
2NAME                            CHART VERSION   APP VERSION     DESCRIPTION
3exastro/exastro                         1.3.24          2.3.0           A Helm chart for Exastro. Exastro is an Open So...
4exastro/exastro-agent                   1.0.3           2.3.0           A Helm chart for Exastro IT Automation. Exastro...
5exastro/exastro-it-automation           1.4.22          2.3.0           A Helm chart for Exastro IT Automation. Exastro...
6exastro/exastro-platform                1.7.14          1.7.0           A Helm chart for Exastro Platform. Exastro Plat...
Update the Helm repository.
Listing 156 Command
1# Update Repository information
2helm repo update
Check that it has been updated to the latest version.
Listing 157 Command
1# Check Repository information
2helm search repo exastro
Listing 158 Run results
1helm search repo exastro
2NAME                            CHART VERSION   APP VERSION     DESCRIPTION
3exastro/exastro                    1.4.3           2.4.0           A Helm chart for Exastro. Exastro is an Open So...
4exastro/exastro-agent              2.4.0           2.4.0           A Helm chart for Exastro IT Automation. Exastro...
5exastro/exastro-it-automation      2.4.1           2.4.0           A Helm chart for Exastro IT Automation. Exastro...
6exastro/exastro-platform           1.8.1           1.8.0           A Helm chart for Exastro Platform. Exastro Plat...

Update

Stop service

  1. Checking the number of running Pods

    Check the number of running Pods before starting the operation and record their status.
    Listing 159 Command
    RS_AG=`kubectl get deploy ita-ag-oase -o jsonpath='{@.spec.replicas}{"\n"}' --namespace exastro`

  2. Stopping background processes

    Set the number of running Pods to zero to stop database updates.
    Listing 160 Command
    kubectl scale deployment ita-ag-oase --namespace exastro --replicas=0

  3. Check the number of running Pods

    Verify that the number of target Pods stopped above has reached zero.
    Listing 161 Command
    kubectl get deployment --namespace exastro
    Listing 162 Execution result
    NAME                                     READY   UP-TO-DATE   AVAILABLE   AGE
ita-ag-oase                              0/1     1            1           29m

Start Update

Start the update.
Listing 163 Command
helm upgrade exastro-agent exastro/exastro-agent \
  --namespace exastro \
  --values exastro-agent.yaml
Listing 164 Output results
Release "exastro-agent" has been upgraded. Happy Helming!
NAME: exastro-agent
LAST DEPLOYED: Mon Apr 22 14:42:59 2024
NAMESPACE: exastro
STATUS: deployed
REVISION: 2
TEST SUITE: None

Restart service

  1. Service Resumption

    Restore the number of running Pods for each Deployment to the values recorded at the time of service shutdown.
    Listing 165 Command
    kubectl scale deployment ita-ag-oase --namespace exastro --replicas=${RS_AG}

  2. Check the number of running Pods

    Verify that the number of target Pods started above has been restored and all are in the READY state.
    Listing 166 Command
    kubectl get deployment --namespace exastro
    Listing 167 Execution result
    NAME                                     READY   UP-TO-DATE   AVAILABLE   AGE
ita-ag-oase                              1/1     1            1           29m

Check Update status

Input the following commands from the command line and check that the agent has been updated.
Listing 168 Command
# Fetc Pod list
kubectl get po --namespace exastro
If running normally, the status will say “Running”.
※The user might have to wait a couple of minutes before the status changes to “Running”.
Listing 169 Output results
NAME                                                      READY   STATUS      RESTARTS   AGE
ita-ag-oase-7ff9488b55-rrn58                              1/1     Running     0             81s

Uninstall

This section explains how to uninstall the OASE Agent.

Uninstall preparation

Warning

We recommend that back up the data before uninstalling.

Uninstall

Start Uninstall

Start the uninstall process.
Listing 170 Command
helm uninstall exastro-agent --namespace exastro
Listing 171 Output results
release "exastro-agent" uninstalled

Delete persistent volumes

This section describes how to delete data if a persistent volume(PV) has been created on Kubernetes using hostPath.
If using external databases (managed databases included), make sure to delete environmental data as well.
For Agents

Warning

If there are multiple persistent volumes for Agents, make sure to delete them all.
Listing 172 Command
kubectl delete pv pv-ita-ag-oase
Listing 173 Execution results
persistentvolume "pv-ita-ag-oase" deleted

Deleting Persistent data

Log in to the Kubernetes Control node and delete the data.
For Agents
The following command is an example where the hostPath is specified to /var/data/exastro-suite/exastro-agent/ita-ag-oase when the Persistent Volume was created.
Listing 174 Command
# Log in to control node that has persistent data
ssh user@contol.node.example

# Delete persistent data
sudo rm -rf /var/data/exastro-suite/exastro-agent/ita-ag-oase