5. Collect function¶
5.1. Introduction¶
5.2. Collect function overview¶
5.2.1. Collect function¶
Collect function overview figure¶

図 5.7 Collect function overview figure¶
Collect function data registration process figure¶

図 5.8 Collect function data registration process figure¶
Tip
5.2.2. Registering to Parameter sheets¶
Collect function requirements¶
- A parameter sheet (with host/operation) is created in themenu.
- The registration information (source file) is linked to the items in the parameter sheet in.
- Collect target device (Host name) is registered to the Device list.
- Operation execution result, the operation has successfully ended.
- Directories and files are arranged in a specific structure as a result of the output of the operation execution.
注釈
5.3. Collect function directory, file structure and Variable handling methods¶
5.3.1. Collect target directory file structure¶
Collect target file format
■File name:RH_snmp.yml ■File contents: VAR_RH_sshd_config: - key: PermitRootLogin value: yes - key: PasswordAuthentication value: no
Collect target directory structure
ITA original variable |
Variable specified contents |
Remarks |
---|---|---|
__parameter_dir__ |
"_parameters" Operation result directory path |
|
__parameters_file_dir__ |
"_parameters_file" Operation result directory path |
|
__parameters_dir_for_epc__ |
"_parameters" Operation result directory path |
|
__parameters_file_dir_for_epc__ |
"_parameters_file" Operation result directory path |
|- _parameters ※1
| |- localhost ※2
| |- SAMPLE.yml ※3
|- _parameters_file ※4
| |- localhost ※2
|- test.txt ※5
- Remarks※1 Collectable directory(Fixed name)※2 Host name(Items registered in the device list are collectable)※3 Collect target file※4 Collectable directory for file uploads(Fixed name)※5 Uploadable file
注釈
Mode |
Mode identifier |
Hierarchy structure |
Remarks |
---|---|---|---|
Ansible-Legacy |
legacy |
/<Upper directory(Ansible)>/legacy/ |
|
Ansible-Pioneer |
pioneer |
/<Upper directory(Ansible)>/pioneer/ |
|
Ansible-LegacyRole |
legacy_role |
/<Upper directory(Ansible)>/legacy_role/ |
注釈
/storage/<Organization>/<Workspace>/driver/ansible/
.Execution mode: Ansible-Legacy
Execution No: 00000000-0000-0000-0000-000000000001
Target: localhost
Execution directory : /storage/Organization/Workspace/driver/ansible/legacy/00000000-0000-0000-0000-000000000001/in/
Execution result directory: /storage/Organization/Workspace/driver/ansible/legacy/00000000-0000-0000-0000-000000000001/out/
Collect target file path and directory structure:
- /storage/Organization/Workspace/driver/ansible/legacy/00000000-0000-0000-0000-000000000001/in/_parameters/localhost/SAMPLE.yml
- /storage/Organization/Workspace/driver/ansible/legacy/00000000-0000-0000-0000-000000000001/in/_parameters/localhost/OS/RH_snmpd.yml
- /storage/Organization/Workspace/driver/ansible/legacy/00000000-0000-0000-0000-000000000001/in/_parameters_file/localhost/TEST.txt
- /storage/Organization/Workspace/driver/ansible/legacy/00000000-0000-0000-0000-000000000001/out/_parameters/localhost/SAMPLE.yml
- /storage/Organization/Workspace/driver/ansible/legacy/00000000-0000-0000-0000-000000000001/out/_parameters/localhost/OS/RH_snmpd.yml
- /storage/Organization/Workspace/driver/ansible/legacy/00000000-0000-0000-0000-000000000001/out/_parameters_file/localhost/TEST.txt
Specification method |
YAML file input method |
Remarks |
---|---|---|
File name specification |
VAR_FILE_NAME : <‘File name> ’ |
|
File path specification (Backwards match) |
VAR_FILE_NAME : ‘/<HierarchyX>/<File name>’ |
Tip
■Structure
【Upper directory】
|- _parameters
| |- localhost
| |- SAMPLE.yml ※Source file
|- _parameters_file
| |- localhost
| |- test.txt ※Upload target file
| |- APP001
|- config ※Upload target file
■Collect target File name:SAMPLE.yml
■File contents
VAR_upload_file: test.txt
VAR_upload_fileX: config
VAR_upload_fileZ: '/APP001/config'
5.3.2. Variable and Variable types¶
- Normal variableCan have one specific value defined per each variable name.
VAR_users: root
- Multiple specific value variableCan have multiple specific values defined per each variable name
VAR_users:
- root
- mysql
- Multistage variableHierarchical variable.
VAR_users:
- user-name: alice #Member variable
authorized: password
Tip
5.5. Collect function procedure¶
5.5.1. Workflow¶
Collect function workflow¶
Workflow details and references
- Create Parameter sheet (with host/operation)'s menu. For more information, see "
- RegisterFrom theand menus, link Source files and Parameter sheet items.For more information, see "Collection item value management".
- Prepare executionPrepare for the execution.For more information, see "Ansible Driver" and "Conductor".
- ExecuteSelect Execution date, Input operation and Movement and execute.For more information regarding execution, see "Ansible Driver" and "Conductor".
- Execute Collect functionUse the Execution number from the Finished execution and use the Collect function to register the values to the parameter sheet.For more information, see "BackYard contents".
- Check Collection statusIn the "" / " " / " "'s menu, users can check the status of the collection process and download the log file.For more information, see "Check Collect status".
5.6. Collect function operation¶
5.6.1. Ansible common¶
Collection item value management¶
- In themenu, users can configure links between the collect items and the paramter sheet items.
図 5.11 Submenu (Collection item value management)¶
- Pressand register a collect item.
図 5.12 Registration/Edit page(Collection item value management)¶
- The items found in themenu are as following.
Item:Collect item(From) |
Description |
Input required |
Restriction |
---|---|---|---|
Parse format |
YAML: Analyses YAML format file and generates Parameter. |
〇 |
※1 |
PREFIX(File name) |
Input the File name without the format extention. |
〇 |
※1 |
Variable name |
Input the Collect target's variable name.
If the variable name is an Array or hash structure, the user must input Member variable as well.
|
〇 |
※1 |
Member variable |
Input if the variable is multiple specific values or multistage variable. |
※1 |
Item:Parameter sheet(To) |
Description |
Input required |
Restrictions |
---|---|---|---|
Menu group:Menu:Item |
Select an Item.
The items are displayed in the following format with the Menu group, Menu name and Item name divided with ":".
|
※2 |
注釈
■File name: SAMPLE.yml
■File contents
VAR_sample_config_1: yes
VAR_sample_config_2: test_parameter
■Values that can be input to Collect item value management's Collect item(FROM)
PREFIX(File name): SAMPLE
Variable name: VAR_sample_config_1
VAR_sample_config_2
■File name: SAMPLE_2.yml
■File contents
VAR_sample2_conf:
- SAMPLE1
- SAMPLE2
- SAMPLE3
■Values that can be input to Collect item value management's Collect item(FROM)
PREFIX(File name): SAMPLE_2
Variable name: VAR_sample2_conf
Member variable: [0]
[1]
[2]
■File name: RH_sshd.yml
■File contents
VAR_RH_sshd_config:
- key: PermitRootLogin
value: yes
- key: PasswordAuthentication
value: no
■Values that can be input to Collect item value management's Collect item(FROM)
PREFIX(File name): RH_sshd
Variable name: VAR_RH_sshd_config:
Member variable: [0].key
[0].value
[1].key
[1].value
■File name: RH_snmp.yml
■File contents
VAR_RH_snmpd_info:
com2sec:
- sec_name: "testsec"
source: "192.168.1.0/24"
community: "public"
- sec_name: "local"
source: "localhost"
community: "private"
■Values that can be input to Collect item value management's Collect item(FROM)
PREFIX(File name): RH_snmp
Variable name: VAR_RH_snmp_config:
Member variable: com2sec[0].sec_name
com2sec[0].source
com2sec[0].community
com2sec[1].sec_name
com2sec[1].source
com2sec[1].community
5.6.2. Ansible-Legacy、Ansible-Pioneer、Ansible-LegacyRole¶
Check Collect status¶

図 5.13 Execution management menu¶
Item |
Description |
Remarks |
---|---|---|
Status |
Collect function execution status
Not target; Not a Collect function target (No target file)
Collected: Collect function executed.
Collected (with notification): If any errors occurred during registration/update.
Collection error:There is something wrong with the Movement operation or Host.
|
※ |
Collect log |
Downloads the Collect function execution log |
Operation status
Status
|
Collect function target |
Target file |
Collect status
Status
|
Collect log |
Remarks |
---|---|---|---|---|---|
Other than Complete |
No |
Not target |
Blank |
Blank |
|
Other than Complete |
Yes |
Not target |
Blank |
Blank |
|
Complete |
No |
Target |
Not target |
With log file |
|
Complete |
Yes |
Target |
Collected |
With log file |
|
Complete |
Yes |
Target |
Collected(with notification) |
With log file |
|
Complete |
Yes |
Target |
Collect error |
With log file |
注釈
Collect START (host name: exastro-test-1 file name: ['Ansible_conf'])
REST DATA (host name: exastro-test-1 menu ID: collect_001 operation NO: 4001e182-51d2-4918-96e0-5981c4dc6d5f)
Collect END (host name: exastro-test-1 file name: ['Ansible_conf'])
Collect START (host name: exastro-test-2 file name: ['Ansible_conf'])
REST DATA (host name: exastro-test-2 menu ID: collect_001 operation NO: 4001e182-51d2-4918-96e0-5981c4dc6d5f)
Collect END (host name: exastro-test-2 file name: ['Ansible_conf'])
Collect START (host name: exastro-test-3 file name: ['Ansible_conf'])
REST DATA (host name: exastro-test-3 menu ID: collect_001 operation NO: 4001e182-51d2-4918-96e0-5981c4dc6d5f)
Collect END (host name: exastro-test-3 file name: ['Ansible_conf'])
Collect START (host name: exastro-test-1 file name: ['Ansible_conf'])
{'item_1': [{'status_code': '', 'msg_args': '', 'msg': 'Regular expression error (1,Value:sample_value)'}]}
[Process] Failed to register or update data in CMDB. (1/1)
Collect END (host name: exastro-test-1 file name: ['Ansible_conf'])
Collect START (host name: exastro-test-2 file name: ['Ansible_conf'])
{'item_1': [{'status_code': '', 'msg_args': '', 'msg': 'Regular expression error (1,Value:sample_value)'}]}
[Process] Failed to register or update data in CMDB. (1/1)
Collect END (host name: exastro-test-2 file name: ['Ansible_conf'])
Collect START (host name: exastro-test-3 file name: ['Ansible_conf'])
{'item_1': [{'status_code': '', 'msg_args': '', 'msg': 'Regular expression error (1,Value:sample_value)'}]}
[Process] Failed to register or update data in CMDB. (1/1)
Collect END (host name: exastro-test-3 file name: ['Ansible_conf'])
[Process] Operation is abolished, so registration and update processing is skipped (Operation No: 4001e182-51d2-4918-96e0-5981c4dc6d5f)
[Process] The registration / update process is skipped because the target device has not been registered or has been abolished (host name: exastro-test-1).
[Process] The registration / update process is skipped because the target device has not been registered or has been abolished (host name: exastro-test-2).
[Process] The registration / update process is skipped because the target device has not been registered or has been abolished (host name: exastro-test-3).
[Process] There is no file in the collection target directory.
5.6.3. BackYard contents¶
Acquire Collection interface information
- Acquire list over completed operations (with Normal end)Collect targetStatus: Complete
- Acquire the following information from the collectable operation no.- Operation information- Target host- Target source file
- Inquire wether the target host is registered in the Device list or notRegistered: Collect targetNot registered: Not target
- Acquire the Menu ID of the target Parameter sheet from the source file and the Collection item value list※If there are multiple Target soruce files, the File names will be processed in rising order.
- Create RESTAPI Parameter with the information gathered in Step 1-4.Query the Menu ID for data and determine the RESTAPI Execution type.Register: Unique operation and Host combination data is not registered.Update: Unique operation and Host combination data is registered.
- Register/Update the data using ITA Standard RESTAPI functions
- Update the status of the Collection status to the Operation No.
5.7. Appendix¶
5.7.1. References¶
- Exastro Playbook Collection
- 2. Ansible config collecting and Parameter creating Playbook.
- name: make yaml file blockinfile: create: yes mode: 0644 insertbefore: EOF marker: "" dest: "{{ __parameter_dir__ }}/{{ inventory_hostname }}/Ansible_conf.yml" content: | ansible_architecture: {{ ansible_architecture }} ansible_bios_version: {{ ansible_bios_version }} ansible_default_ipv4__address: {{ ansible_default_ipv4.address }} ansible_default_ipv4__interface: {{ ansible_default_ipv4.interface }} ansible_default_ipv4__network: {{ ansible_default_ipv4.network }} ansible_distribution: {{ ansible_distribution }} ansible_distribution_file_path: {{ ansible_distribution_file_path }} ansible_distribution_file_variety: {{ ansible_distribution_file_variety }} ansible_distribution_major_version: {{ ansible_distribution_major_version }} ansible_distribution_release: {{ ansible_distribution_release }} ansible_distribution_version: {{ ansible_distribution_version }} ansible_machine: {{ ansible_machine }} ansible_memtotal_mb: {{ ansible_memtotal_mb }} ansible_nodename: {{ ansible_nodename }} ansible_os_family: {{ ansible_os_family }} ansible_pkg_mgr: {{ ansible_pkg_mgr }} ansible_processor_cores: {{ ansible_processor_cores }} ansible_processor_count: {{ ansible_processor_count }} ansible_processor_threads_per_core: {{ ansible_processor_threads_per_core }} ansible_processor_vcpus: {{ ansible_processor_vcpus }} ansible_product_name: {{ ansible_product_name }} ansible_product_serial: {{ ansible_product_serial }} ansible_product_uuid: {{ ansible_product_uuid }} ansible_product_version: {{ ansible_product_version }} ansible_python__executable: {{ ansible_python.executable }} ansible_python_version: {{ ansible_python_version }} ansible_service_mgr: {{ ansible_service_mgr }} vconsole_config: vconsole.conf yum_config: yum.conf - name: get vconsole config fetch: src: /etc/vconsole.conf dest: "{{ __parameters_file_dir__ }}/{{ inventory_hostname }}/" flat: yes - name: get yum config fetch: src: /etc/yum.conf dest: "{{ __parameters_file_dir__ }}/{{ inventory_hostname }}/" flat: yes
注釈
- hosts: all
remote_user: "{{ __loginuser__ }}"
gather_facts: yes
become: yes
5.7.2. Collect function examples¶
Handling Collect target file values¶
VAR_TEST: TEST
VAR_STR_TEST1: 'TEST1'
VAR_STR_TEST2: "TEST2"
VAR_null: null
VAR_NULL: NULL
VAR_STR_null: "null"
VAR_STR_NULL: "NULL"
VAR_true: true
VAR_false: false
VAR_STR_true: "true"
VAR_STR_false: "false"
VAR_YES: YES
VAR_NO: NO
VAR_STR_YES: "YES"
VAR_STR_NO: "NO"
VAR_NON:
VAR_Quotation: ''
VAR_WQuotation: ""
VAR_INT: 100
No |
Key |
Value |
Remarks |
---|---|---|---|
1 |
VAR_TEST |
TEST |
|
2 |
VAR_STR_TEST1 |
'TEST1' |
|
3 |
VAR_STR_TEST2 |
"TEST2" |
|
4 |
VAR_null |
null |
|
5 |
VAR_NULL |
NULL |
|
6 |
VAR_STR_null |
"null" |
|
7 |
VAR_STR_NULL |
"NULL" |
|
8 |
VAR_true |
true |
|
9 |
VAR_false |
false |
|
10 |
VAR_STR_true |
"true" |
|
11 |
VAR_STR_false |
"false" |
|
12 |
VAR_YES |
YES |
|
13 |
VAR_NO |
NO |
|
14 |
VAR_STR_YES |
"YES" |
|
15 |
VAR_STR_NO |
"NO" |
|
16 |
VAR_NON |
||
17 |
VAR_Quotation |
'' |
|
18 |
VAR_WQuotation |
"" |
|
19 |
VAR_INT |
100 |
No |
Collect target (Key:Value) |
Parameter sheet
Item name
|
RESTAPI response
Value
|
RESTAPI response
Data type
|
WEB page display |
---|---|---|---|---|---|
1 |
VAR_TEST: TEST |
Parameter/VAR_TEST |
"TEST" |
string |
TEST |
2 |
VAR_STR_TEST1: 'TEST1' |
Parameter/VAR_STR_TEST1 |
"TEST1" |
string |
TEST1 |
3 |
VAR_STR_TEST2: "TEST2" |
Parameter/VAR_STR_TEST2 |
"TEST2" |
string |
TEST2 |
4 |
VAR_null: null |
Parameter/VAR_null |
null |
null |
|
5 |
VAR_NULL: NULL |
Parameter/VAR_NULL |
null |
null |
|
6 |
VAR_STR_null: "null" |
Parameter/VAR_STR_null |
"null" |
string |
null |
7 |
VAR_STR_NULL: "NULL" |
Parameter/VAR_STR_NULL |
"NULL" |
string |
NULL |
8 |
VAR_true: true |
Parameter/VAR_true |
true |
boolean |
true |
9 |
VAR_false: false |
Parameter/VAR_false |
false |
boolean |
false |
10 |
VAR_STR_true: "true" |
Parameter/VAR_STR_true |
"true" |
string |
true |
11 |
VAR_STR_false: "false" |
Parameter/VAR_STR_false |
"false" |
string |
false |
12 |
VAR_YES: YES |
Parameter/VAR_YES |
true |
boolean |
true |
13 |
VAR_NO: NO |
Parameter/VAR_NO |
false |
boolean |
false |
14 |
VAR_STR_YES: "YES" |
Parameter/VAR_STR_YES |
"YES" |
string |
YES |
15 |
VAR_STR_NO: "NO" |
Parameter/VAR_STR_NO |
"NO" |
string |
NO |
16 |
VAR_NON: |
Parameter/VAR_NON |
null |
null |
|
17 |
VAR_Quotation: '' |
Parameter/VAR_Quotation |
"" |
string |
|
18 |
VAR_WQuotation: "" |
Parameter/VAR_WQuotation |
"" |
string |
|
19 |
VAR_INT: 100 |
パラメータ/VAR_INT |
"100" |
string |
100 |
Target parameter sheet on WebUI display result

図 5.15 Target parameter sheet on WebUI display result 1¶

図 5.16 Target parameter sheet on WebUI display result 2¶
Results fetched with Target Parameter sheet's RESTAPI(filter)
{
"data": [
{
"file": {},
"parameter": {
"base_datetime": "2023/05/17 13:58:47",
"discard": "0",
"host_name": "exastro-test-1",
"item_1": "TEST",
"item_10": "true",
"item_11": "false",
"item_12": "true",
"item_13": "false",
"item_14": "YES",
"item_15": "NO",
"item_16": null,
"item_17": "",
"item_18": "",
"item_19": "100",
"item_2": "TEST1",
"item_3": "TEST2",
"item_4": null,
"item_5": null,
"item_6": "null",
"item_7": "NULL",
"item_8": "true",
"item_9": "false",
"last_execute_timestamp": "2023/05/17 13:58:47",
"last_update_date_time": "2023/05/17 14:00:07.944031",
"last_updated_user": "Collect execute function",
"operation_date": "2023/01/01 00:00:00",
"operation_name_disp": "OP1",
"operation_name_select": "2023/01/01 00:00_OP1",
"remarks": null,
"uuid": "ede113f8-14b1-48bf-8b3f-409f68b107ae"
}
}
],
"message": "SUCCESS",
"result": "000-00000",
"ts": "2023-05-17T05:54:03.937Z"
}
If multiple files with the same file names are Upload target files¶
【Upper directory】
|- _parameters
| |- localhost
| |- SAMPLE.yml
|- _parameters_file
| |- localhost
|- APP001
|- config #①
|- APP002
|- config #②
|- APP003
|- config #③
|- APP002
|- config #④
■Collect target file name:SAMPLE.yml
■File contents
VAR_upload_file_1: config
VAR_upload_file_2: ‘/APP002/config’
VAR_upload_file_3: ‘/APP001/config’
VAR_upload_file_4: ‘/APP003/APP002/config’
Collect item(FROM)/Variable name |
Target file |
Remarks |
---|---|---|
VAR_upload_file_1 |
Random from File ①、②、③、④ |
|
VAR_upload_file_2 |
Random from File ②、④ |
|
VAR_upload_file_3 |
File ① is Target |
|
VAR_upload_file_4 |
File ④ is Target |
Target file contents when File is deleted.¶
【Upper directory】
|- _parameters
| |- localhost
| |- SAMPLE.yml ※Source file
|- _parameters_file
| |- localhost
■Collect target file name: SAMPLE.yml
■File contents
VAR_upload_file: "" ※Upload target file