Test Design of TestCase Label

The Design of Label Mechanism of Test Case

The Label of test case is used to filter case conveniently.

The Usage

The labels can be added into test case by below way:

start:test_case_name
description: The description of current case 
os:Linux
hcp:openbmc
label:label1,label2,label3,.....
.....
end

Suppose in future developer/FVT can filter expected case list by below way:

# xcattst -s (label1 and label2) or (label3 and label4) or (! label5)

Label Mechanism

There are two primary dimensions of labels, ones are ones are horizontal dimension labels that decides the category of cases.

Vertical Dimension Labels

Vertical dimension labels that decides the running order of cases. As long as follow below order, anyone can finish a whole test against xCAT based on a brand new environment.

| level | Label name|case example| |-------|-----------| |1|xcat_install|install_xCAT_on_rhels_sles| |2|mn_only|lsdef_,xcatd_restart| |3|sn_diskful|SN_setup_case| | |sn_diskless|SN_diskless_setup_case| |4|flat_cn_diskful|reg_linux_diskfull_installation_flat| | |flat_cn_diskless|reg_linux_diskless_installation_flat| | |hierarchy_cn_diskful|reg_linux_diskfull_installation_hierarchy| | |hierarchy_cn_diskless|reg_linux_diskless_installation_hierarchy| |5|cn_bmc_ready|rpower_| | |cn_os_ready|xdcp_*| |6|others|nodeset_cmd|

There are some rules for vertical dimension labels

• It is mandatory label. That means one cases must have ONE label in above table and belong to one specific level.

• Level 1,3,4 can be treated as part of test environment setup. That means if need not to setup environment in your case, you can skip some levels.

  For example:

  • If you have set up a MN with expected xcat installed and want to do smoking test, just need to run all cases with mn_only label.
  • If you have a MN with brand new os and connected to a CN, want to test hardware control against CN, just need to filter cases from level 1 and level 5, skip level 2-4.

• There are more than one labels in level 3/4/5, when create a kind of whole test against xCAT, only one label in these levels can be chosen.

Horizontal Dimension Labels

db,xcatd, hctrl_gen,hctrl_hmc,hctrl_fsp,hctrl_ipmi,hctrl_openbmc provision,go-xcat,statelite invoke_provison

console, discovery docker GPU KIT migration network postscripts restapi Switch PDU HA

go-coserver dhcp dns IB packaging parallel_cmds scaling security updatenode file-sync xcatprobe cumulus virtualization firmware performance