DAOS-16487 control: Require hostname for nvme set-faulty & replace #15074

tanabarr · 2024-09-05T14:06:05Z

Makes --host option mandatory for dmg storage set nvme-faulty, dmg
storage replace nvme and dmg telemetry metrics. These commands should
only be run on a single host which should be explicitly specified.

Also remove --no-reint flag from replace nvme command.

Features: control
Required-githooks: true

Before requesting gatekeeper:

Two review approvals and any prior change requests have been resolved.
Testing is complete and all tests passed or there is a reason documented in the PR why it should be force landed and forced-landing tag is set.
Features: (or Test-tag*) commit pragma was used or there is a reason documented that there are no appropriate tags for this PR.
Commit messages follows the guidelines outlined here.
Any tests skipped by the ticket being addressed have been run and passed in the PR.

Gatekeeper:

github-actions · 2024-09-05T14:06:28Z

Ticket title is 'dmg storage must precisely identify NVMe device(s)'
Status is 'In Review'
Labels: 'lrz,usability'
https://daosio.atlassian.net/browse/DAOS-16487

daosbuild1 · 2024-09-05T14:40:48Z

Test stage Unit Test on EL 8.8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-15074/1/testReport/

Features: control Required-githooks: true Signed-off-by: Tom Nabarro <tom.nabarro@intel.com>

kjacque

Overall looks good. Ftest dmg infrastructure updates may be needed for the telemetry commands now that the single host command has changed.

src/tests/ftest/util/dmg_utils_base.py

daosbuild1 · 2024-09-12T01:34:17Z

Test stage Functional on EL 8.8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-15074/2/testReport/

src/tests/ftest/util/dmg_utils.py

knard38

LGTM.
+1 with kris for adding the telemetry feature before approving.

Features: control telemetry Required-githooks: true Signed-off-by: Tom Nabarro <tom.nabarro@intel.com>

Features: pool telemetry Required-githooks: true Signed-off-by: Tom Nabarro <tom.nabarro@intel.com>

daltonbohning

ftest LGTM

daosbuild1 · 2024-09-13T21:31:02Z

Test stage Functional Hardware Medium completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-15074/4/display/redirect

daosbuild1 · 2024-09-13T21:47:52Z

Test stage Functional Hardware Medium Verbs Provider completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-15074/4/display/redirect

daosbuild1 · 2024-09-15T05:34:06Z

Test stage Functional Hardware Medium completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-15074/5/execution/node/505/log

phender · 2024-09-16T15:35:06Z

src/tests/ftest/util/nvme_utils.py

@@ -44,7 +44,7 @@ def set_device_faulty(test, dmg, server, uuid, pool=None, has_sys_xs=False, **kw
    Args:
        test (Test): avocado test class
        dmg (DmgCommand): a DmgCommand class instance
-        server (NodeSet): host on which to issue the dmg storage set nvme-faulty
+        server (str): host on which to issue the dmg storage set nvme-faulty


Minor, but we should probably leave this as a NodeSet input type. This docstring change implies that the user will need to unnecessarily convert a NodeSet object into a string when this is not actually needed. Through the FormattedParameter object the DmgCommand.host.value NodeSet object will be converted into a string when creating the dmg command string. Ideally, we want to keep all host information as a NodeSet object in the test harness as it simplifies other operations we may need to perform.

daltonbohning · 2024-09-16T20:56:31Z

deployment/disk_failure.py and vmd/fault_reintegration.py failures in
https://build.hpdd.intel.com/blue/organizations/jenkins/daos-stack%2Fdaos/detail/PR-15074/5/tests
are manual tests and not expected to pass in CI

Features: pool telemetry Required-githooks: true Signed-off-by: Tom Nabarro <tom.nabarro@intel.com>

…tfaulty-replace-hostopt Features: pool telemetry Signed-off-by: Tom Nabarro <tom.nabarro@intel.com>

daltonbohning

ftest LGTM

tanabarr · 2024-09-17T21:10:43Z

https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-15074/6/artifact/Functional%20Hardware%20Medium/vmd/fault_reintegration.py/job.log/*view*/

Passed VMD tests because by chance functional hardware medium ran on VMD enabled cluster (PCI addresses begin with non-zero domain)
2024-09-17 07:36:58,816 fault_reintegrat L0068 DEBUG| Verifying: dev_state (NORMAL == NORMAL) and led_state (OFF == OFF)

Whereas https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-15074/5/artifact/Functional%20Hardware%20Medium/vmd/fault_reintegration.py/job.log/*view*/

Failed VMD tests because the same test ran on a non-VMD enabled cluster (PCI addresses begin with zero domain)
2024-09-15 02:58:30,848 fault_reintegrat L0068 DEBUG| Verifying: dev_state (NORMAL == NORMAL) and led_state (NA == OFF)

mjmac

Changes look good to me, but I'll defer to those with more context into requirements for approvals.

kjacque

Looks OK to me as long as telemetry tests pass.

…15074) Makes --host option mandatory for dmg storage set nvme-faulty, dmg storage replace nvme and dmg telemetry metrics. These commands should only be run on a single host which should be explicitly specified. Also remove --no-reint flag from replace nvme command. Signed-off-by: Tom Nabarro <tom.nabarro@intel.com>

…15074) (#15260) Makes --host option mandatory for dmg storage set nvme-faulty, dmg storage replace nvme and dmg telemetry metrics. These commands should only be run on a single host which should be explicitly specified. Also remove --no-reint flag from replace nvme command. Signed-off-by: Tom Nabarro <tom.nabarro@intel.com>

tanabarr added 2 commits September 10, 2024 22:43

DAOS-16487 control: Require hostname for nvme set-faulty & replace

068f115

Features: control Required-githooks: true Signed-off-by: Tom Nabarro <tom.nabarro@intel.com>

simplify singleHostCmd implementation

2f3ca2f

Features: control Required-githooks: true Signed-off-by: Tom Nabarro <tom.nabarro@intel.com>

tanabarr force-pushed the tanabarr/control-setfaulty-replace-hostopt branch from ea480b9 to 2f3ca2f Compare September 11, 2024 21:59

tanabarr marked this pull request as ready for review September 11, 2024 22:04

tanabarr requested review from a team as code owners September 11, 2024 22:04

tanabarr self-assigned this Sep 11, 2024

tanabarr added control-plane work on the management infrastructure of the DAOS Control Plane go Pull requests that update Go code usability Changes specific to user facing tools or behaviour. labels Sep 11, 2024

tanabarr requested review from mjmac, kjacque and knard38 September 11, 2024 22:06

tanabarr mentioned this pull request Sep 11, 2024

DAOS-16487 control: Only use hostlist from cfg on cfg-aware dmg cmds #15087

Draft

18 tasks

kjacque reviewed Sep 12, 2024

View reviewed changes

src/tests/ftest/util/dmg_utils_base.py Show resolved Hide resolved

daltonbohning reviewed Sep 12, 2024

View reviewed changes

src/tests/ftest/util/dmg_utils.py Show resolved Hide resolved

src/tests/ftest/util/dmg_utils.py Show resolved Hide resolved

knard38 reviewed Sep 13, 2024

View reviewed changes

address review comments from daltonbohning

c7882c1

Features: control telemetry Required-githooks: true Signed-off-by: Tom Nabarro <tom.nabarro@intel.com>

tanabarr requested review from daltonbohning, kjacque and knard38 September 13, 2024 13:39

correct telemetry metrics query and list host option name

9d038a6

Features: pool telemetry Required-githooks: true Signed-off-by: Tom Nabarro <tom.nabarro@intel.com>

daltonbohning reviewed Sep 13, 2024

View reviewed changes

knard38 previously approved these changes Sep 16, 2024

View reviewed changes

phender reviewed Sep 16, 2024

View reviewed changes

tanabarr added 2 commits September 16, 2024 22:00

address comments from phender

077ddeb

Features: pool telemetry Required-githooks: true Signed-off-by: Tom Nabarro <tom.nabarro@intel.com>

Merge remote-tracking branch 'origin/master' into tanabarr/control-se…

02dc210

…tfaulty-replace-hostopt Features: pool telemetry Signed-off-by: Tom Nabarro <tom.nabarro@intel.com>

tanabarr dismissed knard38’s stale review via 02dc210 September 16, 2024 21:03

tanabarr requested review from phender, knard38 and daltonbohning September 16, 2024 21:03

daltonbohning reviewed Sep 17, 2024

View reviewed changes

mjmac reviewed Sep 17, 2024

View reviewed changes

kjacque approved these changes Sep 17, 2024

View reviewed changes

knard38 approved these changes Sep 18, 2024

View reviewed changes

tanabarr requested a review from a team September 18, 2024 08:40

mjmac merged commit 049f710 into master Sep 18, 2024
60 checks passed

mjmac deleted the tanabarr/control-setfaulty-replace-hostopt branch September 18, 2024 12:27

techbasset mentioned this pull request Oct 14, 2024

Merge upstream/release/2.6 into upstream/google/2.6 #15317

Merged

mjmac mentioned this pull request Nov 13, 2024

mjmac/DAOS 16787 google 2.6 #15498

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DAOS-16487 control: Require hostname for nvme set-faulty & replace #15074

DAOS-16487 control: Require hostname for nvme set-faulty & replace #15074

tanabarr commented Sep 5, 2024 •

edited

Loading

github-actions bot commented Sep 5, 2024 •

edited

Loading

daosbuild1 commented Sep 5, 2024

kjacque left a comment

daosbuild1 commented Sep 12, 2024

knard38 left a comment •

edited

Loading

daltonbohning left a comment

daosbuild1 commented Sep 13, 2024

daosbuild1 commented Sep 13, 2024

daosbuild1 commented Sep 15, 2024

phender Sep 16, 2024 •

edited

Loading

tanabarr Sep 17, 2024

daltonbohning commented Sep 16, 2024

daltonbohning left a comment

tanabarr commented Sep 17, 2024 •

edited

Loading

mjmac left a comment

kjacque left a comment

DAOS-16487 control: Require hostname for nvme set-faulty & replace #15074

DAOS-16487 control: Require hostname for nvme set-faulty & replace #15074

Conversation

tanabarr commented Sep 5, 2024 • edited Loading

Before requesting gatekeeper:

Gatekeeper:

github-actions bot commented Sep 5, 2024 • edited Loading

daosbuild1 commented Sep 5, 2024

kjacque left a comment

Choose a reason for hiding this comment

daosbuild1 commented Sep 12, 2024

knard38 left a comment • edited Loading

Choose a reason for hiding this comment

daltonbohning left a comment

Choose a reason for hiding this comment

daosbuild1 commented Sep 13, 2024

daosbuild1 commented Sep 13, 2024

daosbuild1 commented Sep 15, 2024

phender Sep 16, 2024 • edited Loading

Choose a reason for hiding this comment

tanabarr Sep 17, 2024

Choose a reason for hiding this comment

daltonbohning commented Sep 16, 2024

daltonbohning left a comment

Choose a reason for hiding this comment

tanabarr commented Sep 17, 2024 • edited Loading

mjmac left a comment

Choose a reason for hiding this comment

kjacque left a comment

Choose a reason for hiding this comment

tanabarr commented Sep 5, 2024 •

edited

Loading

github-actions bot commented Sep 5, 2024 •

edited

Loading

knard38 left a comment •

edited

Loading

phender Sep 16, 2024 •

edited

Loading

tanabarr commented Sep 17, 2024 •

edited

Loading