Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FVT][OpenBMC]rflash NODERANGE -d does not have the same output and result. #5057

Closed
tingtli opened this issue Apr 4, 2018 · 4 comments
Closed

[FVT][OpenBMC]rflash NODERANGE -d does not have the same output and result. #5057

tingtli opened this issue Apr 4, 2018 · 4 comments
Assignees
@bybai
Labels
component:hw_control sprint1 status:won't fix type:bug
Milestone
2.14

Comments

@tingtli
Copy link
Contributor

tingtli commented Apr 4, 2018

xCAT 2.14 on rhels7.5.

rflash operates on 2 nodes. One node is flat and one node is in hierarchy. But seems the output and result is not the same ,please see following output for details.

[root@boston02 /]# rflash mid08tor03cn01,mid08tor03cn10 -d /firmnew
Attempting to upload /firmnew/obmc-phosphor-image-witherspoon.ubi.mtd.tar and /firmnew/witherspoon.pnor.squashfs.tar, please wait...
Attempting to upload /firmnew/obmc-phosphor-image-witherspoon.ubi.mtd.tar and /firmnew/witherspoon.pnor.squashfs.tar, please wait...
mid08tor03cn01: Firmware upload successful. Attempting to activate firmware: ibm-v2.0-0-r44-0-g843c2e1 (ID: 78d09908)
mid08tor03cn01: Firmware upload successful. Attempting to activate firmware: IBM-witherspoon-ibm-OP9_v1.19_1.154 (ID: 4f901d3d)
mid08tor03cn10: Firmware upload successful. Attempting to activate firmware: ibm-v2.0-0-r44-0-g843c2e1 (ID: 78d09908)
mid08tor03cn10: Firmware upload successful. Attempting to activate firmware: IBM-witherspoon-ibm-OP9_v1.19_1.154 (ID: 4f901d3d)
mid08tor03cn01: Firmware IBM-witherspoon-ibm-OP9_v1.19_1.154 activation successful.
mid08tor03cn01: Firmware ibm-v2.0-0-r44-0-g843c2e1 activation successful.
mid08tor03cn01: BMC reboot
mid08tor03cn10: Firmware IBM-witherspoon-ibm-OP9_v1.19_1.154 activation successful.
mid08tor03cn10: Firmware ibm-v2.0-0-r44-0-g843c2e1 activation successful.
mid08tor03cn10: BMC reboot
mid08tor03cn10: Retry BMC state, wait for 15 seconds ...
mid08tor03cn10: Retry BMC state, wait for 15 seconds ...
mid08tor03cn10: Retry BMC state, wait for 15 seconds ...
mid08tor03cn10: Retry BMC state, wait for 15 seconds ...
mid08tor03cn10: Retry BMC state, wait for 15 seconds ...
mid08tor03cn10: Retry BMC state, wait for 15 seconds ...
mid08tor03cn10: BMC Ready
mid08tor03cn10: reset
[root@boston02 /]# echo $?
0
[root@boston02 /]# rflash mid08tor03cn01,mid08tor03cn10 -l
mid08tor03cn10: ID       Purpose State      Version
mid08tor03cn10: -------------------------------------------------------
mid08tor03cn10: 25a5c3e0 Host    Active     IBM-witherspoon-ibm-OP9-v1.21.2-2.6
mid08tor03cn10: 78d09908 BMC     Active(*)  ibm-v2.0-0-r44-0-g843c2e1
mid08tor03cn10: 7a1ce9fe BMC     Active     ibm-v2.0-0-r43-0-gdb0b7b3
mid08tor03cn10: 4f901d3d Host    Active(*)  IBM-witherspoon-ibm-OP9_v1.19_1.154
mid08tor03cn10:
mid08tor03cn01: ID       Purpose State      Version
mid08tor03cn01: -------------------------------------------------------
mid08tor03cn01: 25a5c3e0 Host    Active(*)  IBM-witherspoon-ibm-OP9-v1.21.2-2.6
mid08tor03cn01: 78d09908 BMC     Active(*)  ibm-v2.0-0-r44-0-g843c2e1
mid08tor03cn01: 63c95399 BMC     Active     ibm-v2.0-0-r13.4-0-g7fd26e9
mid08tor03cn01: 4f901d3d Host    Active(+)  IBM-witherspoon-ibm-OP9_v1.19_1.154
mid08tor03cn01:
@tingtli tingtli added this to the 2.14 milestone Apr 4, 2018
@bybai
Copy link
Contributor

bybai commented Apr 4, 2018

Hi @tingtli , I tried in xcatdebugmode=2, I cannot reproduce above issue. Let me try non-xcatdebugmode to reproduce it.

[root@boston02 /]# rflash mid08tor03cn01,mid08tor03cn10 -d /firm |tee rflash.log
Wed Apr  4 01:17:40 2018 OpenBMC: [openbmc_debug_perl]
Attempting to upload /firm/obmc-phosphor-image-witherspoon.ubi.mtd.tar and /firm/witherspoon.pnor.squashfs.tar, please wait...
Wed Apr  4 01:17:40 2018 mid08tor03cn01: [openbmc_debug_perl] curl -k -c cjar -H "Content-Type: application/json" -d '{ "data": ["root", "xxxxxx"] }' https://172.21.226.1/login
Wed Apr  4 01:17:40 2018 mid08tor03cn01: [openbmc_debug_perl] login_response 200 OK
Wed Apr  4 01:17:41 2018 mid08tor03cn01: [openbmc_debug_perl] RFLASH_FILE_UPLOAD_RESPONSE: CMD: curl -b /tmp/_xcat_cjar.mid08tor03cn01 -k -H 'Content-Type: application/octet-stream' -X PUT -T /firm/obmc-phosphor-image-witherspoon.ubi.mtd.tar https://172.21.226.1/upload/image/
Attempting to upload /firm/obmc-phosphor-image-witherspoon.ubi.mtd.tar and /firm/witherspoon.pnor.squashfs.tar, please wait...
Wed Apr  4 01:17:47 2018 mid08tor03cn01: [openbmc_debug_perl] RFLASH_FILE_UPLOAD_RESPONSE: CMD: curl -b /tmp/_xcat_cjar.mid08tor03cn01 -k -H 'Content-Type: application/octet-stream' -X PUT -T /firm/witherspoon.pnor.squashfs.tar https://172.21.226.1/upload/image/
Wed Apr  4 01:17:56 2018 mid08tor03cn01: [openbmc_debug_perl] curl -k -b cjar -X GET -H "Content-Type: application/json" https://172.21.226.1/xyz/openbmc_project/software/enumerate
Wed Apr  4 01:17:57 2018 mid08tor03cn01: [openbmc_debug_perl] rflash_update_check_id_response 200 OK
Wed Apr  4 01:17:57 2018 mid08tor03cn01: [openbmc_debug_perl] CHECK_ID_RESPONSE: Looking for software ID: ibm-v2.1-215-g6e7eacb-r2-0-gecb3626 IBM-witherspoon-ibm-OP9-v1.21.2-2.6...
Wed Apr  4 01:17:57 2018 mid08tor03cn01: [openbmc_debug_perl] CHECK_ID_RESPONSE: key_url=/xyz/openbmc_project/software/25a5c3e0 version=IBM-witherspoon-ibm-OP9-v1.21.2-2.6
mid08tor03cn01: Firmware upload successful. Attempting to activate firmware: IBM-witherspoon-ibm-OP9-v1.21.2-2.6 (ID: 25a5c3e0)
Wed Apr  4 01:17:57 2018 mid08tor03cn01: [openbmc_debug_perl] CHECK_ID_RESPONSE: key_url=/xyz/openbmc_project/software/84fb16be version=ibm-v2.1-215-g6e7eacb-r2-0-gecb3626
mid08tor03cn01: Firmware upload successful. Attempting to activate firmware: ibm-v2.1-215-g6e7eacb-r2-0-gecb3626 (ID: 84fb16be)
Wed Apr  4 01:17:57 2018 mid08tor03cn01: [openbmc_debug_perl] CHECK_ID_RESPONSE: key_url=/xyz/openbmc_project/software/78d09908 version=ibm-v2.0-0-r44-0-g843c2e1
Wed Apr  4 01:17:57 2018 mid08tor03cn01: [openbmc_debug_perl] CHECK_ID_RESPONSE: key_url=/xyz/openbmc_project/software/4f901d3d version=IBM-witherspoon-ibm-OP9_v1.19_1.154
Wed Apr  4 01:17:57 2018 mid08tor03cn01: [openbmc_debug_perl] curl -k -b cjar -X PUT -H "Content-Type: application/json" -d '{"data":"xyz.openbmc_project.Software.Activation.RequestedActivations.Active"}' https://172.21.226.1/xyz/openbmc_project/software/84fb16be/attr/RequestedActivation
Wed Apr  4 01:17:58 2018 mid08tor03cn01: [openbmc_debug_perl] rflash_update_activate_response 200 OK
Wed Apr  4 01:17:58 2018 mid08tor03cn01: [openbmc_debug_perl] curl -k -b cjar -X PUT -H "Content-Type: application/json" -d '{"data":"xyz.openbmc_project.Software.Activation.RequestedActivations.Active"}' https://172.21.226.1/xyz/openbmc_project/software/25a5c3e0/attr/RequestedActivation
Wed Apr  4 01:17:58 2018 mid08tor03cn01: [openbmc_debug_perl] rflash_update_host_activate_response 200 OK
Wed Apr  4 01:17:58 2018 mid08tor03cn01: [openbmc_debug_perl] curl -k -b cjar -X GET -H "Content-Type: application/json" https://172.21.226.1/xyz/openbmc_project/software/enumerate
Wed Apr  4 01:17:59 2018 mid08tor03cn01: [openbmc_debug_perl] rflash_update_check_state_response 200 OK
mid08tor03cn01: Firmware ibm-v2.1-215-g6e7eacb-r2-0-gecb3626 activation successful.
mid08tor03cn01: Firmware IBM-witherspoon-ibm-OP9-v1.21.2-2.6 activation successful.
Wed Apr  4 01:17:59 2018 mid08tor03cn01: [openbmc_debug_perl] curl -k -b cjar -X PUT -H "Content-Type: application/json" -d '{"data":"xyz.openbmc_project.State.BMC.Transition.Reboot"}' https://172.21.226.1/xyz/openbmc_project/state/bmc0/attr/RequestedBMCTransition
Wed Apr  4 01:18:00 2018 mid08tor03cn01: [openbmc_debug_perl] rpower_reset_response 200 OK
mid08tor03cn01: BMC reboot
Wed Apr  4 01:18:15 2018 mid08tor03cn01: [openbmc_debug_perl] curl -k -b cjar -X GET -H "Content-Type: application/json" https://172.21.226.1/xyz/openbmc_project/state/enumerate
Wed Apr  4 01:18:17 2018 mid08tor03cn01: [openbmc_debug_perl] rpower_bmc_status_response 503 Service Unavailable
mid08tor03cn01: Retry BMC state, wait for 15 seconds ...
mid08tor03cn10: Firmware upload successful. Attempting to activate firmware: IBM-witherspoon-ibm-OP9-v1.21.2-2.6 (ID: 25a5c3e0)
mid08tor03cn10: Firmware upload successful. Attempting to activate firmware: ibm-v2.1-215-g6e7eacb-r2-0-gecb3626 (ID: 84fb16be)
mid08tor03cn10: Firmware ibm-v2.1-215-g6e7eacb-r2-0-gecb3626 activation successful.
mid08tor03cn10: Firmware IBM-witherspoon-ibm-OP9-v1.21.2-2.6 activation successful.
mid08tor03cn10: BMC reboot
Wed Apr  4 01:18:32 2018 mid08tor03cn01: [openbmc_debug_perl] curl -k -b cjar -X GET -H "Content-Type: application/json" https://172.21.226.1/xyz/openbmc_project/state/enumerate
mid08tor03cn10: Retry BMC state, wait for 15 seconds ...
Wed Apr  4 01:18:50 2018 mid08tor03cn01: [openbmc_debug_perl] rpower_bmc_status_response 503 Service Unavailable
mid08tor03cn01: Retry BMC state, wait for 15 seconds ...
Wed Apr  4 01:19:05 2018 mid08tor03cn01: [openbmc_debug_perl] curl -k -b cjar -X GET -H "Content-Type: application/json" https://172.21.226.1/xyz/openbmc_project/state/enumerate
mid08tor03cn10: Retry BMC state, wait for 15 seconds ...
Wed Apr  4 01:20:05 2018 mid08tor03cn01: [openbmc_debug_perl] rpower_bmc_status_response 503 Service Unavailable
mid08tor03cn01: Retry BMC state, wait for 15 seconds ...
Wed Apr  4 01:20:20 2018 mid08tor03cn01: [openbmc_debug_perl] curl -k -b cjar -X GET -H "Content-Type: application/json" https://172.21.226.1/xyz/openbmc_project/state/enumerate
Wed Apr  4 01:20:33 2018 mid08tor03cn01: [openbmc_debug_perl] rpower_bmc_status_response 404 Not Found
mid08tor03cn01: Retry BMC state, wait for 15 seconds ...
Wed Apr  4 01:20:48 2018 mid08tor03cn01: [openbmc_debug_perl] curl -k -c cjar -H "Content-Type: application/json" -d '{ "data": ["root", "xxxxxx"] }' https://172.21.226.1/login
Wed Apr  4 01:20:48 2018 mid08tor03cn01: [openbmc_debug_perl] login_response_general 200 OK
Wed Apr  4 01:20:48 2018 mid08tor03cn01: [openbmc_debug_perl] curl -k -b cjar -X GET -H "Content-Type: application/json" https://172.21.226.1/xyz/openbmc_project/state/enumerate
Wed Apr  4 01:20:50 2018 mid08tor03cn01: [openbmc_debug_perl] rpower_bmc_status_response 200 OK
Wed Apr  4 01:21:05 2018 mid08tor03cn01: [openbmc_debug_perl] curl -k -b cjar -X GET -H "Content-Type: application/json" https://172.21.226.1/xyz/openbmc_project/state/enumerate
Wed Apr  4 01:21:05 2018 mid08tor03cn01: [openbmc_debug_perl] rpower_bmc_status_response 200 OK
mid08tor03cn01: BMC Ready
Wed Apr  4 01:21:05 2018 mid08tor03cn01: [openbmc_debug_perl] curl -k -b cjar -X PUT -H "Content-Type: application/json" -d '{"data":"xyz.openbmc_project.State.Chassis.Transition.Off"}' https://172.21.226.1/xyz/openbmc_project/state/chassis0/attr/RequestedPowerTransition
Wed Apr  4 01:21:05 2018 mid08tor03cn01: [openbmc_debug_perl] rpower_off_response 200 OK
Wed Apr  4 01:21:05 2018 mid08tor03cn01: [openbmc_debug_perl] curl -k -b cjar -X GET -H "Content-Type: application/json" https://172.21.226.1/xyz/openbmc_project/state/enumerate
Wed Apr  4 01:21:07 2018 mid08tor03cn01: [openbmc_debug_perl] rpower_check_response 200 OK
Wed Apr  4 01:21:07 2018 mid08tor03cn01: [openbmc_debug_perl] curl -k -b cjar -X PUT -H "Content-Type: application/json" -d '{"data":"xyz.openbmc_project.State.Host.Transition.On"}' https://172.21.226.1/xyz/openbmc_project/state/host0/attr/RequestedHostTransition
Wed Apr  4 01:21:09 2018 mid08tor03cn01: [openbmc_debug_perl] rpower_on_response 200 OK
mid08tor03cn01: reset
mid08tor03cn10: Retry BMC state, wait for 15 seconds ...
mid08tor03cn10: Retry BMC state, wait for 15 seconds ...
mid08tor03cn10: BMC Ready
mid08tor03cn10: reset
[root@boston02 /]# rflash mid08tor03cn01,mid08tor03cn10 -l
Wed Apr  4 01:26:23 2018 OpenBMC: [openbmc_debug_perl]
Wed Apr  4 01:26:24 2018 mid08tor03cn01: [openbmc_debug_perl] curl -k -c cjar -H "Content-Type: application/json" -d '{ "data": ["root", "xxxxxx"] }' https://172.21.226.1/login
Wed Apr  4 01:26:24 2018 mid08tor03cn01: [openbmc_debug_perl] login_response 200 OK
Wed Apr  4 01:26:24 2018 mid08tor03cn01: [openbmc_debug_perl] curl -k -b cjar -X GET -H "Content-Type: application/json" https://172.21.226.1/xyz/openbmc_project/software/enumerate
Wed Apr  4 01:26:26 2018 mid08tor03cn01: [openbmc_debug_perl] rflash_list_response 200 OK
mid08tor03cn01: ID       Purpose State      Version
mid08tor03cn01: -------------------------------------------------------
mid08tor03cn01: 25a5c3e0 Host    Active(*)  IBM-witherspoon-ibm-OP9-v1.21.2-2.6
mid08tor03cn01: 84fb16be BMC     Active(*)  ibm-v2.1-215-g6e7eacb-r2-0-gecb3626
mid08tor03cn01: 78d09908 BMC     Active     ibm-v2.0-0-r44-0-g843c2e1
mid08tor03cn01: 4f901d3d Host    Active     IBM-witherspoon-ibm-OP9_v1.19_1.154
mid08tor03cn01:
mid08tor03cn10: ID       Purpose State      Version
mid08tor03cn10: -------------------------------------------------------
mid08tor03cn10: 25a5c3e0 Host    Active(*)  IBM-witherspoon-ibm-OP9-v1.21.2-2.6
mid08tor03cn10: 84fb16be BMC     Active(*)  ibm-v2.1-215-g6e7eacb-r2-0-gecb3626
mid08tor03cn10: 78d09908 BMC     Active     ibm-v2.0-0-r44-0-g843c2e1
mid08tor03cn10: 4f901d3d Host    Active     IBM-witherspoon-ibm-OP9_v1.19_1.154
mid08tor03cn10:

@robin2008
Copy link
Member

robin2008 commented Apr 4, 2018 •
edited
Loading

This is the same as 2.13.11, may not be a block issue for pre-GA, document if possible.

@bybai
Copy link
Contributor

bybai commented Apr 4, 2018

I retried several times, cannot reproduce this issue. Based on issue log and code, I guess mid08tor03cn01 give response 200, but "curl -k -b cjar -X GET -H "Content-Type: application/json" https://172.21.226.1/xyz/openbmc_project/state/enumerate" contain unexpected results, or mid08tor03cn01 is busy. This issue is accidental happen, hope to find a good way to reproduce it.

[root@boston02 /]# rflash mid08tor03cn01,mid08tor03cn10 -d /firm 
Attempting to upload /firm/obmc-phosphor-image-witherspoon.ubi.mtd.tar and /firm/witherspoon.pnor.squashfs.tar, please wait...
Attempting to upload /firm/obmc-phosphor-image-witherspoon.ubi.mtd.tar and /firm/witherspoon.pnor.squashfs.tar, please wait...
mid08tor03cn10: Firmware upload successful. Attempting to activate firmware: IBM-witherspoon-ibm-OP9-v1.21.2-2.6 (ID: 25a5c3e0)
mid08tor03cn10: Firmware upload successful. Attempting to activate firmware: ibm-v2.1-215-g6e7eacb-r2-0-gecb3626 (ID: 84fb16be)
mid08tor03cn01: Firmware upload successful. Attempting to activate firmware: IBM-witherspoon-ibm-OP9-v1.21.2-2.6 (ID: 25a5c3e0)
mid08tor03cn01: Firmware upload successful. Attempting to activate firmware: ibm-v2.1-215-g6e7eacb-r2-0-gecb3626 (ID: 84fb16be)
mid08tor03cn10: Firmware ibm-v2.1-215-g6e7eacb-r2-0-gecb3626 activation successful.
mid08tor03cn10: Firmware IBM-witherspoon-ibm-OP9-v1.21.2-2.6 activation successful.
mid08tor03cn10: BMC reboot
mid08tor03cn01: Firmware ibm-v2.1-215-g6e7eacb-r2-0-gecb3626 activation successful.
mid08tor03cn01: Firmware IBM-witherspoon-ibm-OP9-v1.21.2-2.6 activation successful.
mid08tor03cn01: BMC reboot
mid08tor03cn01: Retry BMC state, wait for 15 seconds ...
mid08tor03cn10: Retry BMC state, wait for 15 seconds ...
mid08tor03cn01: Retry BMC state, wait for 15 seconds ...
mid08tor03cn10: Retry BMC state, wait for 15 seconds ...
mid08tor03cn10: Retry BMC state, wait for 15 seconds ...
mid08tor03cn01: Retry BMC state, wait for 15 seconds ...
mid08tor03cn10: BMC Ready
mid08tor03cn10: reset
mid08tor03cn01: BMC Ready
mid08tor03cn01: reset

@bybai
Copy link
Contributor

bybai commented Apr 4, 2018 •
edited
Loading

The work around for this issue:
1, check the bmc state, found "Active(+) IBM-witherspoon-ibm-OP9_v1.19_1.154" on mid08tor03cn01

[root@boston02 /]# rflash mid08tor03cn01 -l
mid08tor03cn01: ID       Purpose State      Version
mid08tor03cn01: -------------------------------------------------------
mid08tor03cn01: 25a5c3e0 Host    Active(*)  IBM-witherspoon-ibm-OP9-v1.21.2-2.6
mid08tor03cn01: 78d09908 BMC     Active(*)  ibm-v2.0-0-r44-0-g843c2e1
mid08tor03cn01: 63c95399 BMC     Active     ibm-v2.0-0-r13.4-0-g7fd26e9
mid08tor03cn01: 4f901d3d Host    Active(+)  IBM-witherspoon-ibm-OP9_v1.19_1.154
mid08tor03cn01:

2, reboot the mid08tor03cn01:

rpower mid08tor03cn01 boot

@bybai bybai closed this as completed Apr 10, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bybai bybai

Labels
component:hw_control sprint1 status:won't fix type:bug
Projects
None yet
Milestone
2.14
Development

No branches or pull requests
3 participants
@bybai @tingtli @robin2008