Improve service checker for gnmi/telemetry container #19153
Merged
+89
−8
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ganglyu
changed the title
|
/azpw ms_conflict |
|
@qiluo-msft , can you please review this PR ? an if it is approved, can you please merge ? |
|
@qiluo-msft kindly reminder to review. it should go to 202311 and 202405 |
| """ | ||
| try: | ||
| DOCKER_CLIENT = docker.DockerClient(base_url='unix://var/run/docker.sock') | ||
| DOCKER_CLIENT.images.get(image_name) |
There was a problem hiding this comment.
Is there an assumption that images.get function will drop an exception when image is not there? Should we also check what it returns other than depend the logic of images.get function? For example, if the images.get refined to return a null for a search miss, it would still return True by check_docker_image. @ganglyu
There was a problem hiding this comment.
This function will raise exception if the image does not exist:
https://docker-py.readthedocs.io/en/stable/images.html#docker.models.images.ImageCollection.get
Raises:
docker.errors.ImageNotFound – If the image does not exist.
docker.errors.APIError – If the server returns an error.
StormLiangMS
approved these changes
Jun 12, 2024
There was a problem hiding this comment.
this change looks like more a work around, but I'm ok with it to deal with the difficulty of warm reboot case.
@ganglyu could you help to update the description about the details of difficulty of warm reboot case?
|
@qiluo-msft to take a look before merge. |
|
@qiluo-msft , Can you please approve and merge ? |
mssonicbld
pushed a commit
to mssonicbld/sonic-buildimage
that referenced
this pull request
Jun 18, 2024
Why I did it Fix sonic-net#19081 We have used gnmi container to replace telemetry container, and telemetry is still enabled after upgrade. service_checker script reads from features table and check if the container is running, telemetry is enabled but there's no telemetry container. It's difficult to disable telemetry in feature table for warm reboot and cold reboot, we need to check docker image in db migrator and minigraph.py. When we use warm reboot to upgrade from 202305 to 202311, config_db still has telemetry configuration, and we can't simply remove related configuration. Work item tracking Microsoft ADO (number only): How I did it I modify service_checker script: If there's docker-sonic-telemetry image, check telemetry container. If there's no docker-sonic-telemetry image, check gnmi container instead. If there's no docker-sonic-telemetry image and docker-sonic-gnmi image, do not check telemetry. How to verify it Run unit test and end to end test.
|
Cherry-pick PR to 202405: #19332 |
mssonicbld
pushed a commit
that referenced
this pull request
Jun 18, 2024
Why I did it Fix #19081 We have used gnmi container to replace telemetry container, and telemetry is still enabled after upgrade. service_checker script reads from features table and check if the container is running, telemetry is enabled but there's no telemetry container. It's difficult to disable telemetry in feature table for warm reboot and cold reboot, we need to check docker image in db migrator and minigraph.py. When we use warm reboot to upgrade from 202305 to 202311, config_db still has telemetry configuration, and we can't simply remove related configuration. Work item tracking Microsoft ADO (number only): How I did it I modify service_checker script: If there's docker-sonic-telemetry image, check telemetry container. If there's no docker-sonic-telemetry image, check gnmi container instead. If there's no docker-sonic-telemetry image and docker-sonic-gnmi image, do not check telemetry. How to verify it Run unit test and end to end test.
mssonicbld
added
Included in 202405 Branch
and removed
Created PR to 202405 Branch
labels
|
@yxieca , Can you please cherry pick to 202311 ? |
|
@ganglyu can you provide the ADO number? |
mssonicbld
pushed a commit
to mssonicbld/sonic-buildimage
that referenced
this pull request
Jun 20, 2024
Why I did it Fix sonic-net#19081 We have used gnmi container to replace telemetry container, and telemetry is still enabled after upgrade. service_checker script reads from features table and check if the container is running, telemetry is enabled but there's no telemetry container. It's difficult to disable telemetry in feature table for warm reboot and cold reboot, we need to check docker image in db migrator and minigraph.py. When we use warm reboot to upgrade from 202305 to 202311, config_db still has telemetry configuration, and we can't simply remove related configuration. Work item tracking Microsoft ADO (number only): How I did it I modify service_checker script: If there's docker-sonic-telemetry image, check telemetry container. If there's no docker-sonic-telemetry image, check gnmi container instead. If there's no docker-sonic-telemetry image and docker-sonic-gnmi image, do not check telemetry. How to verify it Run unit test and end to end test.
|
Cherry-pick PR to 202311: #19360 |
yxieca
pushed a commit
that referenced
this pull request
Jun 21, 2024
Why I did it Fix #19081 We have used gnmi container to replace telemetry container, and telemetry is still enabled after upgrade. service_checker script reads from features table and check if the container is running, telemetry is enabled but there's no telemetry container. It's difficult to disable telemetry in feature table for warm reboot and cold reboot, we need to check docker image in db migrator and minigraph.py. When we use warm reboot to upgrade from 202305 to 202311, config_db still has telemetry configuration, and we can't simply remove related configuration. Work item tracking Microsoft ADO (number only): How I did it I modify service_checker script: If there's docker-sonic-telemetry image, check telemetry container. If there's no docker-sonic-telemetry image, check gnmi container instead. If there's no docker-sonic-telemetry image and docker-sonic-gnmi image, do not check telemetry. How to verify it Run unit test and end to end test. Co-authored-by: ganglv <88995770+ganglyu@users.noreply.github.com>
arun1355492
pushed a commit
to arun1355492/sonic-buildimage
that referenced
this pull request
Jul 26, 2024
Why I did it Fix sonic-net#19081 We have used gnmi container to replace telemetry container, and telemetry is still enabled after upgrade. service_checker script reads from features table and check if the container is running, telemetry is enabled but there's no telemetry container. It's difficult to disable telemetry in feature table for warm reboot and cold reboot, we need to check docker image in db migrator and minigraph.py. When we use warm reboot to upgrade from 202305 to 202311, config_db still has telemetry configuration, and we can't simply remove related configuration. Work item tracking Microsoft ADO (number only): How I did it I modify service_checker script: If there's docker-sonic-telemetry image, check telemetry container. If there's no docker-sonic-telemetry image, check gnmi container instead. If there's no docker-sonic-telemetry image and docker-sonic-gnmi image, do not check telemetry. How to verify it Run unit test and end to end test.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why I did it
Fix #19081
We have used gnmi container to replace telemetry container, and telemetry is still enabled after upgrade.
service_checker script reads from features table and check if the container is running, telemetry is enabled but there's no telemetry container.
It's difficult to disable telemetry in feature table for warm reboot and cold reboot, we need to check docker image in db migrator and minigraph.py.
When we use warm reboot to upgrade from 202305 to 202311, config_db still has telemetry configuration, and we can't simply remove related configuration.
Work item tracking
How I did it
I modify service_checker script:
If there's docker-sonic-telemetry image, check telemetry container.
If there's no docker-sonic-telemetry image, check gnmi container instead.
If there's no docker-sonic-telemetry image and docker-sonic-gnmi image, do not check telemetry.
How to verify it
Run unit test and end to end test.
Which release branch to backport (provide reason below if selected)
Tested branch (Please provide the tested image version)
Description for the changelog
Link to config_db schema for YANG module changes
A picture of a cute animal (not mandatory but encouraged)