Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sdk: Implement basic os resource detector #3992

Merged
merged 18 commits into from
Aug 1, 2024

Conversation

Zirak
Copy link
Contributor

@Zirak Zirak commented Jun 23, 2024

Description

Implement basic os resource detector.

Based on OS resource semantics: https://opentelemetry.io/docs/specs/semconv/resource/os/

Currently implements os.type and os.version, attempting to be in line with
what's reported by other runtimes (like java and node).

I have not yet tested on some more exotic OSs such as hp-ux, aix, or z/os.

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

  • Checked platform.system and platform.release on a variety of operating systems
  • Ran java and node agents in similar environments, seeing values are in alignment
  • Replicated non-trivial cases as unit test patches

Does This PR Require a Contrib Repo Change?

Answer the following question based on these examples of changes that would require a Contrib Repo Change:

  • The OTel specification has changed which prompted this PR to update the method interfaces of opentelemetry-api/ or opentelemetry-sdk/

  • The method interfaces of test/util have changed

  • Scripts in scripts/ that were copied over to the Contrib repo have changed

  • Configuration files that were copied over to the Contrib repo have changed (when consistency between repositories is applicable) such as in

    • pyproject.toml
    • isort.cfg
    • .flake8
  • When a new .github/CODEOWNER is added

  • Major changes to project information, such as in:

    • README.md
    • CONTRIBUTING.md
  • Yes. - Link to PR:

  • No.

Checklist:

  • Followed the style guidelines of this project
  • Changelogs have been updated
  • Unit tests have been added
  • Documentation has been updated

I'm unsure what the practice to do the two items above (changelog & documentation) would actually require. It seems like opening a PR is a prerequisite to generating a changelog. I haven't seen any special documentation around resource detectors. Is my understanding correct?

(Edit: Commits have since made sure to add the CHANGELOG entry and write proper pydocs around implementation)

Based on OS resource semantics: https://opentelemetry.io/docs/specs/semconv/resource/os/

Currently implements `os.type` and `os.version`, attempting to be in line with
what's reported by other runtimes (like java and node).
@Zirak Zirak requested a review from a team June 23, 2024 06:34
Copy link
Contributor

@xrmx xrmx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Give than os.type is required we should probably make the resource detector mandatory in https://github.com/open-telemetry/opentelemetry-python/pull/3992/files#diff-d2c46fd8da8afdf67c6ad3dd42b0a243c5049f3c720d9bb3351f1d5522338b06R182-R187
and in order to give it a name you need to add an entry in the sdk pyproject.toml inside project.entry-points.opentelemetry_resource_detector

@Zirak
Copy link
Contributor Author

Zirak commented Jun 25, 2024

Great points, thank you! I've completely missed the entrypoint.

Regarding making it a default, I've done the following simple diff:

@@ -180,11 +180,13 @@ class Resource:
         resource = _DEFAULT_RESOURCE
 
         otel_experimental_resource_detectors = environ.get(
-            OTEL_EXPERIMENTAL_RESOURCE_DETECTORS, "otel"
+            OTEL_EXPERIMENTAL_RESOURCE_DETECTORS, "otel,os"
         ).split(",")
 
         if "otel" not in otel_experimental_resource_detectors:
             otel_experimental_resource_detectors.append("otel")
+        if "os" not in otel_experimental_resource_detectors:
+            otel_experimental_resource_detectors.append("os")
 
         for resource_detector in otel_experimental_resource_detectors:
             resource_detectors.append(

Running locally it looks good (yay!), but I'm having trouble with testing. A lot of things expect _DEFAULT_RESOURCE to be the baseline for all future resources. I'm currently tinkering with decorating the entire TestResources with a platform.uname patch, alongside with extending _DEFAULT_RESOURCE as part of __init__ and rewriting existing test cases to use it (Edit: This has since been pushed), e.g.

@@ -61,12 +62,26 @@ except ImportError:
     psutil = None
 
 
+@patch("platform.uname", lambda: platform.uname_result(
+            system="Linux",
+            node="node",
+            release="1.2.3",
+            version="4.5.6",
+            machine="x86_64",
+            processor="x86_64"
+        ))
 class TestResources(unittest.TestCase):
     def setUp(self) -> None:
         environ[OTEL_RESOURCE_ATTRIBUTES] = ""
+        self.mock_platform = {
+            OS_TYPE: "linux",
+            OS_VERSION: "1.2.3",
+        }
+        self.default_resource = _DEFAULT_RESOURCE.merge(Resource(self.mock_platform))
@@ -86,6 +101,7 @@ class TestResources(unittest.TestCase):
             TELEMETRY_SDK_VERSION: _OPENTELEMETRY_SDK_VERSION,
             SERVICE_NAME: "unknown_service",
         }
+        expected_attributes.update(self.mock_platform)
@@ -431,7 +447,7 @@ class TestResources(unittest.TestCase):
         resource_detector.raise_on_error = False
         self.assertEqual(
             get_aggregated_resources([resource_detector]),
-            _DEFAULT_RESOURCE.merge(
+            self.default_resource.merge(

It feels a bit icky. Am I missing a better, simpler way?

@Zirak Zirak requested a review from ocelotl June 30, 2024 16:56
@xrmx
Copy link
Contributor

xrmx commented Jul 1, 2024

@Zirak Please add an entry in the changelog

@xrmx xrmx added the Approve Public API check This label shows that the public symbols added or changed in a PR are strictly necessary label Jul 3, 2024
@Zirak
Copy link
Contributor Author

Zirak commented Jul 3, 2024

Noticed the tests failing on python 3.8 - that's strange, will take a look

@xrmx
Copy link
Contributor

xrmx commented Jul 9, 2024

@Zirak lint and docs are failing too

@Zirak
Copy link
Contributor Author

Zirak commented Jul 14, 2024

Apologies for the wait, life got in the way. I've pushed 4 commits:

  • Catching up with main (lmk if there's another preferred way of doing so)
  • Linting, I somehow missed that in the commit amendments
  • Fix the code on python 3.8 (including pypy), very good catch from the robots
  • Actually write docs in rst, my first real time writing in rst so it was an adventure, lmk if it can be improved

@lzchen
Copy link
Contributor

lzchen commented Jul 15, 2024

I'm not sure if I agree with @xrmx 's comment regarding making the resource detector loaded by default. We have OtelResourceDetector loaded by default and it populates service.name by default which is already marked stable as an attribute. Even though some of the fields are required, I believe this is required IF the resource detector exists, not that it is required by default as part of the sdk. The attributes are also marked as experimental in the sem conv so not too eager to have this default behavior until it is stable.

Copy link
Contributor

@lzchen lzchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment about defaulting logic.

@Zirak
Copy link
Contributor Author

Zirak commented Jul 15, 2024

Thanks @lzchen. How are discussions like this usually handled? Comments here, the CNCF slack, SIG topic, etc.? I'd be happy to present it at a SIG if necessary.

@xrmx
Copy link
Contributor

xrmx commented Jul 16, 2024

I'm not sure if I agree with @xrmx 's comment regarding making the resource detector loaded by default. We have OtelResourceDetector loaded by default and it populates service.name by default which is already marked stable as an attribute. Even though some of the fields are required, I believe this is required IF the resource detector exists, not that it is required by default as part of the sdk. The attributes are also marked as experimental in the sem conv so not too eager to have this default behavior until it is stable.

I'm fine on not making it enabled by default

@xrmx xrmx self-requested a review July 16, 2024 14:57
@Zirak
Copy link
Contributor Author

Zirak commented Jul 17, 2024

Coolio, will revert it to not be a default

@Zirak Zirak requested a review from lzchen July 23, 2024 12:11
@Zirak
Copy link
Contributor Author

Zirak commented Jul 25, 2024

@lzchen It's no longer a default, could you take another look?

Or @xrmx, what's the next step?

CHANGELOG.md Outdated Show resolved Hide resolved
@lzchen lzchen merged commit e16dc48 into open-telemetry:main Aug 1, 2024
284 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Approve Public API check This label shows that the public symbols added or changed in a PR are strictly necessary
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants