Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace usage of pkg_resources (setuptools) with importlib(._)resources? #2927

Closed
jenshnielsen opened this issue Sep 8, 2022 · 1 comment · Fixed by #3047
Closed

Replace usage of pkg_resources (setuptools) with importlib(._)resources? #2927

jenshnielsen opened this issue Sep 8, 2022 · 1 comment · Fixed by #3047
Assignees

Comments

@jenshnielsen
Copy link
Contributor

Is your feature request related to a problem?

Importing opentelemetry is rather slow due to the import of pkg_resources in opentelemetry.context here

It is a known shortcoming of pkg_resources that at import time it will pass all installed packages which is potentially slow.
In my use case it adds ~0.7 s to the import time.

For this reason setuptools actively discourages the use of pkg_resources See here

Describe the solution you'd like

Replace the use of pkg_resources with importlib.metadata / importlib.resources which is part of the python standard library in newer versions of python and use their backports in importlib_metadata / importlib_resources as needed for older python versions.

In this particular use case pkg_resources is used to find entry points and that should be replaceable by doing

from importlib_metadata import entry_points
...
(entry_point,) = entry_points(group="opentelemetry_context", name=configured_context)
_RUNTIME_CONTEXT = entry_point.load()

Describe alternatives you've considered

  • Keep the current state. Using code that is slow and discouraged by its owners.

Additional context

This will also mean that opentelemetry can potentially drop the need for a runtime dependency on setuptools and when older python versions (<3.9) are dropped only use the std library for entry point selection.

@jenshnielsen
Copy link
Contributor Author

Clarification:

This is not the only place pkg_resources is used but just the first. The others should be replaceable in the same way.

❯ rg pkg_resources
opentelemetry-semantic-conventions\tests\test_semconv.py
18:from pkg_resources import DistributionNotFound, require

opentelemetry-sdk\src\opentelemetry\sdk\_configuration\__init__.py
26:from pkg_resources import iter_entry_points

opentelemetry-sdk\src\opentelemetry\sdk\resources\__init__.py
66:import pkg_resources
138:_OPENTELEMETRY_SDK_VERSION = pkg_resources.get_distribution(

opentelemetry-sdk\src\opentelemetry\sdk\error_handler\__init__.py
65:from pkg_resources import iter_entry_points

opentelemetry-proto\tests\test_proto.py
18:from pkg_resources import DistributionNotFound, require

opentelemetry-api\tests\__init__.py
14:import pkg_resources
22:pkg_resources.declare_namespace(__name__)

opentelemetry-api\src\opentelemetry\util\_providers.py
19:from pkg_resources import iter_entry_points

opentelemetry-api\src\opentelemetry\trace\__init__.py
219:                ``pkg_resources.get_distribution(instrumenting_library_name).version``.

opentelemetry-api\src\opentelemetry\propagate\__init__.py
75:from pkg_resources import iter_entry_points

opentelemetry-api\src\opentelemetry\context\__init__.py
22:from pkg_resources import iter_entry_points

opentelemetry-api\src\opentelemetry\metrics\_internal\__init__.py
122:                ``pkg_resources.get_distribution(instrumenting_library_name).version``.

opentelemetry-api\tests\propagators\test_propagators.py
56:    @patch("pkg_resources.iter_entry_points")

docs-requirements.txt
8:# doesn't work for pkg_resources.

exporter\opentelemetry-exporter-opencensus\src\opentelemetry\exporter\opencensus\util.py
19:import pkg_resources
29:OPENTELEMETRY_VERSION = pkg_resources.get_distribution(

@ocelotl ocelotl self-assigned this Nov 21, 2022
ocelotl added a commit to ocelotl/opentelemetry-python that referenced this issue Nov 23, 2022
ocelotl added a commit to ocelotl/opentelemetry-python that referenced this issue Nov 23, 2022
ocelotl added a commit to ocelotl/opentelemetry-python that referenced this issue Nov 30, 2022
ocelotl added a commit to ocelotl/opentelemetry-python that referenced this issue Dec 1, 2022
ocelotl added a commit to ocelotl/opentelemetry-python that referenced this issue Dec 5, 2022
ocelotl added a commit to ocelotl/opentelemetry-python that referenced this issue Dec 6, 2022
ocelotl added a commit to ocelotl/opentelemetry-python that referenced this issue Dec 14, 2022
ocelotl added a commit to ocelotl/opentelemetry-python that referenced this issue Jan 18, 2023
ocelotl added a commit to ocelotl/opentelemetry-python that referenced this issue Jan 20, 2023
ocelotl added a commit to ocelotl/opentelemetry-python that referenced this issue Feb 1, 2023
ocelotl added a commit that referenced this issue Feb 1, 2023
* Remove usage of pkg_resources

Fixes #2927

* WIP

* WIP

* Add test case for _load_providers

* Fix test case for 3.9

* Fix lint

* Fix 3.7 sdk tests

* Fix lint

* Fix lint

* Fix mypy again

* WIP

* WIP

* WIP

* WIP

* Move to module

* Fix SDK

* Fix mypy

* Refactor load call

* Fix mypy

* Fix lint

* Fix opencensus exporter

* Refactor implementation

* Add missing dependency

* Undo changes in shim

* Fix dependency

* Revert "Undo changes in shim"

This reverts commit bd82b8f.

* Update dependencies

* Update dependency for opencensus exporter

* Add descriptive error
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants