Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

regression in processor.resolve_resource: module may be a namespace package only #916

Closed
bertsky opened this issue Oct 5, 2022 · 1 comment · Fixed by #917
Closed

regression in processor.resolve_resource: module may be a namespace package only #916

bertsky opened this issue Oct 5, 2022 · 1 comment · Fixed by #917

Comments

@bertsky
Copy link
Collaborator

bertsky commented Oct 5, 2022

Since #904 we look up …

def module(self):
"""
The top-level module this processor belongs to.
"""
return self.__module__.split('.')[0]
… to determine the moduledir location.

But that fails if the processor module itself is distributed via PEP-420 namespace packages (so the top level does not contain any resources).

For example, in case of ocrd-eynollah-segment, we have qurator.eynollah as the actual processor module, but qurator itself is empty:

  File "/data/ocr-d/ocrd_all/venv/lib/python3.7/site-packages/qurator/eynollah/processor.py", line 47, in process
    'dir_models': self.resolve_resource(self.parameter['models']),
  File "/data/ocr-d/ocrd_all/venv/lib/python3.7/site-packages/ocrd/processor/base.py", line 219, in resolve_resource
    cwd=cwd, moduled=self.moduledir)
  File "/data/ocr-d/ocrd_all/venv/lib/python3.7/site-packages/ocrd/processor/base.py", line 257, in moduledir
    return resource_filename(self.module, '')
  File "/data/ocr-d/ocrd_all/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1154, in resource_filename
    return get_provider(package_or_requirement).get_resource_filename(
  File "/data/ocr-d/ocrd_all/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 361, in get_provider
    return _find_adapter(_provider_factories, loader)(module)
  File "/data/ocr-d/ocrd_all/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1403, in __init__
    self.module_path = os.path.dirname(getattr(module, '__file__', ''))
  File "/usr/lib/python3.7/posixpath.py", line 156, in dirname
    p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType

I don't know what's the correct way to deal with this, but it seems to suffice checking that a .__file__ exists at the module level – for namespaces this is None.

@bertsky
Copy link
Collaborator Author

bertsky commented Oct 5, 2022

I don't know what's the correct way to deal with this, but it seems to suffice checking that a .__file__ exists at the module level – for namespaces this is None.

Judging by the documentation of __file__ and __package__, I do think this is the test we have to make: traverse the module path from top to current and stop as soon as we find a package that has_location.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant