Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accept pathlib.Path in loaders and dumpers #2458

Open
multimeric opened this issue Dec 14, 2024 · 2 comments
Open

Accept pathlib.Path in loaders and dumpers #2458

multimeric opened this issue Dec 14, 2024 · 2 comments
Assignees
Labels
community-generated developer-days smallish tickets that can be considered "maintenance" and fixed within a single session generator-misc Pertaining to more than one generator, or perhaps one that doesn't exist yet

Comments

@multimeric
Copy link

What is your feature request?
The loaders and dumpers don't seem to understand pathlike types other than strings.

from linkml_runtime.loaders import JSONLoader
from pathlib import Path
JSONLoader().load(Path("example.json"), dict)

You get:

TypeError: object of type 'PosixPath' has no len()

Full traceback:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/migwell/Programming/json-ld-test/.venv/lib/python3.12/site-packages/linkml_runtime/loaders/loader_root.py", line 76, in load
    results = self.load_any(*args, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/migwell/Programming/json-ld-test/.venv/lib/python3.12/site-packages/linkml_runtime/loaders/json_loader.py", line 32, in load_any
    data_as_dict = self.load_as_dict(source, base_dir=base_dir, metadata=metadata)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/migwell/Programming/json-ld-test/.venv/lib/python3.12/site-packages/linkml_runtime/loaders/json_loader.py", line 21, in load_as_dict
    data = self._read_source(source, base_dir=base_dir, metadata=metadata, accept_header="application/ld+json, application/json, text/json")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/migwell/Programming/json-ld-test/.venv/lib/python3.12/site-packages/linkml_runtime/loaders/loader_root.py", line 167, in _read_source
    data = hbread(source, metadata, base_dir, accept_header)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/migwell/Programming/json-ld-test/.venv/lib/python3.12/site-packages/hbreader/__init__.py", line 258, in hbread
    open_info.source_file_size = len(source)
                                 ^^^^^^^^^^^
TypeError: object of type 'PosixPath' has no len()

How important is this feature to you? Select from the options below:
Low

Additional context
I believe the best practice way to do this is not to add a special case for Path, but rather to accept str, bytes or os.PathLike as described here: https://docs.python.org/3/glossary.html#term-path-like-object.

@dalito
Copy link
Collaborator

dalito commented Dec 14, 2024

The traceback is from an external lib that is not accepting Pathlib objects which was already reported there a while ago: hsolbrig/hbreader#11 - Since it hasn't been fixed yet, LinkML would better convert to str before calling hbreader.

On your additional context: When interfacing external code, being tolerant on how the path is received/passed is good. However, within LinkML the consistent use of Pathlib objects is helpful to detect problems and from what I observe it is gradually going into that direction.

@nlharris nlharris added community-generated generator-misc Pertaining to more than one generator, or perhaps one that doesn't exist yet labels Dec 16, 2024
@sierra-moxon sierra-moxon self-assigned this Dec 20, 2024
@sierra-moxon sierra-moxon added the developer-days smallish tickets that can be considered "maintenance" and fixed within a single session label Dec 20, 2024
@sierra-moxon
Copy link
Member

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
community-generated developer-days smallish tickets that can be considered "maintenance" and fixed within a single session generator-misc Pertaining to more than one generator, or perhaps one that doesn't exist yet
Projects
None yet
Development

No branches or pull requests

4 participants