Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

get_package in Zenodo adapter out of date? #1682

Open
jonaslenz opened this issue Sep 9, 2024 · 0 comments
Open

get_package in Zenodo adapter out of date? #1682

jonaslenz opened this issue Sep 9, 2024 · 0 comments

Comments

@jonaslenz
Copy link

Getting packages from zenodo fails.

The example on https://framework.frictionlessdata.io/docs/portals/zenodo.html

from pprint import pprint
from frictionless import portals, Package

package = Package("https://zenodo.org/record/7078760")
package.infer()
print(package)

fails with the error FrictionlessException: [error] Package/s not found
This might be related to #1607.

It seems that due to an update of the zenodo api by now the string "/content" is appended to every file-URL. (e.g. https://zenodo.org/api/records/7078760/files/datapackage.json/content). I'm not very experienced in frictionless code, but tried a little:

One could delete "/content" from the URL in the get_package() for file recognition and reappend it for downloading:

def get_package(record: Record, title: str, formats: List[str]) -> Package:  # type: ignore
    package = Package(title=title)
    package.title = title
    for file in record.data["files"]:  # type: ignore
        path = file["links"]["self"] # type: ignore
<<<<<
        path = path.replace("/content", "") # delete "/content" from path url
>>>>>
        is_resource_file = any(path.endswith(ext) for ext in formats)  # type: ignore
        if path.endswith(("datapackage.json")):  # type: ignore
<<<<<
            return Package.from_descriptor(path+"/content", title=title)  # type: ignore, reappend "/content"
>>>>>
        if path.endswith("zip") and not is_resource_file:  # type: ignore
            try:
<<<<<
                package = Package(path+"/content")  # type: ignore, reappend "/content"
>>>>>
                package.title = title
                return package
            except FrictionlessException as exception:
                # Skips package descriptor not found exception
                # and continues reading files.
                if "[Errno 2] No such file or directory" not in str(exception):
                    raise exception
        if is_resource_file:
            package.basepath = f'https://zenodo.org/api/files/{file["bucket"]}'
            resource = Resource(path=file["key"])  # type: ignore
            package.add_resource(resource)
    return package

But at least their remains an issue with the if is_resource_file: - The Key "bucket" does not exist.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant