Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add port-specific modules to support matrix #7725

Merged

Conversation

dhalbert
Copy link
Collaborator

@dhalbert dhalbert commented Mar 14, 2023

Also ran docs/shared_bindings_matrix.py through black.

@Neradoc
Copy link

Neradoc commented Mar 14, 2023

When trying to test it locally, I get an error because the content is empty when trying to retrieve the previous files.json.

RELEASE_TAG=8.0.0 DEBUG=1 python build_board_info.py
Not logged in
Traceback (most recent call last):
  File "/Volumes/circuitpython-dev/repositories/test-pr/tools/build_board_info.py", line 268, in <module>
    generate_download_info()
  File "/Volumes/circuitpython-dev/repositories/test-pr/tools/build_board_info.py", line 207, in generate_download_info
    git_info, current_info = get_current_info()
  File "/Volumes/circuitpython-dev/repositories/test-pr/tools/build_board_info.py", line 103, in get_current_info
    current_list = json.loads(base64.b64decode(response["content"]).decode("utf-8"))
  File "/usr/local/Cellar/python@3.9/3.9.16/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/local/Cellar/python@3.9/3.9.16/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/Cellar/python@3.9/3.9.16/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

I believe the reason is that the file is now above 1MB:
https://docs.github.com/en/rest/repos/contents#size-limits

Between 1-100 MB: Only the raw or object custom media types are supported. Both will work as normal, except that when using the object media type, the content field will be an empty string and the encoding field will be "none". To get the contents of these larger files, use the raw media type.

Calling it like this retrieves just the file, but not the associated sha:

    response = github.get(
        "/repos/adafruit/circuitpython-org/contents/_data/files.json?ref=" + commit_sha,
        headers={"Accept": "application/vnd.github.raw+json"}
    )

Or we can retrieve the file if the content field is empty:

    if response["content"] != "":
        # if the file is there
        current_list = json.loads(base64.b64decode(response["content"]).decode("utf-8"))
    else:
        # if too big, the file is not included
        download_url = response["download_url"]
        response = requests.get(download_url)
        if not response.ok:
            print(response.text)
            raise RuntimeError("cannot get previous files.json")
        current_list = response.json()

With that, the PR does as expected.

@dhalbert
Copy link
Collaborator Author

The problem @Neradoc observed is now happening even before this PR is merged:
https://github.com/adafruit/circuitpython/actions/runs/4421756938/jobs/7752932114#step:8:1
This is on my 8.0.4 release.

@dhalbert
Copy link
Collaborator Author

@Neradoc I tried both of the above patches to build_board_info.py by hand, but the problem I encountered is that I need response["sha"], and neither gets that. However, I may have done something wrong. Could you show me your patched build_board_info.py.

@Neradoc
Copy link

Neradoc commented Mar 15, 2023

Here it is:
8.0.x...Neradoc:get-files-json-when-big

Copy link
Collaborator

@microdev1 microdev1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks, @dhalbert and @Neradoc.

@microdev1 microdev1 merged commit 76d590b into adafruit:main Mar 15, 2023
@dhalbert dhalbert deleted the port-specific-modules-in-support-matrix branch March 29, 2023 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

port-specific modules not included in module support matrix
3 participants