python: Add defconfig for RISC-V QEMU and Documentation entries for Python on NuttX

[tmedicci]

Summary

• risc-v/qemu-rv/rv-virt: Add CPython (python) defconfig: enables building NuttX's port of Python for RISC-V QEMU.
• Documentation: Add entry for Python on NuttX

Impact

Document how to build Python for NuttX (added by apache/nuttx-apps#2879) on RISC-V QEMU.

Testing

Tested building rv-virt:python with the instructions at apache/nuttx-apps#2879 and the documentation with make html.

[nuttxpr]

[Experimental Bot, please feedback here]

Yes, this PR appears to meet the NuttX requirements, although some sections could be more explicit. While the information is present, expanding on some points would improve clarity and completeness.

Here's a breakdown:

• Summary: Good, clearly states the "what" and "why". Mentioning the specific files changed (e.g., configs/rv-virt/python/defconfig) would be beneficial.

• Impact: Needs more detail. While it mentions documentation updates, it should explicitly answer all the questions with "YES" or "NO" followed by a description. For example:

  • Is new feature added? YES (Python support for RISC-V QEMU)
  • Is existing feature changed? NO
  • Impact on user... YES (Users can now build and run Python on RISC-V QEMU)
  • Impact on build... YES (New defconfig added)
  • Impact on hardware... NO
  • Impact on documentation... YES (Documentation added explaining how to build and use Python)
  • Impact on security... Potentially YES (Adding Python could introduce new security considerations depending on its usage). This needs careful review.
  • Impact on compatibility... NO (or YES if there are any known compatibility issues)

• Testing: Needs significant improvement. Simply linking to another PR isn't sufficient. Provide specific build host and target details, and include relevant snippets of the actual logs, demonstrating the functionality before and after the change. For example:

  • Build Host(s): macOS Ventura, Apple M1 Pro, clang version 14.0.0

  • Target(s): QEMU-RV (riscv64)

  • Testing logs before change: Attempting to build make rv-virt_defconfig would previously result in an error because the python defconfig did not exist. (show the error message)

  • Testing logs after change: make rv-virt_python_defconfig now builds successfully. (Show a snippet of the successful build output, and ideally, a snippet showing Python running on QEMU).

By providing more specific information in the Impact and Testing sections, the PR review process will be much smoother and faster.

[tmedicci]

About the CI, It's failing because of the flags used to build Python. I can fix them, but I'm afraid that it'd take too long to build the test because it requires building python for the host too. @lupyuen , do you have any suggestions on how to skip this test (or at least running it only from time to time)?

[lupyuen]

@tmedicci Yep could you add this patch for risc-v-06.dat? It will exclude rv-virt:python from the CI Build. Thanks!

• lupyuen2@ec481c3

[tmedicci]

@tmedicci Yep could you add this patch for risc-v-06.dat? It will exclude rv-virt:python from the CI Build. Thanks!

• lupyuen2@ec481c3
  Thanks!

[xiaoxiang781216]

About the CI, It's failing because of the flags used to build Python. I can fix them, but I'm afraid that it'd take too long to build the test because it requires building python for the host too. @lupyuen , do you have any suggestions on how to skip this test (or at least running it only from time to time)?

but should we fix the build issue first before disabling it from ci? I saw the real build problem from the previous build, it isn't good to merge the unworkable defconfig.

[tmedicci]

About the CI, It's failing because of the flags used to build Python. I can fix them, but I'm afraid that it'd take too long to build the test because it requires building python for the host too. @lupyuen , do you have any suggestions on how to skip this test (or at least running it only from time to time)?

but should we fix the build issue first before disabling it from ci? I saw the real build problem from the previous build, it isn't good to merge the unworkable defconfig.

It's usable. The problem here is that ci sets EXTRAFLAGS="-Wno-cpp -Werror". I'm fixing it by disabling such warnings in the nuttx-apps (as other apps do) and I'll submit it soon.

However, although we have this "issue", the rv-virt:python build requires building the whole Python for the host system (yes, we need Python to build Python), so it'd be a huge load for the CI and that's why I think we should skip it on CI.

[lupyuen]

FYI in case we wish to skip rv-virt:python for CI Build: I verified that latest patch will correctly skip rv-virt:python for risc-v-06:
https://gist.github.com/lupyuen/68e8d34b14a2dc2eaceda3b3f599ca47

$ sudo docker run -it \
  ghcr.io/apache/nuttx/apache-nuttx-ci-linux:latest \
  /bin/bash
# cd
# git clone https://github.com/tmedicci/incubator-nuttx nuttx --branch feature/python
# git clone https://github.com/apache/nuttx-apps apps
NuttX Source: https://github.com/apache/nuttx/tree/2bb644047687d411a58123855d472ff77dba534b
NuttX Apps: https://github.com/apache/nuttx-apps/tree/9c5a2ad062069cadc3209186f35338260360efd4
# cd nuttx/tools/ci
# ./cibuild.sh -c -A -N -R testlist/risc-v-06.dat 
Skipping: rv-virt/python

[tmedicci]

About the CI, It's failing because of the flags used to build Python. I can fix them, but I'm afraid that it'd take too long to build the test because it requires building python for the host too. @lupyuen , do you have any suggestions on how to skip this test (or at least running it only from time to time)?

but should we fix the build issue first before disabling it from ci? I saw the real build problem from the previous build, it isn't good to merge the unworkable defconfig.

It's usable. The problem here is that ci sets EXTRAFLAGS="-Wno-cpp -Werror". I'm fixing it by disabling such warnings in the nuttx-apps (as other apps do) and I'll submit it soon.

However, although we have this "issue", the Python build requires building the whole Python for the host system (yes, we need Python to build Python), so it'd be a huge load for the CI and that's why I think we should skip it on CI.

Ideally, we should exclude it from the "PR-triggered" CI but run it on a scheduled basis (from time to time). I don't know if our CI allows it somehow.

[xiaoxiang781216]

About the CI, It's failing because of the flags used to build Python. I can fix them, but I'm afraid that it'd take too long to build the test because it requires building python for the host too. @lupyuen , do you have any suggestions on how to skip this test (or at least running it only from time to time)?

but should we fix the build issue first before disabling it from ci? I saw the real build problem from the previous build, it isn't good to merge the unworkable defconfig.

It's usable. The problem here is that ci sets EXTRAFLAGS="-Wno-cpp -Werror". I'm fixing it by disabling such warnings in the nuttx-apps (as other apps do) and I'll submit it soon.

However, although we have this "issue", the rv-virt:python build requires building the whole Python for the host system (yes, we need Python to build Python), so it'd be a huge load for the CI and that's why I think we should skip it on CI.

in this case, we should either fix the warning in CPython or add -Wno-xxx to Makefile.

[tmedicci]

Please check apache/nuttx-apps#2886

The roadmap here is: wait for apache/nuttx-apps#2886 to be merged (warnings are disabled for CPython). Then, we can re-run this pipeline.

I temporarily reverted the @lupyuen commit (to reenable the rv-virt:python test). Before merging it, I'll drop it: testing rv-virt:python on every PR is overkill and may substantially increase CI usage.

[tmedicci]

@xiaoxiang781216 , let's wait the CI to run at least once with the rv-virt:python. Then, I'll drop it (to disable test) and we can merge it.

[tmedicci]

@lupyuen , what do you think about creating a new risc-v-07.dat to add just the rv-virt:python and exclude it from the pipelines ran on every PR (but let it running during the "nightly" builds)?

[lupyuen]

@tmedicci Yep sure! This is how we move rv-virt:python from risc-v-06 to risc-v-07:

• lupyuen2@aed7066

The Python Build might slow down the Full Builds at NuttX Mirror Repo. So I'll run risc-v-07 at NuttX Build Farm instead. Any errors will appear at nuttx-dashboard.org (I'm testing a push-alert system with Mastodon)

FYI: NuttX Mirror Repo now runs Full Builds non-stop, roughly every 3 hours, as long as there as new commits. That's why I'm hoping not to slow down the Full Build, also hoping GitHub won't throttle our Mirror Repo for overuse. (Here's the script)

[tmedicci]

@tmedicci Yep sure! This is how we move rv-virt:python from risc-v-06 to risc-v-07:

• lupyuen2@aed7066

The Python Build might slow down the Full Builds at NuttX Mirror Repo. So I'll run risc-v-07 at NuttX Build Farm instead. Any errors will appear at nuttx-dashboard.org (I'm testing a push-alert system with Mastodon)

FYI: NuttX Mirror Repo now runs Full Builds non-stop, roughly every 3 hours, as long as there as new commits. That's why I'm hoping not to slow down the Full Build, also hoping GitHub won't throttle our Mirror Repo for overuse. (Here's the script)

Thanks! Tomorrow morning I will add it to this PR.

About the current pipeline, rv-virt:python test passed successfully, but rv-virt/citest's run test is failing due to timeout (and it doesn't seem to be related to this PR). Any idea?

(I will try to run it locally tomorrow, too)

[lupyuen]

@tmedicci Yep there's a strange problem with CI Test, ps keeps crashing:

• [BUG] rv-virt/citest: test_hello or test_pipe failed #14808 (comment)

[lupyuen]

@tmedicci rv-virt:python is now in NuttX Build Farm, NuttX Dashboard and NuttX Build History! I'm setting up the Push Notifications for Failed Builds (via Mastodon), hopefully it will be easier to monitor :-) (Here's the script)

NuttX Dashboard: https://nuttx-dashboard.org/d/fe2bqg6uk7nr4a/build-logs-from-nuttx-build-farm?from=now-2d&to=now&timezone=browser&var-arch=$__all&var-subarch=$__all&var-board=$__all&var-config=$__all&var-group=risc-v-07&var-Filters=

NuttX Build History: https://nuttx-dashboard.org/d/fe2q876wubc3kc/nuttx-build-history?from=now-7d&to=now&timezone=browser&var-arch=$__all&var-subarch=$__all&var-board=rv-virt&var-config=python&var-group=$__all&var-Filters=

First Build Log: https://gitlab.com/lupyuen/nuttx-build-log/-/snippets/4780917#L165

[tmedicci]

Thanks @lupyuen, this is amazing!