Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blog post on Rust/Cryptography #31

Merged
merged 19 commits into from
Jun 23, 2022
Merged

Blog post on Rust/Cryptography #31

merged 19 commits into from
Jun 23, 2022

Conversation

hoodmane
Copy link
Member

@netlify
Copy link

netlify bot commented Jun 20, 2022

Deploy Preview for pyodide-blog ready!

Name Link
🔨 Latest commit 682fbda
🔍 Latest deploy log https://app.netlify.com/sites/pyodide-blog/deploys/62b4851984a9920008ad77af
😎 Deploy Preview https://deploy-preview-31--pyodide-blog.netlify.app/posts/rust-pyo3-support-in-pyodide
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

Copy link

@alex alex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great to me. Thanks for diving into all of these issues and I'm glad its mostly working now!

Hopefully someday soon we'll be able to drop Python 3.6 and remove the need to patch instant and chrono.

the Rust team for reviewing and merging patches to improve Emscripten support.
Thanks to the PyO3 team for their help and enthusiasm and for adding Emscripten
tests to their continuous integration. Special thanks to the Emscripten team,
particularly Sam Clegg, for their technical advice, for merging patches, and for
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wowa, small world! Long time no see Sam!

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Great to see you name pop up in my feed Alex. I hope you are good!

@messense
Copy link

The post content looks great, but the excerpt from the list of posts looks odd:

image

@reaperhulk
Copy link

Thanks for the ping, this was a fascinating read. Glad to see you found a path and I hope we'll be able to drop 3.6 early next year to improve our deps a bit!

Copy link
Member

@ryanking13 ryanking13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't following your Cryptography work in detail, so I was quite surprised that there was so many issues that I didn't know. Thanks a lot again Hood, it was really fun to read.

content/posts/rust.md Outdated Show resolved Hide resolved
Copy link
Member

@rth rth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this work and your contribution to the ecosystem!

The post reads nicely. It's pretty advanced though and readers not familiar with Emscripten or Rust (or which don't have some general knowledge of build toolchains) might be a bit lost. So I'm not sure what best to do about it,

  • leave it as is and people will decide by themselves whether to read it.
  • explain more of the concepts, but it's already a long post and it touches on a lot of things, so making it even longer would not be ideal.
  • add a disclamer sentence toward the beginning to mention that this post requires minimal knowledge of [...] topics, and give some references to learn more about those. Though then maybe we would lose readers who would actually be interested in reading it, even if they don't have the pre-requisites.

A few comments/suggestions otherwise overall LGTM, though I don't know much about some of the topics discussed.

I agree we can probably publish it on Wednesday or Thursday.

content/posts/rust.md Outdated Show resolved Hide resolved
content/posts/rust.md Outdated Show resolved Hide resolved
content/posts/rust.md Outdated Show resolved Hide resolved
content/posts/rust.md Outdated Show resolved Hide resolved
content/posts/rust.md Outdated Show resolved Hide resolved
content/posts/rust.md Outdated Show resolved Hide resolved
```
Unable to find library `-lc-debug`
```
For some reason, Rust tries to link `libc` into Emscripten dynamic libraries. It
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it's the default behavior on Linux as far as I know, so it's not too surprising. What's probably more unexpected is Emscripten's workflow with libc linked into the main module.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

content/posts/rust.md Outdated Show resolved Hide resolved
content/posts/rust.md Outdated Show resolved Hide resolved
content/posts/rust.md Outdated Show resolved Hide resolved
@hoodmane
Copy link
Member Author

it's already a long post and it touches on a lot of things, so making it even longer would not be ideal.

I would also be fine with removing some of the less interesting sections. But in case anyone else trying to build Rust code for wasm32-unknown-emscripten hits similar errors, I thought it might be helpful to be thorough.

Copy link

@davidhewitt davidhewitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry to have been a bit slow to give this a read. Thanks for writing this, really interesting to see the whole picture, and thanks also for working on the wider problem space & Pyodide!

Had a few quick thoughts, feel free to simply ignore them ofc 😄

content/posts/rust-pyo3-support-in-pyodide.md Outdated Show resolved Hide resolved
Thanks to the Cryptography team for making this necessary and for their
technical advice. Thanks to the <code class="pkg">chrono</code> maintainers and
the Rust team for reviewing and merging patches to improve Emscripten support.
Thanks to the PyO3 team for their help and enthusiasm and for adding Emscripten

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙌

symbol is the `PyInit__my_module` function that Python invokes when loading a
native module.

It would be great to link our other extension modules with `-sSIDE_MODULE=2` but

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a little unsure what this is aspiring to (what "our other extension modules" are).

is not passed to the linker. Conveniently, Rust is good at calculating which
symbols should be exported: public symbols with the `#[no_mangle]` attribute are
exported, other symbols are not. In the case of a PyO3 module, the only exported
symbol is the `PyInit__my_module` function that Python invokes when loading a

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So does Rust pass this symbol automatically to the linker, or did you have to add it to the command line?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rust automatically passes the information to the linker when building a cdylib, this is handled here:
https://github.com/rust-lang/rust/blob/master/compiler/rustc_codegen_ssa/src/back/link.rs#L1882

Co-authored-by: David Hewitt <1939362+davidhewitt@users.noreply.github.com>
@rth
Copy link
Member

rth commented Jun 22, 2022

@hoodmane Feel free to merge once you have responded to above comments. Just need to make sure that the date in the header is today and in the past (otherwise if it's in the future it won't be published).

Also if you have a short description for posting this on Twitter, please send it to me, otherwise, I'll write something. Thanks!

@hoodmane hoodmane merged commit 7be90e4 into pyodide:master Jun 23, 2022
@hoodmane hoodmane deleted the rust branch June 23, 2022 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants