Examples using Python-specific data structures?

[pganssle]

The README (is there more documentation somewhere?) only has a very trivial example of how to use this, and it's not immediately clear how to do anything very complicated. For example, how would I:

1. Create a function that returns a list?
2. Define a class in Rust?
3. Write a function that takes arguments?
4. Write a function that manipulates a Python dictionary?

It is possible that these are "out of scope" for this library (which I gather is very low level), but I would find it difficult to write an interesting Python extension without doing any of these things. If it is out of scope for this library to document the idiomatic use in writing Python backends, maybe you could have a section linking to a few open source projects actually using milksnake so a new user can try to pick up the idioms?

[untitaker]

@pganssle milksnake does not care how you declare your CABI. Basically you can only use datastructures that C gives you, and you have to reimplement everything on top.

Yes, it's difficult, involves a lot of hand-written unsafe code and is rarely worth it. I would recommend keeping your interface between Rust and Python to a minimum because of that.

Consider writing some sort of RPC interface that communicates over a socket instead of all of this.

A project that uses this with a relatively clean CABI is semaphore. Even there we once minimized the function signatures by JSON-encoding function arguments, passing them as single string, and decoding them on the other side, because it was easier and fast enough.

Error handling is also very painful. I wrote https://github.com/untitaker/shippai because of this but it hasn't gained traction. We don't use it inside of sentry, but basically hand-write error enums.

[pganssle]

I'm certainly gathering that milksnake is not particularly suitable for writing dedicated Rust python backends in most circumstances, but I'm saying the documentation for this library should probably be clearer about this.

I do still think a "Projects using milksnake" section in the README would be a good idea, to give a sense of what it's actually used for and what the idioms are.

[untitaker]

milksnake is IMO clear about the fact that it's solely responsible for embedding dynamic libraries into Python packages. It does not generate C bindings, it does not mention Rust except for the example.

I agree wrt "Projects using milksnake".