Support for creating pex binaries

[sitaktif]

Bazel currently has good support for creating zip and par python binary outputs, however these formats don't support producing a single file that is compatible with multiple platforms (e.g. Linux and MacOS) when dependencies include platform-specific dependencies. Pex files support that by including all the various wheels in the output file and doing some magic with sys.path.

How do the rules_python maintainers feel about the addition of a pex_binary rule to the project?

[thundergolfer]

@sitaktif I think Pex support is out-of-scope for this project. There's a number of solutions in this space: Pex, Facebook's XAR, and Google's subpar. I don't think we should have this project invest in internally endorsing and supporting one of them.

That said, I think a goal of rules_python should be extensibility so that other rules (like a hypothetical rules_pex) can be easily built on top of this project.

Have you tried bazel_rules_pex? I haven't myself, and it does look like it doesn't work for recent Bazel versions.

[sitaktif]

(yeah bazel_rules_pex needs quite a bit of rework from what I could see)

Not a problem, one of the aim of this issue was to determine whether pex were an acceptable thing to support.

In general, though, it's a shame we don't have a built in multi-platform solution for python single executables. It does feel to me like supporting this use case would make sense. WDYT?

Happy to close the issue if I get confirmation the above (more generic) requirement is out of scope for the project.

[thundergolfer]

In general, though, it's a shame we don't have a built in multi-platform solution for python single executables.

Oh it's a big problem with Python for sure.

While there's not a clear 'winner' in this problem space I don't think we should consider supporting one of the N options out there.

I would be in favour of closing this and maybe having a tracking issue for "multi-platform solution for python single executables." (ie. not Pex specifically)

[brandjon]

and Google's subpar

FYI, subpar isn't being actively maintained except to the extent it needs updates to keep this repository working.

[thundergolfer]

Thanks yep. I've had no problems using the ZIP output of py_binary so far but my experience with that solution has been limited. We're still using subpar at Canva but Andy has told me Stripe no longer uses Subpar and just uses ZIP output.

[groodt]

FYI, subpar isn't being actively maintained except to the extent it needs updates to keep this repository working.

Yes, I think the native support in Bazel for Python zips is probably a sufficient replacement for subpar now in most cases.

[sitaktif]

I would be in favour of closing this and maybe having a tracking issue for "multi-platform solution for python single executables." (ie. not Pex specifically)

SGTM 👍

[sitaktif]

Created #310

[alexeagle]

Note, rules_py can produce PEX now: https://github.com/aspect-build/rules_py/blob/v0.8.1/docs/rules.md#py_pex_binary