Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wheel support for linux aarch64 #500

Closed
odidev opened this issue Jul 3, 2020 · 18 comments
Closed

Wheel support for linux aarch64 #500

odidev opened this issue Jul 3, 2020 · 18 comments
Assignees

Comments

@odidev
Copy link

odidev commented Jul 3, 2020

Summary
Installing multidict on aarch64 via pip using command "pip3 install multidict" tries to build wheel from source code

Problem description
multidict don't have wheel for aarch64 on PyPI repository. So, while installing multidict via pip on aarch64, pip builds wheel for same resulting in it takes more time to install multidict. Making wheel available for aarch64 will benefit aarch64 users by minimizing multidict installation time.

Expected Output
Pip should be able to download multidict wheel from PyPI repository rather than building it from source code.

@multidict-taem, please let me know if I can help you building wheel/uploading to PyPI repository. I am curious to make multidict wheel available for aarch64. It will be a great opportunity for me to work with you.

@webknjaz
Copy link
Member

webknjaz commented Jul 3, 2020

Hello, there's no standard for such wheels so it's not technically possible to put them on PyPI. It only accepts manylinux wheels.

@odidev
Copy link
Author

odidev commented Jul 3, 2020

@webknjaz Manylinux2014 has given Linux AArch64 support. I have successfully build manylinux2014 wheel for same platform. I can help in this activity.

Also please reopen the issue so that we can work on this and let me know if you have any questions.

Thanks

@webknjaz
Copy link
Member

webknjaz commented Jul 3, 2020

TIL

@webknjaz webknjaz reopened this Jul 3, 2020
@webknjaz
Copy link
Member

webknjaz commented Jul 3, 2020

This will probably require changing https://github.com/aio-libs/multidict/blob/master/.azure-pipelines/deploy.yml

@webknjaz
Copy link
Member

webknjaz commented Jul 3, 2020

The only concern I have is that it's probably hard to cross-compile aarch64 in Azure.

@webknjaz
Copy link
Member

webknjaz commented Jul 3, 2020

I'm curious if https://www.worksonarm.com/projects/ would help with this.

@odidev
Copy link
Author

odidev commented Jul 6, 2020

@webknjaz We can use qemu to run an aarch64 build on x86 machine. Please let me know your thoughts here.

@webknjaz
Copy link
Member

webknjaz commented Jul 6, 2020

If you can make it work, send a PR and I'll review it.

@odidev
Copy link
Author

odidev commented Jul 8, 2020

Thanks @webknjaz I will work on it and update you.

@webknjaz
Copy link
Member

webknjaz commented Jul 8, 2020

I have one extra note on this. Adding a new platform implies increasing the maintenance burden. Please do not expect the maintainers to invest too much effort into this if it starts failing in the future.

If you want to ensure that it'll keep working, you'll need to commit to contributing some effort should any related problems arise.

@odidev
Copy link
Author

odidev commented Jul 8, 2020

sure @webknjaz, I will be taking care of the failures in future. Thanks.

@odidev
Copy link
Author

odidev commented Aug 17, 2020

Hi @webknjaz ,

I have added support for azure pipeline for aarch64 using qemu. I am successfully able to run the test cases.
Please have a look on the changes:
odidev@61cd26b

Below is the link for azure pipeline logs for aarch64:
https://dev.azure.com/odidev0187/odidev/_build/results?buildId=33&view=results

I tried running deploy pipeline using deploy.yml but I am getting authorization error.
Please review the changes and give your suggestions for the deployment of wheels.

Thanks

@webknjaz
Copy link
Member

I'd prefer to review a PR instead.

@webknjaz
Copy link
Member

Most of it should be done against the template repo and then used here.

@mattsplats
Copy link

@odidev just checking on this, can you open a PR with your changes?

@webknjaz
Copy link
Member

webknjaz commented Oct 6, 2020

@mattsplats he did, but in the wrong repo: #519

@mattsplats
Copy link

@webknjaz As per comment on the PR, please take a look at the pipelines repo change: aio-libs/azure-pipelines#1

@asvetlov
Copy link
Member

Done by multidict 5.0.0 release

aio-libs-github-bot bot pushed a commit to aio-libs/yarl that referenced this issue Oct 14, 2020
Bumps [multidict](https://github.com/aio-libs/multidict) from 4.7.6 to 5.0.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aio-libs/multidict/releases">multidict's releases</a>.</em></p>
<blockquote>
<h2>multidict 5.0.0</h2>
<h2>Features</h2>
<ul>
<li>Provide wheels for <code>aarch64</code>, <code>i686</code>, <code>ppc64le</code>, <code>s390x</code> architectures on Linux
as well as <code>x86_64</code>.
<code>[#500](aio-libs/multidict#500) &lt;https://github.com/aio-libs/multidict/issues/500&gt;</code>_</li>
<li>Provide wheels for Python 3.9.
<code>[#534](aio-libs/multidict#534) &lt;https://github.com/aio-libs/multidict/issues/534&gt;</code>_</li>
</ul>
<h2>Removal</h2>
<ul>
<li>Drop Python 3.5 support; Python 3.6 is the minimum supported Python version.</li>
</ul>
<h2>Misc</h2>
<ul>
<li><code>[#503](aio-libs/multidict#503) &lt;https://github.com/aio-libs/multidict/issues/503&gt;</code>_</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/aio-libs/multidict/blob/master/CHANGES.rst">multidict's changelog</a>.</em></p>
<blockquote>
<h1>5.0.0 (2020-10-12)</h1>
<h2>Features</h2>
<ul>
<li>Provide wheels for <code>aarch64</code>, <code>i686</code>, <code>ppc64le</code>, <code>s390x</code> architectures on Linux as well as <code>x86_64</code>. <a href="https://github-redirect.dependabot.com/aio-libs/multidict/issues/500">#500</a></li>
<li>Provide wheels for Python 3.9. <a href="https://github-redirect.dependabot.com/aio-libs/multidict/issues/534">#534</a></li>
</ul>
<h2>Removal</h2>
<ul>
<li>Drop Python 3.5 support; Python 3.6 is the minimal supported Python version.</li>
</ul>
<h2>Misc</h2>
<ul>
<li><a href="https://github-redirect.dependabot.com/aio-libs/multidict/issues/503">#503</a></li>
</ul>
<hr />
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/aio-libs/multidict/commit/087f9dcb4b880409a4a9a8f59ded15b1d6b90513"><code>087f9dc</code></a> Fix artifacts path</li>
<li><a href="https://github.com/aio-libs/multidict/commit/421de960a06a58996fd9558b4fe98ee78ffb2617"><code>421de96</code></a> Fix artifacts path</li>
<li><a href="https://github.com/aio-libs/multidict/commit/2b22b2c47b2a881335c688577a6dbd7ab5c06558"><code>2b22b2c</code></a> Fix manylinux build</li>
<li><a href="https://github.com/aio-libs/multidict/commit/eea466d174b52b94312b8e99ffdcca4fff9cd6c2"><code>eea466d</code></a> Fix tests</li>
<li><a href="https://github.com/aio-libs/multidict/commit/001ae3b33c370b8cb29740f7ff05e3b28efaa5c0"><code>001ae3b</code></a> Fix spelling</li>
<li><a href="https://github.com/aio-libs/multidict/commit/e152be053ff91d80fdabf6f1d9be3f4bd095c20f"><code>e152be0</code></a> Update whitelist</li>
<li><a href="https://github.com/aio-libs/multidict/commit/a02e976ff173b377f30a09b1a021a1c8a81d4594"><code>a02e976</code></a> Bump to 5.0.0</li>
<li><a href="https://github.com/aio-libs/multidict/commit/d41493d3a477f50af370a2e52fcb99222902cb4e"><code>d41493d</code></a> Bump mypy version</li>
<li><a href="https://github.com/aio-libs/multidict/commit/9a70da9c9679f466989e44e18a4f11589e7725b7"><code>9a70da9</code></a> Drop azure-pipelines integration</li>
<li><a href="https://github.com/aio-libs/multidict/commit/5fb74cfac3963580113bf7d1e2d3a70ec45af3db"><code>5fb74cf</code></a> Final deployment</li>
<li>Additional commits viewable in <a href="https://github.com/aio-libs/multidict/compare/v4.7.6...v5.0.0">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://api.dependabot.com/badges/compatibility_score?dependency-name=multidict&package-manager=pip&previous-version=4.7.6&new-version=5.0.0)](https://dependabot.com/compatibility-score/?dependency-name=multidict&package-manager=pip&previous-version=4.7.6&new-version=5.0.0)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
If all status checks pass Dependabot will automatically merge this pull request.

[//]: # (dependabot-automerge-end)
aio-libs-github-bot bot pushed a commit to aio-libs/yarl that referenced this issue Dec 14, 2020
Bumps [multidict](https://github.com/aio-libs/multidict) from 5.0.2 to 5.1.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aio-libs/multidict/releases">multidict's releases</a>.</em></p>
<blockquote>
<h2>multidict 5.1.0 release</h2>
<h1>Changes</h1>
<h2>Features</h2>
<ul>
<li>Support <code>GenericAliases</code> (<code>MultiDict[str]</code>) for Python 3.9+
<code>[#553](aio-libs/multidict#553) &lt;https://github.com/aio-libs/multidict/issues/553&gt;</code>_</li>
</ul>
<h2>Bugfixes</h2>
<ul>
<li>Synchronize the declared supported Python versions in <code>setup.py</code> with actually supported and tested ones.
<code>[#552](aio-libs/multidict#552) &lt;https://github.com/aio-libs/multidict/issues/552&gt;</code>_</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/aio-libs/multidict/blob/master/CHANGES.rst">multidict's changelog</a>.</em></p>
<blockquote>
<h1>5.1.0 (2020-12-03)</h1>
<h2>Features</h2>
<ul>
<li>Support <code>GenericAliases</code> (<code>MultiDict[str]</code>) for Python 3.9+
<code>[#553](aio-libs/multidict#553) &lt;https://github.com/aio-libs/multidict/issues/553&gt;</code>_</li>
</ul>
<h2>Bugfixes</h2>
<ul>
<li>Synchronize the declared supported Python versions in <code>setup.py</code> with actually supported and tested ones.
<code>[#552](aio-libs/multidict#552) &lt;https://github.com/aio-libs/multidict/issues/552&gt;</code>_</li>
</ul>
<hr />
<h1>5.0.1 (2020-11-14)</h1>
<h2>Bugfixes</h2>
<ul>
<li>Provide x86 Windows wheels
<code>[#550](aio-libs/multidict#550) &lt;https://github.com/aio-libs/multidict/issues/550&gt;</code>_</li>
</ul>
<hr />
<h1>5.0.0 (2020-10-12)</h1>
<h2>Features</h2>
<ul>
<li>Provide wheels for <code>aarch64</code>, <code>i686</code>, <code>ppc64le</code>, <code>s390x</code> architectures on Linux
as well as <code>x86_64</code>.
<code>[#500](aio-libs/multidict#500) &lt;https://github.com/aio-libs/multidict/issues/500&gt;</code>_</li>
<li>Provide wheels for Python 3.9.
<code>[#534](aio-libs/multidict#534) &lt;https://github.com/aio-libs/multidict/issues/534&gt;</code>_</li>
</ul>
<h2>Removal</h2>
<ul>
<li>Drop Python 3.5 support; Python 3.6 is the minimal supported Python version.</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/aio-libs/multidict/commit/c3eed831bb8bb944bb329013b4fd4b02c66fbf18"><code>c3eed83</code></a> Bump to 5.1.0</li>
<li><a href="https://github.com/aio-libs/multidict/commit/14f433da3b6fd7c443cf85882a6dd1016427262b"><code>14f433d</code></a> Support GenericAliases for Python 3.9+ (<a href="https://github-redirect.dependabot.com/aio-libs/multidict/issues/553">#553</a>)</li>
<li><a href="https://github.com/aio-libs/multidict/commit/e19929eade3571c154b7a6afd57be0360e4a455a"><code>e19929e</code></a> Fix github workflow filter</li>
<li><a href="https://github.com/aio-libs/multidict/commit/b92ea7f32198107f320c56fcfc17bf3b1ea963e1"><code>b92ea7f</code></a> Drop Python 3.5 from setup.py (<a href="https://github-redirect.dependabot.com/aio-libs/multidict/issues/552">#552</a>)</li>
<li>See full diff in <a href="https://github.com/aio-libs/multidict/compare/v5.0.2...v5.1.0">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=multidict&package-manager=pip&previous-version=5.0.2&new-version=5.1.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/configuring-github-dependabot-security-updates)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants