Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hooks: rewrite pygraphviz hook #849

Merged
merged 4 commits into from
Jan 7, 2025
Merged

Conversation

rokm
Copy link
Member

@rokm rokm commented Jan 6, 2025

Rewrite pygraphviz hook to fix discovery and collection of graphviz files under various Linux distributions, in Anaconda environments (Windows, Linux, and macOS), and msys2 environments (Windows).

We now discover active graphviz installation on all OSes by looking for dot program in PATH. To find the location of shared libraries and the plugin directory on non-Windows, we now perform binary dependency analysis on the dot executable and look for known graphviz shared libraries in the results.

This improves the discovery on various platforms, and also fixes the issue where system-installed (on Linux) or Homebrew-installed (on macOS) graphviz shared libraries and plugins would end up being collected instead of Anaconda-packaged ones when using Anaconda environment with Anaconda-packaged graphviz.

Supersedes and closes #520.
Closes #705.

@rokm rokm force-pushed the hook-pygraphviz branch from 5d40b8a to 601d0df Compare January 6, 2025 20:27
Add a functional test for `pygraphviz` that tries to use different
`graphviz` programs and output formats, in order to ensure that
`graphviz` programs and plugins are properly collected.

This complements the existing test that ensures that `pygraphviz.AGraph`
resolves the bundled copies of `graphviz` programs, but does not
test their actual use.
@rokm rokm force-pushed the hook-pygraphviz branch from 601d0df to 138d38c Compare January 6, 2025 20:29
Rewrite `pygraphviz` hook to fix discovery and collection of `graphviz`
files under various Linux distributions, in Anaconda environments
(Windows, Linux, and macOS), and msys2 environments (Windows).

We now discover active `graphviz` installation on all OSes by looking
for `dot` program in `PATH`. To find the location of shared libraries
and the plugin directory on non-Windows, we now perform binary
dependency analysis on the `dot` executable and look for known
`graphviz` shared libraries in the results.

This improves the discovery on various platforms, and also fixes the
issue where system-installed (on Linux) or Homebrew-installed
(on macOS) `graphviz` shared libraries and plugins would end up being
collected  instead of Anaconda-packaged ones when using Anaconda
environment with Anaconda-packaged `graphviz`.
@rokm rokm force-pushed the hook-pygraphviz branch from 138d38c to ebf0a53 Compare January 6, 2025 23:25
@rokm rokm marked this pull request as ready for review January 7, 2025 19:05
@rokm
Copy link
Member Author

rokm commented Jan 7, 2025

Tested with:

  • Linux:
    • Fedora with system-provided graphviz, system-provided python & pip-installed pygraphviz.
    • Fedora with Anaconda environment (Anaconda-packaged python, graphviz, pygraphviz). (With system-installed graphviz also available).
    • Debian Bookworm in container (debian:bookworm) with system-provided graphviz, system-provided python & pip-installed pygraphviz.
    • Arch Linux in container (archlinux:latest) with system-provided graphviz, system-provided python & pip installed- pygraphviz.
  • Windows
    • python.org python, official graphviz installer, pip-installed pygraphviz.
    • msys/ming64 environment with msys-packaged python and graphviz, pip-installed pygraphviz.
    • Anaconda environment with Anaconda-packaged python, graphviz, pygraphviz (with official build from first step also being available and its c:\program files\graphviz\bin being in PATH).
  • macOS
    • python.org python, Homebrew-installed graphviz, pip-installed pygraphviz.
    • Anaconda environment with Anaconda-packaged python, graphviz, pygraphviz. (With Homebrew-installed graphviiz also available).

@rokm rokm requested a review from bwoodsend January 7, 2025 19:13
Enable `pygraphviz` tests on linux and macOS; requires installing
`graphviz` via `brew install` (macOS runner) and `apt-get install`
(Ubuntu runner).
@rokm rokm force-pushed the hook-pygraphviz branch 2 times, most recently from e8bdbe7 to 0b1296f Compare January 7, 2025 21:51
On `macos-14` arm64 runners, Homebrew is installed in `/opt/homebrew`
instead of `/usr/local` prefix, and its headers and shared libraries
are not in the default search path. Add them by setting `CPPFLAGS`
and `LDFLAGS`. This is required by `pygraphviz`, which needs to
build its extension and link it against Homebrew-installed `graphviz`.
@rokm rokm force-pushed the hook-pygraphviz branch from 0b1296f to f8a1c2b Compare January 7, 2025 21:52
@rokm rokm merged commit f73f04a into pyinstaller:master Jan 7, 2025
26 checks passed
@rokm rokm deleted the hook-pygraphviz branch January 7, 2025 23:03
github-actions bot pushed a commit to wxx9248/Pickle-Rush that referenced this pull request Jan 16, 2025
…25.0 (#121)

Bumps
[pyinstaller-hooks-contrib](https://github.com/pyinstaller/pyinstaller-hooks-contrib)
from 2024.11 to 2025.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/releases">pyinstaller-hooks-contrib's
releases</a>.</em></p>
<blockquote>
<h2>v2025.0</h2>
<p>Please see the <a
href="https://www.github.com/pyinstaller/pyinstaller-hooks-contrib/tree/v2025.0/CHANGELOG.rst">changelog</a>
for more details</p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/blob/master/CHANGELOG.rst">pyinstaller-hooks-contrib's
changelog</a>.</em></p>
<blockquote>
<h2>2025.0 (2025-01-16)</h2>
<p>New hooks</p>
<pre><code>
* Add hook for ``fsspec`` to collect the package's submodules
and ensure the protocol plugins are working.
(`[#856](pyinstaller/pyinstaller-hooks-contrib#856)

&lt;https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/856&gt;`_)
* Add hook for ``intake`` to collect its plugins (registered via the
``intake.drivers`` entry-point).
(`[#853](pyinstaller/pyinstaller-hooks-contrib#853)

&lt;https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/853&gt;`_)
* Add hook for ``ruamel.yaml`` to collect its plugins, and ensure that
  plugins' ``__plug_in__`` modules are collected as source .py files
(which is necessary for their discovery).
(`[#844](pyinstaller/pyinstaller-hooks-contrib#844)

&lt;https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/844&gt;`_)
* Add hook for ``sam2`` (Segment Anything Model 2).
(`[#847](pyinstaller/pyinstaller-hooks-contrib#847)

&lt;https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/847&gt;`_)
* Add hook for ``zarr`` to collect the package's metadata.
(`[#855](pyinstaller/pyinstaller-hooks-contrib#855)

&lt;https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/855&gt;`_)
<p>Updated hooks
</code></pre></p>
<ul>
<li>Revise the search for OpenSSL shared library and
<code>ossl-modules</code> directory
in the <code>cryptography</code> hook, in order to mitigate issues with
unrelated
copies of OpenSSL ending up being pulled into the build. Most notably,
the hook should not be searching for OpenSSL shared library when
<code>cryptography</code> PyPI wheel is installed, because those ship
with
extensions that are statically linked against OpenSSL.
(<code>[#846](pyinstaller/pyinstaller-hooks-contrib#846)
&lt;https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/846&gt;</code>_)</li>
<li>Rewrite <code>pygraphviz</code> hook to fix discovery and collection
of <code>graphviz</code>
files under various Linux distributions, in Anaconda environments
(Windows, Linux, and macOS), and msys2 environments (Windows).
(<code>[#849](pyinstaller/pyinstaller-hooks-contrib#849)
&lt;https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/849&gt;</code>_)</li>
<li>Update <code>dask</code> hook to collect template files from
<code>dask/widgets/templates</code>
directory; these file become mandatory when using
<code>dask.array</code> and
<code>jinja2</code> is available.
(<code>[#852](pyinstaller/pyinstaller-hooks-contrib#852)
&lt;https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/852&gt;</code>_)</li>
<li>Update <code>triton</code> hook for compatibility with
<code>triton</code> &gt;= 3.0.0; the
hook should now collect backend-specific modules and data files from
<code>triton.backends</code>.
(<code>[#848](pyinstaller/pyinstaller-hooks-contrib#848)
&lt;https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/848&gt;</code>_)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/commit/06928e9cfd1f1003892002cc26e79e5d2d957189"><code>06928e9</code></a>
Release v2025.0</li>
<li><a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/commit/2c162e9417db3ca686b11662485dc42bc71d66aa"><code>2c162e9</code></a>
hooks: add hook for fsspec (<a
href="https://redirect.github.com/pyinstaller/pyinstaller-hooks-contrib/issues/856">#856</a>)</li>
<li><a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/commit/6c7df30fe94341457032ced8ac2ee79e5a6b6cd6"><code>6c7df30</code></a>
hooks: add hook for zarr (<a
href="https://redirect.github.com/pyinstaller/pyinstaller-hooks-contrib/issues/855">#855</a>)</li>
<li><a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/commit/8d1e52848ec92d194604b613bf4916cfeefc290e"><code>8d1e528</code></a>
hooks: add hook for intake (<a
href="https://redirect.github.com/pyinstaller/pyinstaller-hooks-contrib/issues/853">#853</a>)</li>
<li><a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/commit/0a113f8a5b696e949c1096c3e593cbfecf4e6106"><code>0a113f8</code></a>
Scheduled weekly dependency update for week 02 (<a
href="https://redirect.github.com/pyinstaller/pyinstaller-hooks-contrib/issues/851">#851</a>)</li>
<li><a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/commit/717223dd3580bb5a9cf993609b3f7dba58e18d6e"><code>717223d</code></a>
hooks: dask: collect template files from dask/widgets/templates</li>
<li><a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/commit/10c53079a55e33bcf82d26ffca1e9a2ca19bc759"><code>10c5307</code></a>
hooks: prettytable does not query its version from metadata after
3.12.0</li>
<li><a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/commit/f73f04aa62c2b752f6d2bfa8d77fe0fcddcf4ac2"><code>f73f04a</code></a>
ci: add Homebrew headers and shared libraries to search path</li>
<li><a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/commit/ef78a3f40c397fdf4160a0d33ec087057cfc954a"><code>ef78a3f</code></a>
tests: enable pygraphviz tests</li>
<li><a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/commit/fef9193c4403cf46d4c37edcd2c3f3b9e4ed9165"><code>fef9193</code></a>
hooks: rewrite pygraphviz hook</li>
<li>Additional commits viewable in <a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/compare/v2024.11...v2025.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pyinstaller-hooks-contrib&package-manager=pip&previous-version=2024.11&new-version=2025.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

PyInstaller: Unable to find '/usr/sbin/neato' when adding binary and data files
2 participants