Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mention libvips/sharp gotcha in setup instructions #2932

Closed
wants to merge 5 commits into from

Conversation

duijf
Copy link
Contributor

@duijf duijf commented Oct 18, 2021

In order to document some stuff about Studio, I wanted to set up dvc.org for local development. I ran into this gotcha, and wanted to mention it in the setup instructions to save others some time.

Gatsby has a dependency on the native libvips library via the sharp NPM package. Sharp by default uses the system libvips if it's available, but that can lead to compatibility problems if there is a version mismatch (or build options are different).

On my system I have libvips installed for something else. This led to the following error when running yarn develop:

ERROR #11321  PLUGIN

"gatsby-plugin-manifest" threw an error while running the onPostBootstrap lifecycle:

Input file contains unsupported image format

[ long stacktrace ]

The fix is to make sharp ignore the system libvips. This needs to be done at dependency install time, as documented in this Gatsby issue: gatsbyjs/gatsby#20698

After removing node_modules and running yarn with this env var, the problem went away.

@duijf duijf requested a review from jorgeorpinel October 18, 2021 08:11
@shcheklein shcheklein temporarily deployed to dvc-org-clarify-build-i-leauh2 October 18, 2021 08:12 Inactive
@duijf duijf self-assigned this Oct 18, 2021
@duijf
Copy link
Contributor Author

duijf commented Oct 18, 2021

Link to the changed page on the preview environment: https://dvc-org-clarify-build-i-leauh2.herokuapp.com/doc/user-guide/contributing/docs#development-environment

@rogermparent
Copy link
Contributor

Great catch! I've run into this before as well, it's related to us using old Gatsby packages that have been left behind by more recent versions of libvips. I'd suggest a minor change to the text that makes it read better to me:

⚠️ Do you get an error from gatsby-plugin-manifest about unsupported image
formats? You may be running into
this issue. You can fix it
by reinstalling node_modules with a flag that forces sharp to ignore your
globally installed libvips:

Someone on the docs team might have a different/better suggestion. Other than that, thanks for the contribution!

@duijf duijf force-pushed the clarify-build-instructions branch from fb145c2 to 2caea4c Compare October 18, 2021 14:36
@shcheklein shcheklein temporarily deployed to dvc-org-clarify-build-i-leauh2 October 18, 2021 14:36 Inactive
@duijf
Copy link
Contributor Author

duijf commented Oct 18, 2021

Thanks for the suggestion @rogermparent! Indeed reads better to me. I've updated the PR :)

@iesahin
Copy link
Contributor

iesahin commented Oct 19, 2021

I think it might be better to have a Troubleshooting section header and put this in a bullet list. WDYT @jorgeorpinel ?

@jorgeorpinel jorgeorpinel removed their request for review October 21, 2021 01:08
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-clarify-build-i-leauh2 October 21, 2021 01:08 Inactive
@jorgeorpinel
Copy link
Contributor

jorgeorpinel commented Oct 21, 2021

I've run into this before as well, it's related to us using old Gatsby packages that have been left behind by more recent versions of libvips

So maybe we should make an issue to update Gatsby paks instead @rogermparent ? It's great to be aware of this but documenting it means we don't have a solution.

might be better to have a Troubleshooting section header

@iesahin I think Troubleshooting should be about the products, not about the docs engine.

Comment on lines +89 to +97
> formats? You may be running into
> [this issue](https://github.com/gatsbyjs/gatsby/issues/20698). You can fix it
> by reinstalling `node_modules` with a flag that forces `sharp` to ignore your
> globally installed `libvips`:
>
> ```dvc
> $ rm -r node_modules
> $ SHARP_IGNORE_GLOBAL_LIBVIPS=true yarn
> ```
Copy link
Contributor

@jorgeorpinel jorgeorpinel Oct 21, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we must have this, let's keep it very very short 🙂

Suggested change
> formats? You may be running into
> [this issue](https://github.com/gatsbyjs/gatsby/issues/20698). You can fix it
> by reinstalling `node_modules` with a flag that forces `sharp` to ignore your
> globally installed `libvips`:
>
> ```dvc
> $ rm -r node_modules
> $ SHARP_IGNORE_GLOBAL_LIBVIPS=true yarn
> ```
> formats? You may be running into
> [this issue](https://github.com/gatsbyjs/gatsby/issues/20698#issuecomment-576353427).

☝️ direct link to the solution

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact I'd move this note after the paragraph below.

@jorgeorpinel
Copy link
Contributor

Hmmmm. There's a strange link checker error too: https://github.com/iterative/dvc.org/pull/2932/checks?check_run_id=3958726103. Do you think that's coming from master @rogermparent ? Thanks

@rogermparent
Copy link
Contributor

So maybe we should make an issue to update Gatsby paks instead @rogermparent ? It's great to be aware of this but documenting it means we don't have a solution.

#2851 (merged shortly after this issue was made) actually did upgrade these packages, so I believe we shouldn't run into this anymore.

Hmmmm. There's a strange link checker error too: https://github.com/iterative/dvc.org/pull/2932/checks?check_run_id=3958726103. Do you think that's coming from master

I'm not sure where it's coming from, usually this happens on massive diffs- it seems the action is having trouble finding merge bases so it's working off of the plain diff from master, which seems to be too big for it to handle (definitely a thing we need to handle when addressing link check in general)

@jorgeorpinel
Copy link
Contributor

#2851 (merged shortly after this issue was made) actually did upgrade these packages, so I believe we shouldn't run into this anymore.

Thanks! @duijf any chance you can try again from scratch with the latest repo? If you don't have the issue again I think we can close the PR.

@duijf
Copy link
Contributor Author

duijf commented Oct 22, 2021

Tried again from scratch on ad40ef3 and this is indeed no longer an issue 😄

@duijf duijf closed this Oct 22, 2021
@rogermparent
Copy link
Contributor

Great to hear that it works now!

I'm trying to get a replication of the link-check bug, but can't get one from a new branch even if I clone this one- reopening to hopefully get to the bottom of this.

@rogermparent rogermparent reopened this Oct 22, 2021
@shcheklein shcheklein had a problem deploying to dvc-org-clarify-build-i-cpyjqi October 22, 2021 19:51 Failure
@rogermparent rogermparent temporarily deployed to dvc-org-clarify-build-i-cpyjqi October 22, 2021 20:39 Inactive
@shcheklein shcheklein had a problem deploying to dvc-org-clarify-build-i-zcpsrf October 22, 2021 20:39 Failure
@rogermparent rogermparent temporarily deployed to dvc-org-clarify-build-i-zcpsrf October 22, 2021 20:56 Inactive
@rogermparent rogermparent temporarily deployed to dvc-org-clarify-build-i-jserwz October 22, 2021 22:12 Inactive
@shcheklein shcheklein temporarily deployed to dvc-org-clarify-build-i-haw2uj October 22, 2021 22:12 Inactive
@rogermparent
Copy link
Contributor

Alright, managed to replicate and solve the issue here. It was indeed fetch-depth.

@jorgeorpinel
Copy link
Contributor

Thanks again for the report @duijf and for looking into this @rogermparent 👍

@rogermparent rogermparent deleted the clarify-build-instructions branch November 3, 2021 13:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants