Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for #1214, linked scoped dependencies should not be overwritten #1970

Merged
merged 2 commits into from
Dec 7, 2016

Conversation

tgriesser
Copy link
Contributor

Summary

#1214 identifies a problem with scoped linked dependencies being modified after various yarn commands. After digging into it a bit, it appears the problem is with the package-linker.js. The possibleExtraneous modules do not account for the fact that scoped deps are nested underneath their scope name, so the linked scoped deps are not properly added to the list of ignored packages. This leads yarn to replace the currently linked module with the cached version specified in the lockfile.

In changing the behavior to properly add these modules, I noticed a test was breaking in __tests__/commands/remove.js, however I believe this test to be incorrect. It was testing the entire scope folder was removed rather than just the individual package.

Test plan

Added a test with the directory structure after a package is yarn link'ed, with a symlink existing in both .yarn-cache. The test checks that the linked / scoped package.json is kept intact and that the index.js is not copied from the cached module into the linked folder.

@wclr
Copy link
Contributor

wclr commented Nov 21, 2016

Btw potentially yarn allows to have folder starting with @ that is not a container fo scoped packages but a package itself, think this should be handled.

"@some-package": "file:../some-package"

so node_modules/@some-package/package.json exists.

@wclr
Copy link
Contributor

wclr commented Nov 21, 2016

Also probably while the removal of scoped package it should remove @scoped container folder as well if there is nothing packages left inside.

@tgriesser
Copy link
Contributor Author

Btw potentially yarn allows to have folder starting with @ that is not a container fo scoped packages but a package itself, think this should be handled.

Interesting. What do you think would be the best solution then? Testing for the existence of / in any package name prefixed with @?

Also probably while the removal of scoped package it should remove @Scoped container folder as well if there is nothing packages left inside.

Yeah, I wasn't sure where the most appropriate place to put this check was.

@wclr
Copy link
Contributor

wclr commented Nov 21, 2016

@tgriesser
I've made this fix for myself too, you may check it wclr@94938a9 (it removes @scoped from queue only if it contains linked scoped packages inside)

So it passes removal test without fixes.
I would aslo added test for upgrade cmd?

Simlinks should be carefully handled. Currenlty yarn doesn't remove it if you say remove package it it is simlinked (not sure what should be correct behaviour - should it remove the simlink?)

Also there is RFC about link: dependency if you didn't see.
https://github.com/yarnpkg/rfcs/issues/25


for (const maybeLinked of maybeLinkedModules) {
const linkedPath = path.join(this.linkFolder, maybeLinked);
const isScopedName = maybeLinked.indexOf('@') === 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybeLinked[0] === '@' is probably nicer to read


const maybeLinkedModules = await fs.readdir(this.linkFolder);

for (const maybeLinked of maybeLinkedModules) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybeLinked isn't the best variable name here, how about dir?

for (const maybeLinked of maybeLinkedModules) {
const linkedPath = path.join(this.linkFolder, maybeLinked);
const isScopedName = maybeLinked.indexOf('@') === 0;
const isScopedContainer = isScopedName && !await fs.exists(path.join(linkedPath, 'package.json'));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hardcoding package.json isn't the best idea since internally we're really agnostic. I'd recommend leaving out this check since you can't have a package named @foo without having a slash in it anyway.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Though as I know @scoped packages is npm feature.

you can't have a package named @foo without having a slash in it anyway.

Right you can't have this name in package.json to be published to NPM registry, but yarn allows have a folder name that is differs from package.json name, for example:
"@some-package": "file:../some-package"
will create node_modules/@some-package and will treat as usual module.

(npm doesn't allow this) - so maybe this is not a feature, but a "bug"?

@tgriesser
Copy link
Contributor Author

Updated based on @kittens comments. Not sure what I need to fix the failing windows test though.

@torifat
Copy link
Member

torifat commented Nov 22, 2016

@tgriesser I have figured out the problem.

In Windows stat.isSymbolicLink() return false for yarn-install-dont-overwrite-linked-scoped-<random>/node_modules/@fakescope/fake-dependency.


Update Problem is in Windows, yarn-install-dont-overwrite-linked-scoped-<random>/node_modules/@fakescope/fake-dependency is not actually a symlink.

image

@tgriesser
Copy link
Contributor Author

I tried moving the symlink into the beforeInstall of the test helper, but now it's breaking on all CI's, though it's passing locally on my mac. Any ideas?

@bestander
Copy link
Member

@tgriesser, could you rebase first?
As for error, it seems to not able to find the path


(node:24521) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: ENOENT: no such file or directory, symlink '../../dir-to-link' -> '/tmp/yarn-install-dont-overwrite-linked-scoped-0.563408474619264/.yarn-link/@fakescope/fake-dependency

@torifat
Copy link
Member

torifat commented Nov 26, 2016

@bestander his initial PR was only failing in Windows. I tried to debug it and found it's a problem with symlink.

I tried this too: https://github.com/jprichardson/node-fs-extra#windows

Scoped dependencies are nested one folder deeper than traditional
dependencies, the possiblyExtraneous needed updating to work with the
actual dependencies, not the entire scoped folder.
@tgriesser
Copy link
Contributor Author

tgriesser commented Dec 6, 2016

Alright, I believe I finally got this one. Needed to ensure the directories were created before the symlink in order for the tests to pass.

Also based on the changes made in #2036 - in order to properly remove scoped directories when all modules have been removed, we need to maintain a list of these directories separately and check after the copyBulk step. Adding the parent (scoped) directory to the possiblyExtraneous list directly will mess things up in the copyBulk step.

@bestander bestander merged commit 6cca59f into yarnpkg:master Dec 7, 2016
@tgriesser tgriesser deleted the fix-1214 branch December 7, 2016 18:25
@wclr
Copy link
Contributor

wclr commented Jan 2, 2017

I seem that it wan't published yet? BTW how is it possible to know if a PR was released in published version?

@bestander
Copy link
Member

@whitecolor, no, it is still on master branch, a release is coming next Monday the latest.
To see if commit is released click on it, e.g. 6cca59f.
screen shot 2017-01-03 at 2 35 55 pm

And then see in what branches it is present.

screen shot 2017-01-03 at 2 36 26 pm

Currently it is only in master branch, later you'll see 0.19-stable there

@wclr
Copy link
Contributor

wclr commented Jan 3, 2017

@bestander many thanks for explanation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants