-
Notifications
You must be signed in to change notification settings - Fork 30.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
vm: move options checks from C++ to JS #19398
Conversation
lib/vm.js
Outdated
@@ -206,13 +258,20 @@ function runInNewContext(code, sandbox, options) { | |||
options = Object.assign({}, options, { | |||
[kParsingContext]: sandbox | |||
}); | |||
return createScript(code, options).runInNewContext(sandbox, options); | |||
return createScript(code, options).runInContext(sandbox, options); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this should still be runInNewContext
right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right, thanks.
src/module_wrap.cc
Outdated
|
||
MaybeLocal<Integer> GetLineOffsetArg(Environment* env, | ||
Local<Value> options) { | ||
Local<Integer> defaultLineOffset = Integer::New(env->isolate(), 0); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Style: default_line_offset
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just moved those functions from node_contextify.cc without modifying them. I'd prefer to leave them like that because the goal is to completely remove them in follow-up PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I reverted that move. It isn't necessary.
src/module_wrap.cc
Outdated
|
||
MaybeLocal<Integer> GetColumnOffsetArg(Environment* env, | ||
Local<Value> options) { | ||
Local<Integer> defaultColumnOffset = Integer::New(env->isolate(), 0); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto.
src/module_wrap.cc
Outdated
return defaultColumnOffset; | ||
|
||
return value->ToInteger(env->context()); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These two functions have a lot of common code that can be factored out.
src/node_contextify.cc
Outdated
Utf8Value name_val(env->isolate(), options.name); | ||
ctx->AllowCodeGenerationFromStrings(options.allowCodeGenStrings); | ||
ctx->SetEmbedderData(ContextEmbedderIndex::kAllowWasmCodeGeneration, | ||
options.allowCodeGenWasm); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you line up the argument?
src/node_contextify.cc
Outdated
Local<String> filename = args[1].As<String>(); | ||
|
||
Local<Integer> lineOffset; | ||
Local<Integer> columnOffset; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Snake case.
src/node_contextify.cc
Outdated
bool display_errors = maybe_display_errors.ToChecked(); | ||
bool break_on_sigint = maybe_break_on_sigint.ToChecked(); | ||
CHECK(args[0]->IsNumber()); | ||
int64_t timeout = args[0].As<Integer>()->Value(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The check doesn't match the cast. It's not exactly wrong because Integer::Value()
does the right thing regardless but stil..
You could instead use IntegerValue()
(but please use the overload that takes a Local<Context>
.)
src/node_contextify.cc
Outdated
int64_t timeout = args[0].As<Integer>()->Value(); | ||
|
||
CHECK(args[1]->IsBoolean()); | ||
bool display_errors = args[1].As<Boolean>()->Value(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe just args[1]->IsTrue()
?
src/node_contextify.cc
Outdated
bool display_errors = args[2].As<Boolean>()->Value(); | ||
|
||
CHECK(args[3]->IsBoolean()); | ||
bool break_on_sigint = args[3].As<Boolean>()->Value(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Last two comments apply here as well.
src/node_contextify.h
Outdated
v8::Local<v8::String> name; | ||
v8::Local<v8::String> origin; | ||
bool allowCodeGenStrings; | ||
v8::Local<v8::Boolean> allowCodeGenWasm; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
snake_case
This mixing of bool
and v8::Boolean
is kind of weird although I understand why it is that way.
Partial dup of #19268 |
f2091f7
to
107feca
Compare
@TimothyGu thanks for pointing it out. Let's focus on landing #19268 and I'll update this. |
Interesting side-effect: there is a non-negligible perf improvement:
|
a4f60a4
to
3233dcf
Compare
lineOffset = 0, | ||
columnOffset = 0, | ||
cachedData, | ||
produceCachedData = false, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd like to take the opportunity to do stricter verifications of the *Offset
and produceCachedData
options. Good idea?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Go ahead!
lib/vm.js
Outdated
} | ||
const { | ||
displayErrors = true, | ||
breakOnSigint = false |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ditto: I'd like to validate the type of all options
Rebased. I still have a question about options validation. |
c42dcdd
to
8ddc3a2
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good work! Left some comments
lib/vm.js
Outdated
} else { | ||
timeout = Number(timeout); | ||
if (timeout <= 0) { | ||
throw new ERR_OUT_OF_RANGE('options.timeout', 'a positive number'); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you pass the timeout to the constructor so it gets displayed in the error message?
lib/vm.js
Outdated
throw new ERR_INVALID_ARG_TYPE('contextifiedSandbox', 'Object', sandbox); | ||
} | ||
if (!_isContext(sandbox)) { | ||
throw new ERR_INVALID_ARG_TYPE('contextifiedSandbox', 'vm.Context'); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you pass sandbox to the constructor?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because at this point, sandbox
is guaranteed to be an object, it seems confusing to me if we print "Received type object" in this case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@targos Then perhaps this should be ERR_INVALID_ARG_VALUE
?
lib/internal/bootstrap/loaders.js
Outdated
const script = new ContextifyScript(code, options); | ||
return script.runInThisContext(); | ||
function runInThisContext(code, filename) { | ||
const script = new ContextifyScript(code, filename); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At this point do we even need this helper? Maybe just inline it in the try-catch block?
Updated again. PTAL |
2626409
to
c0cea8a
Compare
CITGM found some real bugs 🎉 ! Fixing... |
So... This breaks gulp 3 because it depends on
How do we proceed? /cc @nodejs/tsc |
Gulp 3 depends on a very old version of vynil-fs (0.3.0) https://github.com/gulpjs/gulp/blob/v3.9.1/package.json#L41. vynil-fs since version 2.0.0 uses gracefuls-fs@4, which does not depend on natives anymore. IMHO we should see if it's possible to update gulp@3 to vynil-fs@2. @phated what do you think? |
Yeah, that was already a known issue two years ago, see #5213 and #8149 (comment) from @phated. I think we made peace with the fact that projects that don't upgrade stop working sooner or later. We probably don't want to back-port but for sure it can go into Node.js 10. gulp 4 is not affected, by the way, see gulpjs/gulp#1604. |
I agree with Ben, but just in case it matters: I have taken over maintenance for |
Unfortunately
|
@jdalton gulp@3 depends on vinyl-fs@0.3.0, which depends on graceful-fs@3.0.0. The approach used in graceful-fs@3.0.0 is deprecated and unmaintainable. If gulp@3 wants to keep working on Node 10, it needs to update vinyl-fs to graceful-fs@4.0.0 (I'm not sure what this would entail). gulp@4 is unaffected. |
So
So It would be great, if Node needs action from folks maintaining projects, for you all to clearly spell out what is requested from each project (pointing to line numbers, versions, and possible solutions). This might be done by opening issues on those projects, to give project specific context, and linking the issues back here or #19786 for broader discussion. |
@mcollina Yeah, that's the interesting thing. We would have to push a patch release to vinyl-fs 0.3 that relies on graceful-fs v4, but IIRC there were more changes in graceful-fs v3 -> v4 that were breaking changes that would bubble up to vinyl-fs and thus gulp. I'd rather not commit semver violations (and break our users code) if we don't absolutely have to. If there was a clean patch release made to graceful-fs v3 then we don't even have to do anything - it will get automatically pulled in by vinyl-fs 0.3. I'm not sure why we're being pulled into this when the breakage isn't in our project, but in one of our dependencies. If any project should be backporting fixes it should be graceful-fs (not that this is their fault - see #8149 (comment)). Correct me if I'm missing information here as I've just hopped into this, I know @phated has been dealing with this for a long time (#8149 (comment) 2016) and I feel like our response from gulp has been pretty consistent. |
@contra as far as I understand, the breaking changes in graceful-fs@4 are needed because of the removal of
As far as I get it, you have been pinged by @jdalton because we are at an impasse. The net result would be that gulp@3 would likely not work on Node 10, or maybe 11 (if @addaleax can manage to get us those extra 6 months). On the good side, it might be a good moment to push for upgrading to gulp@4. |
Fundamentally this is a problem in |
FWIW I had originally blocked on the PR and only removed my -1 based on the commitment that Node 10 would be shipped in a way that would not break Gulp 3. If we cannot find a solution before the release I will be opening a revert |
If gulp@3 cannot or does not want to upgrade, then it's going to break sooner or later anyway. Reverting is pointless, that's just putting off the inevitable. |
+1 to what bnoordhuis said, if we were trying to keep all packages working forever node wouldn't have versions |
@bnoordhuis if gulp@3 cannot upgrade and it is going to break an insignificant number of developers who are not going to understand why we should have a very very good reason to do so. While I find this PR to be a good thing, I do not find it compelling enough to justify the breakage |
@MylesBorins I would've agreed if this is breaking gulp@4, but I don't think Node v10 breaking gulp@3 should that blocking. This is putting a burden both on Node and gulp. If the user cannot upgrade to gulp@4, then it's reasonable to expect things would stop working on a new major version of Node. People who have to stay with an older version of gulp can also stay with an older of version of Node to be safe. |
Just for context, gulp@4 is only a pre-release at this point. But yes, I agree, if gulp doesn’t want to be broken, then at some point they’re just going to have to release a version of gulp that uses public, semver-covered APIs… 2 years is plenty of time to do that. |
This thread is absolutely ridiculous. Many of your are literally paid to work on this stuff and many others have jobs that allow you to work on it (on or off time). I'm getting blown up by this thread in the middle of interviews and yet none of you have offered to help write docs or get your companies to contribute funding for the project. I've given 2 years to this project, without a job, unpaid and now you demand me to do more work for free so you won't break our publicly available and documented project or expect me to push out an un/under-documented version? I seriously can't believe this. |
@phated I am sorry about what you've been through but the amount of paid labour in Node core is probably much less that you thought. In case you didn't know, at least three Node.js collaborators on this thread are students. By referring to gulp, we do not mean you personally, but the project in general. Node.js changes breaking unmaintained packages is not uncommon and I'll have to say, if things are just unsustainable, it's fine to drop things and move on. By keeping undermaintained pakcages working with newer version of Node probably would do more harm to the ecosystem in the long run. |
@joyeecheung the project is not unmaintained - it's just undermaintained (by only me). The pay is not "much less than I thought", as I'm guessing I have much more insight into companies/politics/etc. If you are an unpaid person working on node core, you should really take a step back and realize how much money people are making off your back. |
@phated Serious question: If somebody here prepared an alternative But yes, we do respect that you are already putting a ton of work into gulp and we do know that Node.js is special in the sense that some people are getting paid for the open source work that they are doing. And we certainly don’t want to break gulp, or put it in a bad light – I’m really doing my best to keep
This might just be me, but I don’t think anybody here has an opinion like that. |
@addaleax We can't do your suggestion because gulp 4 is published on npm as the
Sorry, I meant that in relation to @joyeecheung's comment about it being unmaintained - especially because I've been maintaining it daily until very recently. |
I'll reiterate that I think the best (and most correct) solution here is for a non-breaking graceful-fs v3 patch to be published. As long as the API has no breakages, I don't understand how swapping the underlying native library is impossible do to as a patch. I'm willing and able to do whatever is necessary to patch/whatever on our end if need be but a new major release it out of the question. As an aside, we really don't need to be discussing gulp, our project, our maintainers, etc. - not really relevant, thanks for worrying about us but we're okay thanks. Let's leave that out of this and just focus on fixing this please. |
Can't graceful-fs just be updated in a patch version in the Gulp 3 branch? |
@benjamingr that's not how semver works. Sorry. |
Sorry if I came across that way, I was aware that it is still maintained at the moment, but your comments sounded like the maintenance is unsustainable in the sense that the maintainer is not paid and is not in a situation where that is OK, which means it is possible to go unmaintained, even if that would be very unfortunate. By saying Node.js changes breaking unmaintained packages is not uncommon, I was not talking about gulp as it currently is. I was talking about our approach on breaking unmaintained packages in general. I respect your work on gulp and I hope it does not go unmaintained, of course, and I certainly recognize that it is unfair to expect packages get maintained automatically, but I also think it is unfair to expect Node.js core to maintain that level of compatibility - compatibility of undocumented internals - automatically. I was not telling you to ship an underdocumented version, and again, sorry if I came across that way. If I understand the situation correctly, it's not urgent that gulp@4 must be shipped as soon as Node v10 is out - gulp@3 still works on multiple active release lines of Node.js. Things might be urgent once all the active release lines go EOL but by then gulp@4 may not be underdocumented anymore. Also to my understanding a lot of users don't even start to adopt an even-numbered version until it goes LTS.
Well, I am writing in the middle of the night during a national holiday..probably says a lot about me.
I will recuse myself from further commenting on those issues, sorry. |
@phated People generally do perform semver-major updates of dependencies in patch releases if that doesn’t break the dependent module, so I guess it might be helpful to ask: In what way does |
@addaleax The issue is that graceful-fs v4 contains more changes than just this fix, most importantly the change from patching the Our code doesn't rely on this behavior, but our users (especially windows users, where The last fix for this was backported to v3 here - isaacs/node-graceful-fs@331ac8d so if the |
I have an alternative view on this. Once 10.x goes into LTS it will be the primary link on the website of where we drive download. Gulp has close to 1M downloads a week and is embedded in TONS of tools which are not necessarily going to get an upgrade in the near future. This is setting people up for failure. The real issue here is not node, gulp, vinyl-fs, graceful-fs... but the plethora of tools sitting on top of that stack and the arguably millions of people relying on those to keep working the way they expect it to. Those are the individuals we need to be focused on supporting, collectively. I still have not seen a response in this thread about why this particular change is time sensitive and important enough to break such a widely used tool in our ecosystem in an LTS release. At the very least we can kick this off by 6 months and try to actively work towards making a better solution. This is not too different from the issue we have seen with buffer constructor in that each Semver-Major this issue rears it's head and makes our lives difficult. We've applied different fixes, but have not found a way to fix this permanently. My personal feeling is that if we can't land this in a way that doesn't break gulp 3 it shouldn't land in a branch that will go LTS, full stop. We can always keep this on master and actively work towards fixing the breakage before we cut 11... I am not sure what "fix" means... but it is obvious that we all need to work together and be strategic about this. |
@MylesBorins ❤️ ❤️ ❤️ ❤️ ❤️ |
node code gets left behind in the dust all the time, otherwise we would still have api compatability with node 0.10. if it makes people super uncomfortable to be updating internals that older package release rely on we can put up a notice "this release changed stuff with natives that really big modules like gulp depend on etc etc feel free to stop by the gulp issue tracker to help them reach version 4 and by the time node 10 is LTS the issue should have solved itself. any individuals or companies with their own internal gulp setups should know what a major node release means and that packages on npm published before that version of node was released might not work. |
@MylesBorins, I get where you're coming from but there's an alternative view here that needs to be considered: any code that is knowingly or unknowingly using a third-party modified version of a core module is most likely going to break no matter what we do and could break at any time. Even semver-patch and semver-minor changes could be breaking in such cases. We saw that happen with the introduction of and movement of code to Use of and dependency on modules like The 10.0.0 is not an LTS release. We have six months until 10.x becomes an LTS release. We have reverted semver-majors within LTS-bound release streams before if the ecosystem troubles become too great of a burden. I'm -1 on reverting this in 10.0.0 but I'm open to considering a revert before 10.x goes into LTS. In the meantime, let's see if we can get gulpjs more support in getting v4 out the door. |
@MylesBorins I don’t know if your comment implies something else, but this particular issue is fixed on the |
@jasnell I think that is a reasonable compromise, although I would like to see if we can get a temporary fix in before 10.0.0 and am willing to spend some time trying to help out with it. Although it seems like @addaleax is implying that this has already been fixed... so I am a bit confused. I likely should boot some things up and see what breaks 😄 @addaleax by "Fix" I meant long term solution... I don't know what that is. |
@addaleax If the issue is fixed in |
I tried to keep the type checking as it was done in C++ (not very strict for some options).-> Now all options are checked more strictly.Checklist
make -j4 test
(UNIX), orvcbuild test
(Windows) passes