Make self update work on top of run mechanism

[ibilon]

#284 (comment)

[nadako]

Ooh, you should've just force-pushed into previous PR branch, that PR has some interesting discussion, but oh well.

[andyli]

Thanks a lot for the work!
I've just reviewed the design, but haven't looked into the code yet. It sounds pretty good to me.

I can see that there are still issues. I describe them below, follows with a proposal to mitigate.

1. This PR's implementation depends on how haxelib run works. What if we want to change that in the future? e.g. It has been discussed we may replace "run.n" with a "Run.hx" that is run by the macro interpreter, such that we can remove the neko dependency.
2. The existing haxelibs may not be able to perform selfupdate. We should document the issue, but I guess we cannot assume everyone will read...

My proposed solution: Implement a way to prevent people with certain old haxelib version downloading new versions of "haxelib_client". We can let the haxelib client add an additional GET param that states it version when downloading libraries. The server will return an error if the requesting client has an version incompatible to the new version. For existing haxelib clients, they will not send out this GET param, thus the server will know that it must be an old client.

What do you think? If you agree and don't have another design, you may merge this PR and we can start to work from here.

[nadako]

This PR's implementation depends on how haxelib run works. What if we want to change that in the future? e.g. It has been discussed we may replace "run.n" with a "Run.hx" that is run by the macro interpreter, such that we can remove the neko dependency.

haxelib run already supports interp version, if you specify main field in the haxelib.json file, that should be enough. I didn't really review the changeDir stuff in the PR, it may have some issues if we're going to change the behaviour regarding cwd in run (see #272).

The existing haxelibs may not be able to perform selfupdate. We should document the issue, but I guess we cannot assume everyone will read...

Actually, on windows I think they will be able, since it nekoboots a new executable. The problem is with unixes that was discussed in #284

My proposed solution: Implement a way to prevent people with certain old haxelib version downloading new versions of "haxelib_client". We can let the haxelib client add an additional GET param that states it version when downloading libraries. The server will return an error if the requesting client has an version incompatible to the new version. For existing haxelib clients, they will not send out this GET param, thus the server will know that it must be an old client.

I wonder if that can/should be implemented by using our "api version" mechanism. This requires more thought as we clearly don't want to introduce a lot of code for such migration.

[ibilon]

Point one: Indeed we are dependent of how run works, but on the other hand we use it here because we know it's working really well.
If we change how it works then haxelib is just like all affected libraries.

Point two: no they can, and it'd work as long as they keep the first "new" haxelib the upgrade to, unless it's broken already for them (bundled nekobooted version from an haxe installer on linux) but nothing we can do here. (edit, just like @nadako said)

Interesting solution, so filtering the result of SiteApi.infos based on the transmitted version (or lack thereof) of haxelib and if an error is thrown it'll get printed, maybe too much just for that, and not sure how we'd actually do it.

The changeDir is true by default and will do the same thing as before, it's set to false when running the haxelib_client so that it's behaving just like if you had launched it from the directory where you launched haxelib.

[nadako]

Okay, I looked at the changeDir logic. It feels a bit weird to use different run mechanism here.

Maybe it would be better to conform the current run protocol and check for HAXELIB_RUN env variable, pop the cwd from the last argument and do Sys.setCwd again. That way we can run with both haxelib run haxelib_client and just haxelib. We also don't need to add --safe arg when calling haxelib_client from haxelib, just check for EITHER --safe or HAXELIB_RUN env var (if we do that, we can rename --safe to something more intuitive, but i'm not sure what exactly atm).

Personally, I'm not a fan of this cwd dance, but at least it would be consistent. And we have another issue about reworking that (#272).

[nadako]

As for forbidding updates, another option would be to just rename the library, so it won't be updated for older clients, but newer ones will check for the new name. I think I like this approach, since it's not like people are expecting selfupdate to work anyways :)

[ibilon]

For the changeDir, that would indeed allow to let run as is, I'll change it.

I like the renaming approach, and it would fully work, how about just haxelib?

[nadako]

I like the renaming approach, and it would fully work, how about just haxelib?

Ok with me

[nadako]

@andyli do you think this gets it all covered?

[nadako]

I'm itching to merge this \o/

[ibilon]

The lib name isn't changed yet, should other people vote on it? Not like it's a big deal.

[nadako]

I don't think anyone besides Andy cares as long as it works :)

[andyli]

Yeah, I think the plan is all good now.

I like renaming "haxelib_client". I understand the existing haxelibs may selfupdate correctly, but they may not as well, so I think it is good to force everyone to get the new haxelib alongside with haxe 3.3, and selfupdate will work reliably from now on.

I still think that it would be useful to send additional params when the clients request to install new libs. It could gives us statistics about the number of currently active haxe/haxelib versions etc. But it could be implemented later.

So, please go ahead to change the doRun("haxelib_client") to doRun("haxelib") (and other related parts), and then merge as you wish : )

[nadako]

Okay, then I'll merge this and then change library name and cleanup leftovers.

After that I think there's some work to do regarding installers, linux/homebrew packages and whatnot. Will you handle that @andyli?

[andyli]

No problem, I will handle those. It is merely changing the haxe Makefile to use a nekobooted haxelib instead of the scripted version.

[nadako]

Merged manually in b4799e8

[andyli]

Oh, one more consideration:
When haxe 3.4/4 is released and bundle a even newer haxelib. The user's "haxelib haxelib" may still exist with an older version. So the newer haxelib may call the older haxelib... Should there be a sam-ver check to ensure the newer one is always prefered (except --safe)?

[nadako]

Hmm, maybe we should check if installed version greater than current!

[nadako]

I made a meta-issue with current stuff related to this: #293