Add LIBEXECDIR relative folder constant

[musm]

cc @staticfloat is this how we want the API ?

Base.LIBEXECDIR return ..\\libexecdir and users will need to do

exe7z = joinpath(Sys.BINDIR, Base.LIBEXECDIR, "7z.exe")

This mirrors how some of the other constants are computed.

[musm]

Maybe @vtjnash can also review.

If agreeable with all reviewers this needs a backport to the RC.

[musm]

I'll also add a basic 7z check test if I receive further approval on this.

[rapus95]

Why not assign an absolute path (like Sys.BINDIR * "../libexecdir") as it is done for Sys.STDLIB? Even though there might be other cases, I think it will be more useful since there is no situation I can think of where one wants the relative path. In other words, if you already propose usage like joinpath(Sys.BINDIR, Base.LIBEXECDIR, "7z.exe") then why not directly combine both?

On the other hand, after reading into the new Artifacts-system I became uncertain whether making this a public API is wanted as it could end in file/version conflicts in that directory if not properly managed. What do you think about it?

[staticfloat]

The reason why they're not absolute values is because we want Julia itself to be relocatable; if you compile Julia in one location, then move the executable (like when you download it off of the internet) that absolute path is no longer valid.

We actually do a surprising amount of work in the C runtime to special-case Sys.BINDIR such that you can write Julia code that has Sys.BINDIR baked into it and still have it work properly after things have been moved around. We don't want to duplicate that work for a bunch of other constants, so it's easier to have everything else be defined in relation to Sys.BINDIR.

We could, of course, export something like Sys.LIBEXECDIR() which does the joinpath() for you, but that would be a function rather than a String, and we'd then change the rest of the similar APIs as well.

[rapus95]

The reason why they're not absolute values is because we want Julia itself to be relocatable; if you compile Julia in one location, then move the executable (like when you download it off of the internet) that absolute path is no longer valid.

That's the exact reason, why I suggested:

Why not assign an absolute path (like Sys.BINDIR * "../libexecdir" ...

I really meant to assign that line into Sys.LIBEXECDIR (Though I guess the first run inlines that, thus inlining and resolving as constant would need to be delayed, or is that the actual problem?). Then it is built relative from bindir but always usable as an absolute path. Best from both worlds in my opinion.

EDIT: Would eval(quote Sys.LIBEXECDIR=Sys.BINDIR * "../libexecdir" end) work here?

We could, of course, export something like Sys.LIBEXECDIR() which does the joinpath() for you, but that would be a function rather than a String, and we'd then change the rest of the similar APIs as well.

Is there any way to somehow fill a variable on every new run? (Preventing @pure inlining and constant propagation?) That'd be the way how to fill such a variable. Somewhat like a static const in Java which would be filled upon loading of the class.
If that all doesn't work, then how about providing some function like Sys.dir(:LIBEXECDIR), Sys.dir(:BINDIR) etc or maybe even macros for them? In my opinion the fact that BINDIR gets special casing should be entirely hidden thus making the public api to access the LIBEXECDIR in the same way as the BINDIR would be important for universality. (just my 2 cents)

Well, at the end it might be the best to just screw that and only make accessing those execs via artifacts part of the public api. I guess that would be the best solution anyway.
What do you think about my Artifacts uncertainty?

[KristofferC]

FWIW

julia> Base.INCLUDEDIR
"../include"

julia> Base.LIBDIR
"../lib"

are already relative so might make sense to keep things consistent.

[rapus95]

FWIW

julia> Base.INCLUDEDIR
"../include"

julia> Base.LIBDIR
"../lib"

are already relative so might make sense to keep things consistent.

So the consistency between BINDIR and the relative ones comes from the different module location I guess. whatever DIR is found in Base is to be assumed relative. For Sys it is assumed to be absolute?

[musm]

OK test failures are unrelated ( LoadError: LoadError: "hard kill repl test"
) . This is ready on my end.

[musm]

It is certainly possible to add a runtime global const that gives you Sys.LIBEXEC

[musm]

However after discussion, adding a runtime global const that gives the absolute path isn't the best course to take. There are plans to replace the whole 7z machinery et. al with equivalents in pure Julia , which will avoid their need completely. As that is the case @staticfloat and I agree that this is a good solution until then, since we should opt for consistency.

[musm]

Should we merge this and add it to 1.3

Yeah I was initially very pro LIBEXECDIR but given the larger plans it's a bit annoying to introduce it then potentially remove it in a future release. However, I suppose it's not so bad since it's guaranteed to only hold 7z and the name of the folder is a variable so it can change (even if an exotic configuration) and thus it's safer to introduce it for robustness.

[rapus95]

Should we merge this and add it to 1.3

Yeah I was initially very pro LIBEXECDIR but given the larger plans it's a bit annoying to introduce it then potentially remove it in a future release. However, I suppose it's not so bad since it's guaranteed to only hold 7z and the name of the folder is a variable so it can change (even if an exotic configuration) and thus it's safer to introduce it for robustness.

How about labeling it as 1.4 only in the docs and adding a description/link to the upcoming system?

[musm]

This is more relevant to address in 1.3, since the libexec changes happened in this release.

[rapus95]

Ah, well, of course my question should be rephrased: How about specifying that the constant will be available only until version 1.x where it will be replaced with a new, better system? Am 12. November 2019 16:42:31 schrieb "Mustafa M." <notifications@github.com>:

…

This is more relevant to address in 1.3, since the libexec changes happened in this release. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[nalimilan]

I don't think we need to remove this in a future release. We also provide LIBDIR, which isn't going to be removed, and the cost of an additional variable is quite low.

[musm]

@staticfloat if you approve can you merge this or are you waiting on someone else to also review?

I don't think we need to remove this in a future release. We also provide LIBDIR, which isn't going to be removed, and the cost of an additional variable is quite low.

Sure, but at the same time I think it will be empty on windows after removing 7z since it's the only exe that lives in the folder on Windows. (Not arguing against, just thought I should mention this)

[staticfloat]

LIBEXECDIR being empty is fine; there's no reason to drop it; it's a standard filesystem path.

[rapus95]

Those are where my question was pointing to, whether the usage by packages will soon be discouraged in favour of the new artifacts system? Am 12. November 2019 18:28:34 schrieb Milan Bouchet-Valat <notifications@github.com>:

…

I don't think we need to remove this in a future release. We also provide LIBDIR, which isn't going to be removed, and the cost of an additional variable is quite low. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[nalimilan]

@KristofferC Do you think this can be backported to 1.3? Since the 7z change has been introduced in that release, it would be cool to provide the solution to access it too.

[KristofferC]

@staticfloat, please squash PRs like this.

[KristofferC]

It's on the backport branch now.

[musm]

Thanks @KristofferC !

[musm]

BTW since this and the other constants defined in the same file are private, this doesn't need a news item right? (In all the other constants are undocumented)