Add option to run arbitrary command from project root

[casey]

Add a option which allows running a command from the project root:

# for example, to edit your Cargo.toml file from any subdirectory of your project
$ just --cmd vim Cargo.toml

[casey]

Closing, since I'm not sure this is really that important. Feel free to comment if you want this!

[casey]

Gah, I can't decide on this. I think I should just implement it and see how it feels.

[ssokolow]

Couldn't someone just write a cmd +args: task which does this in fewer typed characters, given that cmd is shorter than --cmd?

[casey]

They definitely could, and it would be more ergonomic, however because of the weird way that argument substitution works, arguments would be split in unexpected ways. For example, if the recipe was written:

cmd +args:
  {{args}}

Then:

just --cmd echo 'a b; c' d

would be different from:

just cmd echo 'a b; c' d

Since in the second case the ; would get subbed into the recipe and be executed as two commands, the second of which would fail, since c isn't a binary on the path.

[ssokolow]

Then it sounds like the proper solution is to add some annotation which allows any multiple-argument token to be switched from $* behaviour to "$@" behaviour, rather than special-casing this example.

[casey]

I agree that some kind of annotation would be a better solution, but it might not be worth it, given that this feature is probably not high priority. Unless there are other things that it would enable.

I think the best solution would actually be to do #161.

If there was a good shell which was implemented as a library, just could use that directly. Having access to the execution context of the shell would enable all sorts of really great features.

All just variables could be made available to the shell without having to export them and polluting the namespace of child processes.

x = 'a b; c'

foo:
  echo $x

1038 - 1145

and just foo would print a b; c. The {{...}} syntax could still be available for when you really want to do direct text substitution, but the common case would be to use $.

The shell runtime would distinguish between scalar values and list values (which zsh and bash already do), so if you had:

man args+:
  man $args

Then it would correctly forward two arguments to man in invocations like man 3 printf. #161 mentions some other features that this would enable.

I'm actually tempted to implement the rc shell in rust for this purpose. It's very simple, is decent to program in, and has support for arrays. (Actually, the only type of variable is arrays.) People generally think of it as being the Plan 9 shell, but it was actually first shipped with version 10 of Unix.

Of course, it has the huge downside of being different from what users have installed on their system and what they're used to.

What are your thoughts?

[ssokolow]

I agree that some kind of annotation would be a better solution, but it might not be worth it, given that this feature is probably not high priority. Unless there are other things that it would enable.

The problem is that there are very good reasons for having access to both $* and "$@" behaviours.

The space-splitting behaviour is necessary any time you want to pass a string of options to a subcommand without that subcommand taking up the one variadic position.

The quoting-preserving behaviour is essential for properly implementing any situation where you're receiving filenames in your variadic arguments.

To this day, legacy document-handling tooling like LaTeX (and, thus, LyX) is hampered by incompatibilities with spaces in filenames... in something meant to process books... books that tend to have spaces in their titles and, thus, intuitively, should have spaces in their filenames.

build-docs opts paths+:
	doc-thingy $opts "$paths"

If there's no syntax to make just build-docs '-vvv --format=rst' 'Thingy A.xml' 'Thingy B.md' produce doc-thingy -vvv --format=rst 'Thingy A.xml' 'Thingy B.md', then it's unavoidably a flaw.

I'm actually tempted to implement the rc shell in rust for this purpose. It's very simple, is decent to program in, and has support for arrays. (Actually, the only type of variable is arrays.) People generally think of it as being the Plan 9 shell, but it was actually first shipped with version 10 of Unix.

Of course, it has the huge downside of being different from what users have installed on their system and what they're used to.

That really is a huge downside because not only is rc strange and unfamiliar, I seriously doubt "Makefiles, with rc syntax" exists as a syntax definition for Just to piggy-back off of, and people rely heavily on syntax highlighting to make up for a limited understanding of a new grammar.

I'll need to get back to you on it when I'm not so tired though.

EDIT: I should clarify. "strange and unfamiliar" in the sense that it falls into the uncanny valley of shell scripting, looking just similar enough to make me very unnerved about what gotchas are likely to be lurking in my assumptions about how to use it.

(And, as someone whose experience with functional programming doesn't extend much further than loving things like first-class functions, list comprehensions, and generators in Python and reading up on some other concepts like tail recursion (so, more experienced than most mainstream programmers), I have an impulsive impression that seeing (~ (paren, tilde, space) as part of an ostensibly well-formated code example never bodes well. It tickles memories of LISP syntax and other languages where identifiers without alphanumeric content are allowed and at least some form of dynamically redefinable language grammar (eg. custom infix operators) madness reigns supreme.)

I still think the "gotta write syntax highlighting for 50 million different editors and wait for it to get into LTS distro releases in the case of the non-extensible ones" problem is more significant.

[casey]

Yeah, that difference between behaviors is exactly the reason that the rc shell has pervasive supports for lists. There's a great paper about it. I won't be able to summarize well, but it goes into detail about how string splitting and re-scanning/splitting of string variables contributes to the complexity of the Bourne shell.

It also has a terrifying quote:

..., but it suggests something darker: nobody really knows what the Bourne shell�'s grammar is.

Or at least, it's a terrifying quote when I think about trying to implement a Bourne shell in rust.

The point about syntax highlighting is a good one. And, don't get me wrong, I definitely think that not being sh compatible is a huge downside, both because of lack of syntax highlighting, and because alternatives to sh don't seem to be popular.

However, I think I might be able to implement rc in rust on my own, whereas sh in rust would be a lot more work. But, sh in rust would probably interest a lot of people, either as a library that they can embed into a program, or as a starting point for an extensible shell written in rust, so I might be able to get some help with that.

In terms of supporting this particular use case, we could introduce a function-call syntax, and provide a built-in function to quote items of an array. This could look like:

build-docs opts paths+:
  doc-thingy {{opts}} {{quote(paths)}}

This has a lot of issues though. For example, what happens when a string in paths contains a quote? Should just try to escape it? And if so, will it try to figure out if the task is a shell task or a shebang task, so that it can escape it appropriately depending on the rules of string tokens in whatever particular language?

Ultimately I think that trying to get too clever about integrating with a shell (or any other language that a task might be written in) that isn't part of just itself is probably a bad idea, since there are too many factors to control. Even though it's a lot more work, I think integrating with a rust-written shell library is the way to go.

[bb010g]

What about Ion for a shell?

[casey]

Ion is super cool, but seems to be a bit of a moving target at the moment. Also, I'm hoping for a shell which is very simple, is as close to sh as possible (or is sh compatible), and which works on windows, so that just can have exactly the same behavior on every platform.

[ssokolow]

..., but it suggests something darker: nobody really knows what the Bourne shell�'s grammar is.

Boy, that takes me back. It must have been somewhere between 5 or 10 years since I ran across that quote.

Ultimately I think that trying to get too clever about integrating with a shell (or any other language that a task might be written in) that isn't part of just itself is probably a bad idea, since there are too many factors to control. Even though it's a lot more work, I think integrating with a rust-written shell library is the way to go.

Agreed. Trying to puppet an external shell sounds like an exercise in insanity.

However, I think I might be able to implement rc in rust on my own, whereas sh in rust would be a lot more work.

Is it really necessary to write a whole Bourne-style shell parser to handle tasks with no shebang?

I'd always thought that Make and inspired tools processed each line in isolation using some kind of restricted subset of Bourne shell syntax.

Given that just actually has a proper way to specify full-syntax Bourne shell (use a shebang) and the aforementioned perception (not actually sure if it's a misconception), how much complexity is someone going to try to cram into a single line when it'd be much less painful to just use a shebang?

[casey]

I worry that providing a shell language that was close to but not identical to sh would be annoying when users ran into subtle incompatibilities.

Also, if it was a full fledged sh or rc shell, it would be less often necessary to drop into a system provided shell with a shebang.

[ssokolow]

The question is whether users expect that in the first place.

I don't know about you, but I've never seen a Bourne shell which implements the DOS batch file "prefix a line with @ to prevent echoing it" behaviour.

[kwshi]

By the way, now that set positional-arguments is a thing, the quoting/escaping concerns raised by this comment can now be avoided by another workaround like

set positional-arguments
cmd +args:
  "$@"

which ensures the individual arguments stay quoted.

[casey]

@kwshi Good point!

You're noticing a lot of places where set positional-arguments makes things easier. I wish I had thought of passing positional arguments to commands earlier, since it solves so many problems, I think it should have been the default from the beginning.

I'll most likely make it the default in 1.0 (see #769 and #808). It's a breaking change, since a custom interpreter set with set shell might not ignore additional arguments. But, since, almost all interpreters accept and ignore additional arguments, it's unlikely to be disruptive.

[casey]

I actually wanted this today, so I implemented it.

Of course, it's also possible to do this nicely with a recipe and set positional-arguments. In any case, 6 year issue closed!