Add support for working with shell options

[ghost]

Problem

The double-star (**) glob syntax, meaning a deep recursive directory walk, is commonly used, and is available in Task in sources and generates directives by default.
However, like in other shells, the shell in Task requires the globstar option to be set for the syntax to work.

Because shells are isolated across cmds run by Task, creating a task that uses globstar will require to reset the option in every single command:

tasks:
  default:
    cmds:
      - shopt -s globstar && bin1 ./**
      - shopt -s globstar && bin2 ./**

This is hard to write and read.

Other useful shell options have to be handled in a similarly verbose way.

Request

Allow to set shell options per-task and apply such to all commands.

tasks:
  default:
    shopts:
      - globstar
    cmds:
      - bin1 ./**
      - bin2 ./**

To match generates and sources, the globstar shell option itself should be enabled by default, while any others are to be controlled by the new Taskfile options.

[andreynering]

Original discussion: https://discord.com/channels/974121106208354339/1035164287695589457

If we were going to support a shopts: option, it should certainly be available globally as well, and not only at task level.

But I'm not sure it's really needed. I'm more inclined to just enable globstar for every command and that would likely be enough.

Anyone disagrees?

[pd93]

I'm more inclined to just enable globstar for every command

I don't have any objection to this. My only concern is that for anyone using ** in their Taskfile without globstar set, this will be a breaking change. As this kind of globing is used for fetching a list of files, this could potentially be dangerous if anyone is passing the result into rm or similar.

To be clear, I think this is a very uncommon scenario, but worth considering

[andreynering]

Good point about backward compatibility. Maybe we have to have the option then, and delay the default change to the next major version only.

[ghost]

I'm more inclined to just enable globstar for every command and that would likely be enough.

I am not sure if such are supported by mvdan/sh, but there are various other globbing options, such as extglob, which popped up in #225. Not all options can be reviewed for being enabled or disabled by default, so I think there should be manual user control over this.

To be clear, I think this is a very uncommon scenario, but worth considering

There is a lot of tooling that only operates on files, most being formatters and linters like clang-format or flake8. This is a huge use case for build tools, which Task claims to be. To be a good one, Task needs to be able to work well with such tools, and so has to be able to glob well.

this will be a breaking change

While it will be a breaking change, considering that ** without globstar is equivalent to *, it will be a tiny one. It still probably only has to come in the next major version.

[pd93]

@sernot agreed with everything you said. When I said

I think this is a very uncommon scenario

I was referring to the number of cases where this would be a breaking change, not the use of globstar itself. As you said, not many people (if any) will be using ** without globstar enabled, but on the slim chance that they are, I think the smart thing to do is to defer this to the next major version.

[pd93]

I've opened another issue to track changing the default value. This issue can remain focussed on a shopts keyword or similar.

[voltangle]

Gonna drop my two cents here as well
I am thinking about something like this:

- shopts:
    - globstar
    - shell: zsh

Or (i like this one more):

- task-name:
    - shell: zsh
    - shopts:
        - globstar

So basically, adding an ability to include a shell that will be automatically used in every CLI call, will (possibly) eliminate the requirement of having a shebang on top of a script, bc it will not be required if ran from a shell, and also adds an ability to use features exclusive to a shell

This probably requires a separate issue, but idk, maybe related to this one

[pinpox]

While globbing/globstar is an important option it is not only one. If shell options are to be made configurable, please make it in a more general way, allowing to set the equivalent of set -e and set -o pipefail aswell.

I stumbled across this issue while trying to find a workaround for #392

[cdfa]

I would love to see #929 merged. A task like:

pass:
    cmds:
      - set -o pipefail && exit 1 | echo "Hello"

as suggested in #392 (comment) did not work for me (platform: darwin-aarch64, go-task 3.18.0). Hello gets printed.

Mysteriously, set -xo pipefail does properly enable pipefail, but it messes up the output (which might also be considered a bug?).
I also cannot disable the x option afterwards with set +x.

[pd93]

@cdfa I agree that it would be nice to get this merged. However, as it stands, the implementation isn't perfect. We'd like to see mvdan/sh#962 merged so that we can tidy it up a bit and I simply haven't had time to look into the upstream code yet.

In the meantime, I've tidied up the PR a bit. Hopefully we can get it merged with the "workaround" in place and make some improvements later.