Add ability to configure which type of shell to use for executing shell tasks

[nat-n]

Addresses #15 and #27

The new shell executable resolution logic only considers the $SHELL env var as a hint (if it appears to match the target shell type), and never explicitly looks for WSL on windows. See included updates to documentation for more details.

Example of a shell task using fish shell:

  [tool.poe.tasks.fibonacci]
  help = "Output the fibonacci sequence up to 89"
  interpreter = "fish"
  shell = """
    function fib --argument-names max n0 n1
      if test $max -ge $n0
        echo $n0
        fib $max $n1 (math $n0 + $n1)
      end
    end

    fib 89 1 1
  """

Note that there are a couple of significant limitations with the powershell integration that I don't know how to solve. Firstly when powershell starts it always prints some copyright boilerplate which is not really desirable for this use case, and secondly it doesn't propagate environment variables from the parent process which is quite a limitation (e.g. named args won't work). Suggestions are welcome from anyone more knowledgable about powershell.

[TBBle]

I would suggest doing this only when using powershell, not pwsh, as the two are not fully cross-compatible.

[TBBle]

Firstly when powershell starts it always prints some copyright boilerplate which is not really desirable for this use case

Both powershell (Windows Powershell) and pwsh (PowerShell, aka PowerShell Core, the cross-platform one) support -NoLogo to hide the banner, if that's what you're looking for.

I'd also suggest distinguishing pwsh and powershell, as the former is cross-platform, the latter is Windows-only, and there's plenty of things that pwsh has that aren't in powershell, and also a few things that are in powershell but not pwsh (mostly Windows-specific stuff that calls through into .NET Framework, though). It might be reasonable to use pwsh when powershell is specified, but I definitely wouldn't user powershell when pwsh is specified.

i.e. either separate powershell and pwsh, or treat powershell like posix (try a few variations), and pwsh like bash ("accept no substitutes").

secondly it doesn't propagate environment variables from the parent process

This is bizarre, and not what I'd expect. I also can't replicate it trivially here, setting an env-var in cmd and then starting Powershell and Windows PowerShell see the env-var.

Microsoft Windows [Version 10.0.19042.1415]
(c) Microsoft Corporation. All rights reserved.

C:\Users\paulh> echo %BOB%
%BOB%

C:\Users\paulh> set BOB="Hi"

C:\Users\paulh> pwsh -nologo

C:\Users\paulh
>echo $ENV:BOB
"Hi"
C:\Users\paulh
>exit

C:\Users\paulh> powershell -nologo
PS C:\Users\paulh> echo $ENV:BOB
"Hi"
PS C:\Users\paulh> exit

C:\Users\paulh>

So I don't think this is an endemic issue for PowerShell per-se.

I wonder if this is to do with the way it's launched? I don't see any command-line args, but perhaps it's different if you explicitly pass -Command - to execute stdin, in case that makes a difference?

In fact, I'd do that anyway, as it seems pwsh at least behaves differently. Continuing on from the above, and only the first line is typed by me, the rest is output

C:\Users\paulh> echo echo $ENV:BOB | pwsh -NoLogo

C:\Users\paulh
>echo $ENV:BOB
"Hi"
C:\Users\paulh
>

but

C:\Users\paulh> echo echo $ENV:BOB | pwsh -NoLogo -Command -

"Hi"

i.e. -Command - seems to make it friendlier for scripting, and isn't showing my prompt before each command.

[TBBle]

I just grabbed this branch, and given

[tool.poe.tasks.echo_pwsh]
interpreter = "pwsh"
shell = "echo ${ENV:test_var}"
env = { test_var = "roflcopter" }

this test passes:

@pytest.mark.skipif(
    not (shutil.which("powershell") or shutil.which("pwsh")),
    reason="No powershell available",
)
def test_interpreter_pwsh(run_poe_subproc, is_windows):
    result = run_poe_subproc("echo_pwsh", project="shells")
    assert result.capture == ("Poe => echo ${ENV:test_var}\n")
    assert "roflcopter" in result.stdout
    # Noise due to my local setup, pwsh needs `-NoLogo -Command -` to suppress.
    #assert result.stderr == ""

Powershell doesn't AFAIK have an equivalent of $0, and it would probably fail on my system, since my pwsh is installed as a .NET Global Tool for easier updating, and so (Get-Process -id $pid | Get-Item).Name returns dotnet.exe.

And -NoLogo -Command - on the pwsh command-line returns empty result.stderr correctly, by supressing the interactive noise I get due to a slow profile load-time.

(Quick hack to test that was in _locate_interpreter, but obviously this should be done more-cleanly)

        elif interpreter == "pwsh":
            result = [which("pwsh") or which("powershell"), "-NoLogo", "-Command", "-"]

This is testing on Windows 10, where a bunch of the other tests failed, I'm guessing your testing environment is Linux, so maybe PowerShell has a bug on Linux involving env-vars?

[nat-n]

Hi @TBBle, thanks a lot for the input!

So it turns out I just didn't realise it was necessary to include the $ENV: instead of just $ when referencing env vars 🙄.

Regarding powershell versions: I'm thinking that we should offer {"pwsh7", "pwsh", "powershell"}, where "powershell" matches any version that can be found with a preference for newer versions, and "pwsh" matches any version >=6, so if powershell 7 is present it will always be used. Does this sound sensible?

Do you agree that it's important to distinguish powershell 7 vs 6? I understand that 7 is backwards compatible since forever but does introduce new syntax. This implies checking the version of the pwsh executable and explicitly failing if it's too low.

Is it reasonable to assume that most windows devs or CI environments would have just one version of powershell installed which would be >= 6?

Is it necessary to acknowledge the distinction between Powershell <= 5 and .NET Core editions other than the version number?

In my limited testing I've found that which("bash") is not an entirely reliable way of locating git bash. So I assume that it's worth having the typical locations of powershell hard coded in the same way. Could you help be determine the list of paths that are worth checking?

[TBBle]

I think PowerShell 6 and 7 can be mixed together. The major distinction I've seen is between Windows PowerShell (versions < 6, based on .NET Framework, Windows only, powershell.exe), and PowerShell Core (versions 6+, based on .NET Core, cross platform, pwsh.exe, seems to just be officially referred to as PowerShell, "Core" has kind-of disappeared).

Basically, a bunch of new and nice stuff was added in PowerShell Core, but there are some things that may not work in PowerShell Core if they rely on .NET Framework APIs that are not present in .NET Core, so you can't (AFAIK) 100% assume a Windows PowerShell script works in PowerShell Core.

All interesting Windows machines (except Nanoserver container images, off-hand) have Windows PowerShell installed, it's been bundled for decades, at the hard-coded path already being tested. I think this is one of the reasons PowerShell Core is pwsh.exe, so you know you're getting PowerShell Core, and powershell.exe always gives you Windows PowerShell.

I'm not sure if there's a really good default path for pwsh.exe, as there's a few different installation methods including portable .zip and per-user installs. I'd expect it to be in the path as pwsh.exe if it's installed though. And given how aggressively it prompts for upgrading, I wouldn't expect to hit many PowerShell 6 installs in the wild, particularly because it fell out of support more than 12 months ago.

That said, pwsh may not be that wide-spread. I wouldn't expect it on CI machines unless I specifically asked for (or bundled it in my CI container image, more likely).

In the end, I wouldn't fuss too much with the PowerShell Core versions, if pwsh.exe works, then it's PowerShell Core, and if it's somehow a support-expired version or similar, that's between the script author and the user.

[TBBle]

I'm curious under what circumstances one wouldn't use sys.executable when running a python script?

[nat-n]

I hope it's never needed, in that python or python3 should be on the path in any sane environment, but if somehow they're not then sys.executable gives us the one thing we know about for sure which is the python interpreter of the current process.

[zmarffy]

Forgive me if I am wrong, but I think @TBBle may have meant that using sys.executable is the most desirable option, rather than the last thing in the or chain.

[nat-n]

I see, maybe I misread. That is actually what I thought originally when implementing the script task type, but then I changed my mind about it for reasons that I can't recall with confidence.

However my understanding at this point is that given the many ways that python versions can be installed and managed on a system, it's a safer bet to use whatever is configured in the current environment rather than whatever the poe executable is using.

For example poe might have been installed with the system python, but the task is being executed within a virtual env (perhaps activated by the PoetryExecutor or VirtualenvExecutor inside poe) which is probably more relevant and may reference a different python version (and site_packages) than the poe CLI is using.

[TBBle]

Ah, that's a good point. I've only used Poe installed via Poetry as a task runner, and did not consider a system-installed Poe.