nix dev command for local dev tools and scripts

[roberth]

Is your feature request related to a problem? Please describe.

Many projects define custom scripts, tools, etc that are intended for local use during development.
apps or packages are not a great place to put these, because they're not intended for public consumption (and probably don't even work outside their project's worktree).

The new attribute will align closely with the proposed nix fmt command #6087 (comment).

It can also support dev shells, or dev shells that do not run bash, by invoking a user-provided "shell launcher" dev command instead of Nix's subjectively inflexible nix develop behavior. By convention, this would be nix dev shell and perhaps nix develop could fall back to nix dev shell when devShells.<system>.default isn't set. This way, such shells are supported in a consistent manner.

Describe the solution you'd like

• A new flake attr devTools.<system>.<name>
• A new nix command nix dev <name> <args>

When nix dev runs, it looks up the name in the nearest flake.nix. If not found, proceed to a flake.nix in a higher directory.

When the <name> is found, it executes the main program (similar to nix run), forwarding the remaining command line arguments to the process. The process environment is extended with a FLAKE_DIRECTORY environment variable, as many scripts operate on files relative to the project root. Perhaps also a FLAKE_ROOT variable, which is different when the flake is a subflake.
Alternatively, it could chdir to the FLAKE_DIRECTORY and perhaps preserve the original working directory in FLAKE_INVOCATION_DIRECTORY.

Describe alternatives you've considered

• #!nix develop shebang scripts. These are non-trivial to write and require the flake and the script to agree on a name attribute for the environment.
• nix run. Pollutes (subjectively) the flake outputs and doesn't help with the working directory. Doesn't allow customization of nix develop.
• put commands and aliases in the devShell. This isn't lazy enough and doesn't help with script workdir anchoring.
• alternative name task doesn't fit interactive use cases like nix dev shell

Additional context

[tomberek]

Another approach that is similar to how git does it. Provide a CLI extension mechanism:

apps.<system>.nix-ABCD = ...;  #`nix-` prefix is special?
or
devTools.<system>.ABCD = ...;
or
cliTools.<system>.ABCD = ...;

would extend the CLI such that nix ABCD is the equivalent of nix run .#nix-ABCD or nix dev ABCD. This is already a design pattern familiar to git users.

Separate comment: I'm not a fan of nix dev as it is too close to nix develop and people already have a bit of confusion of when to use what. I can easily see someone making something like nix dev shell. Good/bad? Or can this new concept subsume all that? It provides a path to provide stronger nix develop/nixpkgs develop capability without forcing the nix tool itself to follow nixpkgs stdenv development.

[terlar]

I'm guessing if this is adopted it could also potentially be a basis for fixing the issue #6124.

My initial reaction is also concern about potential confusion of nix dev and nix develop. But do find the concept interesting.

[roberth]

Provide a CLI extension mechanism:

That's not really what I was going for with this. It makes every subcommand addition a potentially breaking change and the tab completion will not be great. nix dev <TAB> will only show relevant commands.

confusion of nix dev and nix develop.

Good thing the CLI is experimental. We can still deprecate develop and replace it by the more powerful nix dev shell.

[thufschmitt]

Since it’s not mentioned in the issue: The most straightforward way of doing that right now is to just put them in the devShell. This definitely asn’t always ideal since it means that they have to be realised eagerly (very annoying if your script is something with a huge closure that only one person will run every once in a while), but that already covers 90% of the use-cases imho.

[roberth]

just put them in the devShell.

Strictness isn't the only problem. These scripts commands cannot be made aware of the project root. In non-flake Nix, you could use ${toString ./.} to anchor scripts' working directories, but since flakes those paths are in the store.
Robust scripts make onboarding much easier and workdir anchoring is essential for that, imo.

It is straightforward though 🤷

I'll add this to the alternatives section.

[thufschmitt]

These scripts commands cannot be made aware of the project root. In non-flake Nix, you could use ${toString ./.} to anchor scripts' working directories, but since flakes those paths are in the store.
Robust scripts make onboarding much easier and workdir anchoring is essential for that, imo.

That’s a fair point :) Maybe we could make nix develop export FLAKE_ROOT as you suggest for the dev tools? Or have a nix flake get-root command to show it (similar to git rev-parse --show-toplevel, but slightly less obscure 😛 )

[roberth]

Maybe we could make nix develop export FLAKE_ROOT as you suggest for the dev tools?

This will be redundant when nix develop is powered by nix dev shell, which is the right hierarchy of generality / search-path for develop (devTools -> devShell -> packages).
You could implement it in the "legacy" nix develop first of course.

have a nix flake get-root command

This will be useful to someone, but it doesn't replace FLAKE_ROOT, because that environment variable guarantees that it points to the worktree of the flake where the dev command is defined.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/flakes-as-a-unified-format-for-profiles/29476/15

[roberth]

Maybe we could make nix develop export FLAKE_ROOT as you suggest for the dev tools?

Another reason this is not a replacement, is that nix dev always runs what's in the flake at the time of invocation, whereas dev shells stick around, out of sync with the changed flake.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-to-build-packages-in-nix-develop-nix-shell-with-zsh/58232/1