Partitions with dev files stored outside of the repository?

[pedorich-n]

Hi!
I love the idea of partitions and dev-only inputs! I've been using dev subflakes for a while now.

Most of my projects use the same dev tools (treefmt, git-pre-commit, deadnix, etc.), so I've developed a single flake that I reuse across repositories as a git submodule.
I tried to migrate my setup to partitions, but it failed with

error: getting status of '/nix/store/q01xsd47sfigj2qapf5y9sqzvwj6z6dz-source/dev': No such file or directory

because git submodules aren't copied to the nix store by nix commands by default, and there's no way to enable it within the flake.nix.

The next thing I tried is to specify my dev flake as an input to the parent flake (yes, it kinda defeats the purpose, but with flake = false; it's just one extra input instead of multiple). Now if I set the extraInputsFlake to builtins.unsafeDiscardStringContext inputs.nix-dev-flake.outPath; and try to use partitions it fails with:

error: cannot call 'getFlake' on unlocked flake reference '/nix/store/qcy8ish4gglcvhnyd8zm0my125z114bj-source', at «none»:0 (use --impure to override)

This seems to be a problem with Nix flakes' behavior. I am using Nix version 2.18.5, in case that’s relevant.

Is there a way around this? Is there a way to use partitions with dev files not being physically present in the same repository and without adding ?submodules=1 to every command in the case of git submodules or --impure with flake input?
I realize that this is an extremely specific corner case and I am probably the only one experiencing it, but maybe I'm missing something.

Thanks!

[roberth]

and there's no way to enable it within the flake.nix.

It seems that this would solve the problem nice and simple.

Now if I set the extraInputsFlake to builtins.unsafeDiscardStringContext inputs.nix-dev-flake.outPath; and try to use partitions it fails

I think we could use the same workaround that I added for path values, but for store paths.

---

Another possible solution is to add your nix-dev-input flake as an input to a ./dev flake instead of a submodule.

./dev/flake.nix

{
  inputs.nix-dev-flake.url = ...; # ie flake = true;
  outputs = { ... }: { };
}

This adds an indirection between nix-dev-input's inputs so that they're not directly available in your dev partition. This might be a good thing, so that changes to the structure of its inputs don't have a direct effect on all your other flakes. For instance a nix flake update could start overriding an input of the main flake if it has the same name.
I think you could still forward inputs like this:

partitions.dev.extraInputsFlake = ./dev;
partitions.dev.extraInputs = { inherit (config.partitions.dev.extraInputs.nix-dev-flake) treefmt-nix; };

[roberth]

Now if I set the extraInputsFlake to builtins.unsafeDiscardStringContext inputs.nix-dev-flake.outPath; and try to use partitions it fails

I think we could use the same workaround that I added for path values, but for store paths.

Haven't tested this yet, but it seems like it could work:

• partitions.<name>.extraInputsFlake: allow string #246

I think the ./dev helper flake is still preferable if you have users who want faster locking.

[pedorich-n]

It seems that this would solve the problem nice and simple.

Yes, I agree. If submodules behaved like normal folders I would be able to get rid of a lot of hacks.

Another possible solution is to add your nix-dev-input flake as an input to a ./dev flake instead of a submodule.

I didn't think of that! It might work. Let me try that.
Thanks!

The "allow string" PR looks like a hack IMO, so maybe don't merge it yet. Unless you think it's a good addition.

[pedorich-n]

So I tested it and I don't think it will work the way I want it to work. My idea is to avoid not only inputs but also to provide some basic config for the tools. What you suggested solves the inputs problem, but the module import issue is still there.

Here's what I have:
./dev/flake.nix

{
  inputs = {
    nix-dev-flake = {
      url = "github:pedorich-n/nix-dev-flake/flake-partitions";
      flake = true;
    };
  };

  outputs = _: { };
}

./partitions.nix

  partitions.dev = {
    extraInputsFlake = ./dev;
    extraInputs = { inherit (config.partitions.dev.extraInputs.nix-dev-flake.inputs) treefmt-nix pre-commit-hooks; };
    module = {
      imports = [
        "${config.partitions.dev.extraInputs.nix-dev-flake}/flake-module.nix"
      ];
    };

The flake-module.nix is still in the nix store which leads to

access to absolute path '/nix/store/fnh35w1nk2jxqsv9j9g6cg44f8q5i2v2-source' is forbidden in pure eval mode (use '--impure' to override)

(/nix/store/fnh35w1nk2jxqsv9j9g6cg44f8q5i2v2-source is nixpkgs btw)

---

With the code from your PR I still get the same error, but If I move the nix-dev-flake back to the "main" flake with flake = false it works now.

Like this:
partitions.nix

  partitions.dev = {
    extraInputsFlake = builtins.unsafeDiscardStringContext inputs.nix-dev-flake.outPath; 
    module = {
      imports = [
        "${inputs.nix-dev-flake}/flake-module.nix"
      ];
    };
  };

---

Unless there's a way to turn "${config.partitions.dev.extraInputs.nix-dev-flake}/flake-module.nix" into a path I don't think what I want is possible without the "strings" hack 😞

[roberth]

It worked when I tested this solution on other flakes.
Which Nix version did you use?
I think fetchTree, which loads the sources for flakes, should always allow its result to be used in the rest of the evaluation, so this could be a bug.

Is the code that creates the partition in a public repo?

---

Fwiw, you could probably refactor that first version of partitions.nix to the following when you add flakeModule to the outputs of nix-dev-flake. I don't think it makes a difference though.

    module = { inputs, ... }: {
      imports = [
        inputs.nix-dev-flake.flakeModule
      ];
    };

[pedorich-n]

I tried to create a small reproducible repo and it turned out working 😮
I then wiped the changes in my original repo and started from scratch and it worked there too. I must've had traces of my previous tests somewhere and they interfered.

FWIW I still pushed the code: https://github.com/pedorich-n/flake-parts-issue-244

So a dev flake with the single input of my nix-dev-flake works.
Thanks!