Merge NixOS into Nixpkgs

[edolstra]

I'd like to merge the NixOS tree into the Nixpkgs repository, as has been discussed a few times in the past. The reasons for merging them are:

• Changes in Nixpkgs often require a change in NixOS and vice versa. With separate repositories, this means you need to do two commits, which is not atomic and creates the possibility of trees being out of sync with each other.
• Branches in Nixpkgs often require corresponding branches in NixOS. For instance, a Nixpkgs branch like multiple-outputs requires a branch in NixOS because it changes the structure of some packages.
• Issue tracking: it's often unclear whether an issue should be reported in the NixOS or Nixpkgs issue tracker, and often the fix is in the other repository.
• The NixOS version is currently a 2-tuple of Git hashes. Merging NixOS and Nixpkgs allows the NixOS version to be identified using a single Git hash.

There are also downsides to merging:

• It makes the Nixpkgs tree a bit bigger, which may be slightly annoying for people who only use Nixpkgs and not NixOS.
• It makes it harder to build a NixOS tree against a different Nixpkgs tree. But given the tight coupling between these trees, that doesn't work very well in practice anyway. OTOH, it becomes much easier to use multiple Nixpkgs trees in a NixOps deployment (since there are no more <nixpkgs> references).

I did a test merge, available here: https://github.com/edolstra/nixpkgs/tree/merge-test. It imports nixos/master as a subdirectory "nixos" in the root directory of Nixpkgs (keeping all history). While I was at it, I also renamed pkgs/lib to lib because it never really made sense to have it in pkgs.

Note that GitHub unfortunately doesn't show history across renames, but "git log --follow" does (and GitHub's "blame" also works fine).

[viric]

It looks to me like a very good idea. I'm for it!

On Fri, Oct 04, 2013 at 03:59:58AM -0700, Eelco Dolstra wrote:

I'd like to merge the NixOS tree into the Nixpkgs repository, as has been discussed a few times in the past. The reasons for merging them are:

• Changes in Nixpkgs often require a change in NixOS and vice versa. With separate repositories, this means you need to do two commits, which is not atomic and creates the possibility of trees being out of sync with each other.
• Branches in Nixpkgs often require corresponding branches in NixOS. For instance, a Nixpkgs branch like multiple-outputs requires a branch in NixOS because it changes the structure of some packages.
• Issue tracking: it's often unclear whether an issue should be reported in the NixOS or Nixpkgs issue tracker, and often the fix is in the other repository.
• The NixOS version is currently a 2-tuple of Git hashes. Merging NixOS and Nixpkgs allows the NixOS version to be identified using a single Git hash.

There are also downsides to merging:

• It makes the Nixpkgs tree a bit bigger, which may be slightly annoying for people who only use Nixpkgs and not NixOS.
• It makes it harder to build a NixOS tree against a different Nixpkgs tree. But given the tight coupling between these trees, that doesn't work very well in practice anyway. OTOH, it becomes much easier to use multiple Nixpkgs trees in a NixOps deployment (since there are no more <nixpkgs> references).

I did a test merge, available here: https://github.com/edolstra/nixpkgs/tree/merge-test. It imports nixos/master as a subdirectory "nixos" in the root directory of Nixpkgs (keeping all history). While I was at it, I also renamed pkgs/lib to lib because it never really made sense to have it in pkgs.

Note that GitHub unfortunately doesn't show history across renames, but "git log --follow" does (and GitHub's "blame" also works fine).

---

Reply to this email directly or view it on GitHub:
#1041

[bluescreen303]

Sounds good to me.
Will this be combined with the upcoming stable release?

Eelco Dolstra notifications@github.com writes:

I'd like to merge the NixOS tree into the Nixpkgs repository, as has
been discussed a few times in the past. The reasons for merging them
are:

• Changes in Nixpkgs often require a change in NixOS and vice
  versa. With separate repositories, this means you need to do two
  commits, which is not atomic and creates the possibility of trees
  being out of sync with each other.
• Branches in Nixpkgs often require corresponding branches in
  NixOS. For instance, a Nixpkgs branch like multiple-outputs requires a
  branch in NixOS because it changes the structure of some packages.
• Issue tracking: it's often unclear whether an issue should be
  reported in the NixOS or Nixpkgs issue tracker, and often the fix is
  in the other repository.
• The NixOS version is currently a 2-tuple of Git hashes. Merging
  NixOS and Nixpkgs allows the NixOS version to be identified using a
  single Git hash.

There are also downsides to merging:

• It makes the Nixpkgs tree a bit bigger, which may be slightly
  annoying for people who only use Nixpkgs and not NixOS.
• It makes it harder to build a NixOS tree against a different Nixpkgs
  tree. But given the tight coupling between these trees, that doesn't
  work very well in practice anyway. OTOH, it becomes much easier to use
  multiple Nixpkgs trees in a NixOps deployment (since there are no more
  ` references).

I did a test merge, available here:
https://github.com/edolstra/nixpkgs/tree/merge-test. It imports
nixos/master as a subdirectory "nixos" in the root directory of
Nixpkgs (keeping all history). While I was at it, I also renamed
pkgs/lib to lib because it never really made sense to have it in pkgs.

Note that GitHub unfortunately doesn't show history across renames,
but "git log --follow" does (and GitHub's "blame" also works fine).

---

Reply to this email directly or view it on GitHub:
#1041

[edolstra]

@bluescreen303 Yeah, that's basically why I want to do this now. If we do it after the release, we'll have to keep two repositories alive for a while.

[vcunat]

For the downsides:

• The sizes of nixpkgs vs. nixos is on the order of 10:1 and I don't think it would decrease much in future, rather the other way.
• I think you can get the combination in your work tree in two commands. First git-checkout the whole tree for the "nixpkgs version", and then git-checkout the nixos subtree for the "nixos version".

[vcunat]

What I'm not clear about: can we somehow "merge" the github issues/PRs? There are quite many for both nixos and nixpkgs.

[edolstra]

There is a script to move issues to another GitHub project, but it doesn't keep comment authorship: https://github.com/collective/collective.developermanual/blob/master/gh-issues-import.py

[the-kenny]

I just wrote a mail to Github asking if they know a way to merge projects while keeping all issues with comments.

[edolstra]

Cool, thanks!

[bjornfor]

I use nixpkgs on Ubuntu on a couple of machines, but still +1 for merging. Looking forward to easily git bisect the whole of NixOS and more easily recreate exact builds (no more manual work to match nixos/nixpkgs versions).

[domenkozar]

+1

[MarcWeber]

I'm fine with this change, too.

There are more reasons to merge: Eg should "creating an iso or an usb image of nixos" be a nixpkgs or a nixos project?

The more I think about it the earlier this should be done.

[garbas]

awesome! 🍺

[offlinehacker]

+1

On Sun, Oct 6, 2013 at 6:06 PM, Rok Garbas notifications@github.com wrote:

awesome! [image: 🍺]

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/1041#issuecomment-25770756
.

[edolstra]

Okay, I'll do the merge now. Cross fingers...

I'll turn off commits to the NixOS repo first.

[offlinehacker]

Do we have to reopen pull requests now or will they be handled in any other way?

[MarcWeber]

Excerpts from Jaka Hudoklin's message of Do Okt 10 19:58:59 +0200 2013:

Do we have to reopen pull requests now or will they be handled in any other way?
The thread talked about a request sent to github about whether there is
a way to "merge" issues and the like. Don't know about the reply or
current state.

Closing and reopening is unlikely to be wrong.

Marc Weber

[domenkozar]

This can be closed?

[edolstra]

Yes, closing now...

[sergei-mironov]

Thanks, +1