Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is .store safe to share between different versions of cabal? #5073

Closed
mitchellwrosen opened this issue Jan 25, 2018 · 15 comments
Closed

Is .store safe to share between different versions of cabal? #5073

mitchellwrosen opened this issue Jan 25, 2018 · 15 comments

Comments

@mitchellwrosen
Copy link

mitchellwrosen commented Jan 25, 2018

Hi, is it safe if different versions of cabal-install share the same .cabal/store? Thanks.

@mitchellwrosen
Copy link
Author

Hm, I suspect not, here is a random failure I just saw after cabal new-build:

In order, the following will be built:
 - tasty-hspec-1.1.3.3 (lib) (first run)
creating /home/mitchell/tasty-hspec/dist-newstyle/build
creating /home/mitchell/tasty-hspec/dist-newstyle/tmp
creating
/home/mitchell/tasty-hspec/dist-newstyle/build/x86_64-linux/ghc-8.2.2/tasty-hspec-1.1.3.3
creating
/home/mitchell/tasty-hspec/dist-newstyle/build/x86_64-linux/ghc-8.2.2/tasty-hspec-1.1.3.3/cache
Using self-exec internal setup method with build-type Simple and args:
["act-as-setup","--build-type=Simple","--","build","--verbose=2","--builddir=/home/mitchell/tasty-hspec/dist-newstyle/build/x86_64-linux/ghc-8.2.2/tasty-hspec-1.1.3.3"]
/home/mitchell/.ghc-switch/cabal-2.0.1 act-as-setup --build-type=Simple --
build --verbose=2
--builddir=/home/mitchell/tasty-hspec/dist-newstyle/build/x86_64-linux/ghc-8.2.2/tasty-hspec-1.1.3.3
Component build order: library
/home/mitchell/.local/bin/ghc-pkg init /home/mitchell/tasty-hspec/dist-newstyle/build/x86_64-linux/ghc-8.2.2/tasty-hspec-1.1.3.3/package.conf.inplace
creating
/home/mitchell/tasty-hspec/dist-newstyle/build/x86_64-linux/ghc-8.2.2/tasty-hspec-1.1.3.3/build
creating
/home/mitchell/tasty-hspec/dist-newstyle/build/x86_64-linux/ghc-8.2.2/tasty-hspec-1.1.3.3/build/autogen
creating
/home/mitchell/tasty-hspec/dist-newstyle/build/x86_64-linux/ghc-8.2.2/tasty-hspec-1.1.3.3/build/autogen
Preprocessing library for tasty-hspec-1.1.3.3..
Building library for tasty-hspec-1.1.3.3..
creating
/home/mitchell/tasty-hspec/dist-newstyle/build/x86_64-linux/ghc-8.2.2/tasty-hspec-1.1.3.3/build
/home/mitchell/.local/bin/ghc --make -fbuilding-cabal-package -O -static -dynamic-too -dynosuf dyn_o -dynhisuf dyn_hi -outputdir /home/mitchell/tasty-hspec/dist-newstyle/build/x86_64-linux/ghc-8.2.2/tasty-hspec-1.1.3.3/build -odir /home/mitchell/tasty-hspec/dist-newstyle/build/x86_64-linux/ghc-8.2.2/tasty-hspec-1.1.3.3/build -hidir /home/mitchell/tasty-hspec/dist-newstyle/build/x86_64-linux/ghc-8.2.2/tasty-hspec-1.1.3.3/build -stubdir /home/mitchell/tasty-hspec/dist-newstyle/build/x86_64-linux/ghc-8.2.2/tasty-hspec-1.1.3.3/build -i -i/home/mitchell/tasty-hspec/dist-newstyle/build/x86_64-linux/ghc-8.2.2/tasty-hspec-1.1.3.3/build -isrc -i/home/mitchell/tasty-hspec/dist-newstyle/build/x86_64-linux/ghc-8.2.2/tasty-hspec-1.1.3.3/build/autogen -i/home/mitchell/tasty-hspec/dist-newstyle/build/x86_64-linux/ghc-8.2.2/tasty-hspec-1.1.3.3/build/global-autogen -I/home/mitchell/tasty-hspec/dist-newstyle/build/x86_64-linux/ghc-8.2.2/tasty-hspec-1.1.3.3/build/autogen -I/home/mitchell/tasty-hspec/dist-newstyle/build/x86_64-linux/ghc-8.2.2/tasty-hspec-1.1.3.3/build/global-autogen -I/home/mitchell/tasty-hspec/dist-newstyle/build/x86_64-linux/ghc-8.2.2/tasty-hspec-1.1.3.3/build -optP-include -optP/home/mitchell/tasty-hspec/dist-newstyle/build/x86_64-linux/ghc-8.2.2/tasty-hspec-1.1.3.3/build/autogen/cabal_macros.h -this-unit-id tasty-hspec-1.1.3.3-inplace -hide-all-packages -Wmissing-home-modules -no-user-package-db -package-db /home/mitchell/.cabal/store/ghc-8.2.2/package.db -package-db /home/mitchell/tasty-hspec/dist-newstyle/packagedb/ghc-8.2.2 -package-db /home/mitchell/tasty-hspec/dist-newstyle/build/x86_64-linux/ghc-8.2.2/tasty-hspec-1.1.3.3/package.conf.inplace -package-id base-4.10.1.0 -package-id hspec-2.4.7-577cbbaaeb3a476f5a178716e078191e8dbdc74a18af150ac0f48103e7f3d5fe -package-id hspec-core-2.4.7-e206a1b6325f69f099cbc37454a30789c364c12d0c4a917a5380910c6d889646 -package-id QuickCheck-2.11.3-a63c049c6b5aa686babd2377246f06526e4639a6bc5d7c1d61cb95e0a74d17ca -package-id tagged-0.8.5-7714bee11ffc9f255c48bb8cd047900367fe895efd281701a86dc93ff52d5028 -package-id tasty-1.0.0.1-994f672b13521675fb4690c0f54b54470e6a360ca3a1a086472eff3faa94a071 -package-id tasty-smallcheck-0.8.1-a8f2a6cc7449525eb3a375482d16cb1cbb386124afc57b50d0a0fc55e35d38c1 -package-id tasty-quickcheck-0.9.2-806e2d09ce2d078043bb3db7f973c169aa664a2c70a0ca1ca8afd5e0a66328d6 -XHaskell2010 Test.Tasty.Hspec -Wall
<command line>: cannot satisfy -package-id hspec-2.4.7-577cbbaaeb3a476f5a178716e078191e8dbdc74a18af150ac0f48103e7f3d5fe: 
    hspec-2.4.7-577cbbaaeb3a476f5a178716e078191e8dbdc74a18af150ac0f48103e7f3d5fe is unusable due to shadowed dependencies:
      HUnit-1.6.0.0-371f958c554e34d2c7a1324ec7571d490e6d0b8ab571c4d7ba80dc45ed932425 QuickCheck-2.11.3-a63c049c6b5aa686babd2377246f06526e4639a6bc5d7c1d61cb95e0a74d17ca base-4.10.1.0 call-stack-0.1.0-d8b7fe8f4751008f1d0cae48341a98728b5d4d8bf61896d7ee7e498d13088c38 hspec-core-2.4.7-e206a1b6325f69f099cbc37454a30789c364c12d0c4a917a5380910c6d889646 hspec-discover-2.4.7-86591673ee6eec3a0482723389b1fb58127e0ec94278f3af60a1dd7a228d2b14 hspec-expectations-0.8.2-3053b969a76574eb772d93424e659ec84efc968a3287c11266ae36cf27da390c transformers-0.5.2.0

@mitchellwrosen
Copy link
Author

I don't understand the failure at all, but here's all I did to cause it:

  • Build a package with ghc-8.0.2 and cabal-install-1.24.0.2
  • Build it again with ghc-8.2.2 and cabal-install-2.0.0.1

@mitchellwrosen
Copy link
Author

I can't seem to reproduce this failure so I'm not sure what was going on a few hours ago.

@phadej
Copy link
Collaborator

phadej commented Jan 26, 2018

new-build is unusable in 1.24, don't use new-build with cabal-install-1.24. It was meant as technology preview back then, and has no support.

I'm not sure about the issue itself, e.g. if cabal-install-2.0 and current master manage the store in the same way.

@hvr
Copy link
Member

hvr commented Jan 26, 2018

@mitchellwrosen going forward (i.e. past cabal 2.0), it should be safe (unless we end up requiring BC changes; at which point we'll hopefully notice and figure out a way to give good warnings/errors), but it can easily be that you end up with different Nix hashes for different cabal versions. Different GHC versions live in different folders anyway; in fact, you can safely run multiple cabal new-build process in parallel (in different source-trees) if they use different GHC versions; and they would operate in different .store subfolders. dist-newstyle however is a bit of a different story.

@mitchellwrosen
Copy link
Author

mitchellwrosen commented Jan 26, 2018

Okay, good to know.

dist-newstyle however is a bit of a different story.

Ah, interesting. I should get in the habit of using different --builddirs with different versions of cabal-install, then?

I've been using this (disclaimer: script is bad) to switch between global installations of ghc and cabal as sort of a poor man's stack (which uses the resolver to build with the appropriate version of ghc).

It's possible I should also build a small "cabal-aware" cabal wrapper that does things like:

  • Use namespaced dist-newstyle directories
  • Don't use new-build with cabal-install-1.24 ;)

@hvr
Copy link
Member

hvr commented Jan 26, 2018

@mitchellwrosen that's an interesting script; however, it seems to use symlinks rather than exploiting cabals built-in ability to switch between different compilers: the -w flag; E.g. if you use my PPA (which works better than older bindists on modern Ubuntu distributions), you can use cabal new-build -w ghc-7.6.3 to quickly use GHC 7.6.3 w/o having to use stateful symlinks. And if you want it stateful, you can simply write with-compiler: ghc-7.6.3 into your cabal.project file (which corresponds to Stack.yaml files in spirit... roughly... it's not a 100% equivalent). Moreover, if you want to push this idea of a stack emulation layer a bit further, I'd strongly suggest taking a look at https://github.com/erikd/jenga which may get you the rest of the way there.

@mitchellwrosen
Copy link
Author

mitchellwrosen commented Jan 26, 2018

Oops! I was not aware of the -w flag or with-compiler setting! Both of those are very helpful. However, I was under the impression that, for Setup.hs files to work reliably, you ought to use a version of cabal-install that was built against a "similar enough" (i.e. matching major- and minor-versions, but not necessarily patch) version of Cabal as ghc itself.

That is to say,

  • cabal-install-1.24.0.0 should be used with ghc-8.0.1
  • cabal-install-1.24.0.2 should be used with ghc-8.0.2
  • cabal-install-2.0.0.0 should be used with ghc-8.2.1
  • cabal-install-2.0.0.1 should be used with ghc-8.2.2

and so on. Am I wrong about this? Is it okay to build, say, a project that specifies an older version of ghc with a newer version of cabal-install? How about building with a newer version of ghc using an older version of cabal-install?

@hvr
Copy link
Member

hvr commented Jan 26, 2018

Am I wrong about this?

It was an issue in the past; but since at least cabal new-build this is "wrong". You are encouraged to use the most recent cabal-install available (cabal-install needs to be at least as recent as the GHC release you're using it with; but it's perfectly fine (sometimes even better) if cabal-install is more recent). For instance, http://matrix.hackage.haskell.org/ uses a cabal-install-2.1.x snapshot, with GHCs back till GHC 7.4 (it should also work with GHC 7.0).

@mitchellwrosen
Copy link
Author

Great, thanks for the info. I still have one outstanding question about dist-newstyle: is it okay to

> cabal new-build -w ghc-8.2.2
> cabal new-build -w ghc-8.0.2

or should I instead

> cabal new-build -w ghc-8.2.2 --builddir=dist-newstyle-8.2.2/
> cabal new-build -w ghc-8.0.2 --builddir=dist-newstyle-8.0.2/

@ezyang
Copy link
Contributor

ezyang commented Jan 26, 2018

Cabal caches the build project but not the install plan. So the cost of switching is we have to redepsolve and replan, but then the compile products will be cached.

@mitchellwrosen
Copy link
Author

I meant to clarify this comment above:

dist-newstyle however is a bit of a different story

with respect to the "safety" of sharing one dist-newstyle directory across builds with multiple versions of ghc. As a hypothetical end-user with absolutely no knowledge of what cabal is doing under the hood (which in my case is more or less accurate), is it okay or ill-advised? You mention there is a "cost" to switching directories, but what might the benefit be (if any)?

@ezyang
Copy link
Contributor

ezyang commented Jan 26, 2018

In an ideal world you could just switch GHC versions and not have to worry about it. This is something we should support. In practice it is supported reasonably well, at least when I've been using it. The benefit is you have to type less in the command line :)

@phadej
Copy link
Collaborator

phadej commented Jan 26, 2018 via email

@mitchellwrosen
Copy link
Author

Ok! thanks all! The fact that the newest cabal-install can safely build with old versions of ghc greatly simplifies things for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants