stack --work-dir fails with directories above the current directory

[acfoltzer]

Steps to reproduce:

1. In any stack project with a parent directory, run stack build --work-dir=../.stack-work

Expected:

The build proceeds using the .stack-work directory of the parent directory

Actual:

Stack errors out with

InvalidRelDir "../../.stack-work"

---

Seems like this ought to work, right?

[mgsloan]

It's this way intentionally, though certainly the error reporting could be better. See the discussion here - #1178 (comment) . The summary is that there is some cabal bug on windows that requires the dist dir to be within the package dir. This is why we have multiple stack-work directories, instead of just one.

I would be in favor of putting everything in a .stack-work dir, if not on windows / not using affected Cabal versions. Then, we could also allow absolute directories for --work-dir, though I'm not sure how we would ensure that different projects sharing a work-dir wouldn't stomp on eachother.

However, allowing absolute paths or ../.stack-work would invite sharing work-dirs between projects, which isn't guaranteed to be graceful. Namely, dirtiness checking uses modification times, and so wouldn't work correctly when multiple projects compile to the same spot.

[mgsloan]

I'm leaving this issue open to track making the error message better. If we want to unify .stack-work into a single dir for the project, allow absolute paths, or allow paths with ../, then that should go in a separate issue.

[magthe]

@mgsloan I think this issue actually is caused by a built-in limitation in path (see the discussion at commercialhaskell/path#8). That is, even if cabal wouldn't have the bug on Windows, path simply doesn't allow a relative path like ../.stack-work!

[magthe]

Also, see this discussion on the mailing list: https://groups.google.com/forum/#!topic/haskell-stack/YvO45vDw62o

[mgsloan]

@magthe Right, but our usage of a function in path which has that limitation is intentional.

If .stack-work directories with .. were allowed currently, then we'd get different packages attempting to write to the same dist directory. We'd need to change the paths for dist directories in order for this to work. I'm not sure how useful --stack-work is when it doesn't allow absolute dirs / relative dirs with ...

[magthe]

For me it's pretty much useless :(

[mgsloan]

Yep. I would accept a patch that makes it useful by doing the following:

• Unifies .stack-work into one dir on non-windows
• Allows absolute .stack-work paths / paths with .. on non-windows

The main problem with this is that if you use one of these options in a script or something, suddenly it won't work on windows. We tend to try to avoid things like that which break cross-platform compatibility. Then again, this seems like a Cabal bug (not sure what the details are). Perhaps by exposing it to users, it'll get fixed sooner rather than later.

[mfine]

👍 for accepting an absolute directory that's not a child of the project directory.

I'm trying to replicate cabal sandbox functionality used here, where .cabal-sandbox is placed under an absolute path (/app/.cabal-sandbox):

$ /tmp/wut > cabal sandbox init
Writing a default package environment file to
/private/tmp/wut/cabal.sandbox.config
Creating a new sandbox at /private/tmp/wut/.cabal-sandbox

$ ~/repos/hello > cabal sandbox init --sandbox /tmp/wut/.cabal-sandbox
Writing a default package environment file to
/Users/mark/repos/hello/cabal.sandbox.config
Using an existing sandbox located at /private/tmp/wut/.cabal-sandbox

$ ~/repos/hello > cabal install
Warning: The package list for 'hackage.haskell.org' is 272.9 days old.
Run 'cabal update' to get the latest list of available packages.
Resolving dependencies...
Notice: installing into a sandbox located at /private/tmp/wut/.cabal-sandbox
...

The larger context is that I have to build a project in an temporary directory that is removed after building and install the project under /app. When an executable depends on contents under .stack-work, e.g., data-files, they can't be found. A relocatable work-dir would be great (along with the --prefix feature in #848). Otherwise I think I'm left with having to copy the directory around (from the absolute location to the temporary directory location). Maybe there's another way around?

[mfine]

Here's the workaround I've instituted for now - 1) copy over source, 2) build and install source, 3) copy back .stack-work directory.

I've been looking through the sources to relax the Rel dir constraint - it seems pretty enmeshed currently and would also involve moving to a single work-dir, both things I'd be 👍 about. I don't know the project that well, but will play around with trying to move to an Abs dir while preserving a Rel dir on windows.

[mfine]

More as an exercise, went through and threaded the work-dir as Either (Path Abs Dir) (Path Rel Dir) in master...mfine:mfine-abs-rel-work-dir. Building and tests passing. There's definitely better ways to handle this, and there's likely cases not being handled correctly. With changing around the default relative path to absolute path in mfine/stack@mfine-abs-rel-work-dir...mfine-abs-rel-work-dir-other, was able to use an absolute path outside of the directory to build and run a project:

$ repos/hellos > ~/.local/bin/stack build --work-dir /Users/mark/tmp/stack-work
Populated index cache.
Did not find .cabal file for stm-containers-0.2.11 with Git SHA of 9410bc0f41fe0b458f7c350ce4a5a17cdbe3f18e
Did not find .cabal file for kansas-comet-0.4 with Git SHA of be4690cfa4a780f71d05182fe5c69bb4ae635bf7
Did not find .cabal file for hledger-lib-0.27 with Git SHA of 9ad40bc0d38fb9fa2a96e2f5211fef8e2b405222
Did not find .cabal file for blank-canvas-0.6 with Git SHA of 1ab8537279ad2dd4cf25f76527633d47e5b16a94
hellos-0.0.0: configure
Configuring hellos-0.0.0...
hellos-0.0.0: build
Preprocessing executable 'hellos' for hellos-0.0.0...
[1 of 1] Compiling Main             ( Main.hs, /Users/mark/tmp/stack-work/dist/x86_64-osx/Cabal-1.22.5.0/build/hellos/hellos-tmp/Main.o )
Linking /Users/mark/tmp/stack-work/dist/x86_64-osx/Cabal-1.22.5.0/build/hellos/hellos ...
hellos-0.0.0: copy/register
Installing executable(s) in
/Users/mark/tmp/stack-work/install/x86_64-osx/lts-5.18/7.10.3/bin

$ repos/hellos > ~/.local/bin/stack exec hellos --work-dir /Users/mark/tmp/stack-work
Setting phasers to stun... (port 3000) (ctrl-c to quit)
"CurrentDirectory /Users/mark/tmp/stack-work/install/x86_64-osx/lts-5.18/7.10.3/share/x86_64-osx-ghc-7.10.3/hello-0.0.0"
"DataDir /Users/mark/repos/hellos"

Would love this kind of flexibility for configuring the work dir!

[mgsloan]

@mfine Cool! I think the path library should have a better datatype for "may be either abs or relative". Then we can do a similar change more elegantly. Thoughts, @chrisdone / @mrkkrp ?

[mfine]

@mgsloan definitely agree! Something that could express either and support </> and toFilePath instead of the silly Either and <\\> and <//> and toFilePath' stuff I used.

[phadej]

@mfine <\\> doesn't make much sense.

What if the project has multiple packages, but I want to have single .stack-work (outside the source tree)!

[mfine]

@phadej not sure in what context you think <\\> doesn't make much sense - potentially you're referring to the <\\> throwing away the absolute path if it's Either value is a Left absolute path already? I'm not familiar enough with the stack internals and how that could impact various areas where <\\> was used. But yeah, a better resolving scheme and support for joining the various directories in stack would be great - I found the directory machinations in the current implementation very, very hard to follow.

[mgsloan]

Due to an issue with Cabal on windows - #1178 (comment) - unfortunately this isn't going to be resolved.

We could consider unifying the .stack-work dirs on other platforms, though.