Possible race condition in zip creation? #23

jroper · 2013-09-02T02:19:12Z

I got the following exception when doing a clean dist:

java.io.IOException: This archives contains unclosed entries.
    at org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream.finish(ZipArchiveOutputStream.java:413)
    at org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream.close(ZipArchiveOutputStream.java:806)
    at com.typesafe.sbt.packager.universal.ZipHelper$.withZipOutput(ZipHelper.scala:94)
    at com.typesafe.sbt.packager.universal.ZipHelper$.archive(ZipHelper.scala:74)
    at com.typesafe.sbt.packager.universal.ZipHelper$.zip(ZipHelper.scala:53)
    at com.typesafe.sbt.packager.universal.Archives$.makeZip(Archives.scala:15)
    at com.typesafe.sbt.packager.universal.UniversalPlugin$$anonfun$com$typesafe$sbt$packager$universal$UniversalPlugin$$makePackageSettingsForConfig$1.apply(UniversalPlugin.scala:31)
    at com.typesafe.sbt.packager.universal.UniversalPlugin$$anonfun$com$typesafe$sbt$packager$universal$UniversalPlugin$$makePackageSettingsForConfig$1.apply(UniversalPlugin.scala:31)
    at scala.Function3$$anonfun$tupled$1.apply(Function3.scala:35)
    at scala.Function3$$anonfun$tupled$1.apply(Function3.scala:34)
    at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
    at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:42)
    at sbt.std.Transform$$anon$4.work(System.scala:64)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
    at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:18)
    at sbt.Execute.work(Execute.scala:244)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
    at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:160)
    at sbt.CompletionService$$anon$2.call(CompletionService.scala:30)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)
[error] (universal:packageBin) java.io.IOException: This archives contains unclosed entries.

I couldn't reproduce it, so I don't know what to think, maybe a race condition?

The text was updated successfully, but these errors were encountered:

jsuereth · 2013-09-03T12:02:41Z

Yeah. If something was touching files in the directory where we were zipping, that might happen....

Any ideas on what was going on the same time this task was running?

jroper · 2013-09-03T14:10:04Z

As far as I'm aware, nothing.

gmjabs · 2013-11-06T22:21:24Z

Maybe I'm doing something wrong, but I'm reliably having this problem when trying to use the universal packager in a subproject. Minimal repro has the following directory structure,

./foo
./foo/conf
./foo/conf/foo.conf
./project
./project/Build.scala
./project/plugins.sbt

with this Build.scala file:

import sbt._
import Keys._
import com.typesafe.sbt.SbtNativePackager
import com.typesafe.sbt.SbtNativePackager._

object Foo extends Build {

  val buildSettings = Defaults.defaultSettings ++ Seq (
    organization := "foo",
    scalaVersion := "2.10.2"
  )

  lazy val foo = Project(
    base = file("foo"),
    id = "foo",
    settings = buildSettings ++ 
      SbtNativePackager.packagerSettings ++
      Seq( mappings in Universal := Seq(file("conf/foo.conf") -> "conf/foo.conf")
    )
  )
}

running universal:packageBin in the subproject fails:

[eus-loaner-22] [02:17 PM] [sb] > sbt
[info] Loading global plugins from /Users/gregj/.sbt/0.13/plugins
[info] Loading project definition from /Users/gregj/scratch/sb/project
[info] Compiling 1 Scala source to /Users/gregj/scratch/sb/project/target/scala-2.10/sbt-0.13/classes...
[info] Set current project to sb (in build file:/Users/gregj/scratch/sb/)
> project foo
[info] Set current project to foo (in build file:/Users/gregj/scratch/sb/)
> universal:packageBin
[trace] Stack trace suppressed: run 'last foo/universal:packageBin' for the full output.
[error] (foo/universal:packageBin) java.io.IOException: This archives contains unclosed entries.
[error] Total time: 0 s, completed Nov 6, 2013 2:17:59 PM
> last foo/universal:packageBin
java.io.IOException: This archives contains unclosed entries.
        at org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream.finish(ZipArchiveOutputStream.java:413)
        at org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream.close(ZipArchiveOutputStream.java:806)
        at com.typesafe.sbt.packager.universal.ZipHelper$.withZipOutput(ZipHelper.scala:107)
        at com.typesafe.sbt.packager.universal.ZipHelper$.archive(ZipHelper.scala:87)
        at com.typesafe.sbt.packager.universal.ZipHelper$.zip(ZipHelper.scala:53)
        at com.typesafe.sbt.packager.universal.Archives$.makeZip(Archives.scala:15)
        at com.typesafe.sbt.packager.universal.UniversalPlugin$$anonfun$com$typesafe$sbt$packager$universal$UniversalPlugin$$makePackageSettingsForConfig$1.apply(UniversalPlugin.scala:31)
        at com.typesafe.sbt.packager.universal.UniversalPlugin$$anonfun$com$typesafe$sbt$packager$universal$UniversalPlugin$$makePackageSettingsForConfig$1.apply(UniversalPlugin.scala:31)
        at scala.Function3$$anonfun$tupled$1.apply(Function3.scala:35)
        at scala.Function3$$anonfun$tupled$1.apply(Function3.scala:34)
        at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
        at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:42)
        at sbt.std.Transform$$anon$4.work(System.scala:64)
        at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
        at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
        at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:18)
        at sbt.Execute.work(Execute.scala:244)
        at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
        at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
        at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:160)
        at sbt.CompletionService$$anon$2.call(CompletionService.scala:30)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
[error] (foo/universal:packageBin) java.io.IOException: This archives contains unclosed entries.

jsuereth · 2013-11-06T22:23:53Z

What OS are you using? I'll have to try it on something non-ubuntu....

gmjabs · 2013-11-06T22:25:15Z

I'm on OSX 10.9
java version "1.7.0_45"
Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)

dbarashev · 2013-11-14T15:51:57Z

FWIW, I suffered from the same problem with "This archives contains unclosed entries." when running play dist. It turned out that it happened because of broken symlink in my dist directory.

gmjabs · 2013-11-18T19:38:56Z

I eventually figured this out, that the mappings aren't being resolved against the base directory of the subproject, rather against the base directory of the containing project, so where

Seq( mappings in Universal := Seq(file("conf/foo.conf") -> "conf/foo.conf")

was generating this error,

Seq( mappings in Universal := Seq(baseDirectory.value / "conf/foo.conf" -> "conf/foo.conf")

works. Seems in general that the packager errors out this way if the mapped input file doesn't exist. Which is logical, but the failure isn't informative of that.

jsuereth · 2013-11-18T19:46:39Z

Wow, great find! Ill see if I can improve the erroring there.

…exist. From sbt#23

muuki88 · 2014-01-28T23:00:43Z

Closing this one as resolved with #83

…exist. From sbt/sbt-native-packager#23

gmjabs pushed a commit to gmjabs/sbt-native-packager that referenced this issue Nov 21, 2013

Error out with a useful message if a mapped file in unversal doesn't …

916422c

…exist. From sbt#23

gmjabs mentioned this issue Nov 21, 2013

Informative failure for missing files mapped in universal #83

Merged

muuki88 closed this as completed Jan 28, 2014

littleRoundaer pushed a commit to littleRoundaer/sbt-native-packager that referenced this issue Jul 19, 2022

Error out with a useful message if a mapped file in unversal doesn't …

70384bd

…exist. From sbt/sbt-native-packager#23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Possible race condition in zip creation? #23

Possible race condition in zip creation? #23

jroper commented Sep 2, 2013

jsuereth commented Sep 3, 2013

jroper commented Sep 3, 2013

gmjabs commented Nov 6, 2013

jsuereth commented Nov 6, 2013

gmjabs commented Nov 6, 2013

dbarashev commented Nov 14, 2013

gmjabs commented Nov 18, 2013

jsuereth commented Nov 18, 2013

muuki88 commented Jan 28, 2014

Possible race condition in zip creation? #23

Possible race condition in zip creation? #23

Comments

jroper commented Sep 2, 2013

jsuereth commented Sep 3, 2013

jroper commented Sep 3, 2013

gmjabs commented Nov 6, 2013

jsuereth commented Nov 6, 2013

gmjabs commented Nov 6, 2013

dbarashev commented Nov 14, 2013

gmjabs commented Nov 18, 2013

jsuereth commented Nov 18, 2013

muuki88 commented Jan 28, 2014