Generate version information when publishing artifacts #1188

marcospereira · 2020-08-20T17:54:55Z

Currently, the only way to get version information from alpakka-kafka artifacts is by reading its MANIFEST.MF file, which may be fragile when sbt-assembly or other fatjar packagers are used: the manifest can be lost, overridden by another one, or multiple artifacts may be using Implementation-Version key.

To overcome this problem, Akka, Play, and Lagom all package a version class containing such information. Therefore, no matter if the application using them is building a fatjar or not, the version information is consistently accessible.

This pull request brings Akka solution and creates both akka.kafka.Version class as well as a version.conf with a key that is less prone to conflict with another artifact.

ennru

LGTM, with one question below.

ennru · 2020-08-21T10:53:59Z

project/VersionGenerator.scala

+
+  val settings: Seq[Setting[_]] = inConfig(Compile)(
+    Seq(
+      resourceGenerators += generateVersion(resourceManaged, _ / "version.conf", """|akka.kafka.version = "%s"


Wouldn't the version.conf file collide with others in a fat-jar scenario?
I guess those could be merged.

If multiple artifacts contain version.conf files, users need to configure a merge strategy when using sbt-assembly, or they will get an error* like this:

[error] 1 error was encountered during merge [error] stack trace is suppressed; run last assembly for the full output [error] (assembly) deduplicate: different file contents found in the following: [error] /Users/marcospereira/.ivy2/local/com.typesafe.akka/akka-stream-kafka_2.13/2.0.4+12-31cf63dd/jars/akka-stream-kafka_2.13.jar:version.conf [error] /Users/marcospereira/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/typesafe/akka/akka-actor_2.13/2.5.30/akka-actor_2.13-2.5.30.jar:version.conf

But since we are using a distinct key for alpakka-kafka version, it is safe to concat the files, as you guessed.

* I'm not sure why sbt-assembly is not handling all filenames ending in .conf as configuration files. Maybe it is worth to send a PR there too?

Yes, merging *.conf files seems reasonable to me.

An alternative for now would be to create a file named akka-kafa-version.conf.

Hum, "for now" implying that it is going to change in the future, right? If so, I would not bother to have another name, and then move to version.conf later. Users should be able to configure what to do pretty easily, so I don't see why to have extra steps here. But also, I don't have a strong opinion on that. My only concern is to handle multiple version files in Cinnamon in the future.

Finally, I created a new issue in sbt-assembly: sbt/sbt-assembly#406. If they decide it make sense to support all .conf files out-of-the-box, I will submit a PR there, and users would only need to update sbt-assembly then.

WDYT?

seglo

LGTM

gurgec · 2020-10-15T08:40:25Z

Hi All. I am not sure I wrote to the right place but this commit is not working in shaded jar. We have an application that we use akka and akka-streams together and when we try to run our app We got following exception since we have only 'akka.kafka.version' in our version.conf instead of 'akka.version'.
Maybe we can keep both. I resolved this issue by changing version to 2.0.4. Thank you.

No configuration setting found for key 'akka.version'
        at com.typesafe.config.impl.SimpleConfig.findKeyOrNull(SimpleConfig.java:156)
        at com.typesafe.config.impl.SimpleConfig.findOrNull(SimpleConfig.java:174)
        at com.typesafe.config.impl.SimpleConfig.findOrNull(SimpleConfig.java:180)
        at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:188)
        at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:193)
        at com.typesafe.config.impl.SimpleConfig.getString(SimpleConfig.java:250)
        at akka.actor.ActorSystem$Settings.<init>(ActorSystem.scala:396)
        at akka.actor.ActorSystemImpl.<init>(ActorSystem.scala:813)
        at akka.actor.ActorSystem$.apply(ActorSystem.scala:272)
        at akka.actor.ActorSystem$.apply(ActorSystem.scala:316)
        at akka.actor.ActorSystem$.apply(ActorSystem.scala:290)
        at akka.actor.ActorSystem$.create(ActorSystem.scala:213)
        at akka.actor.ActorSystem.create(ActorSystem.scala)
        at com.location.App.main(App.java:25)

seglo · 2020-10-15T14:09:49Z

It's not clear from the stack trace that this is related to Alpakka Kafka itself since the issue is resolving akka.version. Are you creating a fat jar? Some effort may be required to merge configuration correctly. There's some discussion in the akka documentation: https://doc.akka.io/docs/akka/current/additional/packaging.html .

gurgec · 2020-10-15T16:42:19Z

It's not clear from the stack trace that this is related to Alpakka Kafka itself since the issue is resolving akka.version. Are you creating a fat jar? Some effort may be required to merge configuration correctly. There's some discussion in the akka documentation: https://doc.akka.io/docs/akka/current/additional/packaging.html .

Yes. I am creating fat jar and I am following the instructions in packaging doc. I opened the jar and saw we do not have 'akka.version' in version.conf. When I downgraded to 2.0.4 I have 'akka.version' and everything works fine.

marcospereira · 2020-10-15T18:07:22Z

How are you packaging the fat jar? Is it using sbt-assembly? If that is the case, you have to configure it to use MergeStrategy.concat for version.conf files:

https://github.com/sbt/sbt-assembly#merge-strategy

gurgec · 2020-10-16T06:22:42Z

How are you packaging the fat jar? Is it using sbt-assembly? If that is the case, you have to configure it to use MergeStrategy.concat for version.conf files:

https://github.com/sbt/sbt-assembly#merge-strategy

I am using gradle. I will have a look at MergeStrategy.concat. Thank you.

ebrurak · 2020-10-23T09:22:46Z

Hi all,

I encountered same problem of 'akka.version' as @gurgec with alpakka-kafka 2.0.5. Everything works well when I downgraded to 2.0.4.
I create fat jar with maven as mentionned in the akka documentation.

Resolved by merging version.conf as well as reference.conf in our case.

https://doc.akka.io/docs/akka/current/additional/packaging.html

ennru · 2020-10-27T07:35:54Z

As described above: You need to configure your fat jar tooling to merge the version.conf files from the multiple jars.

Generate version information when publishing

31cf63d

ennru approved these changes Aug 21, 2020

View reviewed changes

seglo approved these changes Aug 24, 2020

View reviewed changes

ennru added this to the 2.0.5 milestone Aug 24, 2020

ennru merged commit c878803 into akka:master Aug 24, 2020

marcospereira deleted the build/version-generator branch August 24, 2020 22:11

ennru mentioned this pull request Oct 1, 2020

Generate version.conf file in runtime jar akka/akka-grpc#1132

Merged

dependabot bot mentioned this pull request Mar 15, 2021

Bump akka-stream-kafka_2.13 from 1.0.5 to 2.0.7 confluentinc/kafka-tutorials#759

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generate version information when publishing artifacts #1188

Generate version information when publishing artifacts #1188

marcospereira commented Aug 20, 2020

ennru left a comment

ennru Aug 21, 2020

marcospereira Aug 21, 2020 •

edited

Loading

ennru Aug 24, 2020

marcospereira Aug 24, 2020

seglo left a comment

gurgec commented Oct 15, 2020 •

edited

Loading

seglo commented Oct 15, 2020 •

edited

Loading

gurgec commented Oct 15, 2020 •

edited

Loading

marcospereira commented Oct 15, 2020

gurgec commented Oct 16, 2020

ebrurak commented Oct 23, 2020 •

edited

Loading

ennru commented Oct 27, 2020

Generate version information when publishing artifacts #1188

Generate version information when publishing artifacts #1188

Conversation

marcospereira commented Aug 20, 2020

ennru left a comment

Choose a reason for hiding this comment

ennru Aug 21, 2020

Choose a reason for hiding this comment

marcospereira Aug 21, 2020 • edited Loading

Choose a reason for hiding this comment

ennru Aug 24, 2020

Choose a reason for hiding this comment

marcospereira Aug 24, 2020

Choose a reason for hiding this comment

seglo left a comment

Choose a reason for hiding this comment

gurgec commented Oct 15, 2020 • edited Loading

seglo commented Oct 15, 2020 • edited Loading

gurgec commented Oct 15, 2020 • edited Loading

marcospereira commented Oct 15, 2020

gurgec commented Oct 16, 2020

ebrurak commented Oct 23, 2020 • edited Loading

ennru commented Oct 27, 2020

marcospereira Aug 21, 2020 •

edited

Loading

gurgec commented Oct 15, 2020 •

edited

Loading

seglo commented Oct 15, 2020 •

edited

Loading

gurgec commented Oct 15, 2020 •

edited

Loading

ebrurak commented Oct 23, 2020 •

edited

Loading