-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Dependabot Maven not finding latest version of a dependency #6637
Comments
Hi! I don't know much about Java versioning, but after looking a bit into it, it seems Maven agrees with Dependabot?
Could you open an issue with Maven to clarify if this is bug or expected behavior? I think https://issues.apache.org/jira/projects/MNG/issues/MNG-5987?filter=allopenissues may be the place? Regardless of that, I guess we should add logic to Dependabot to consider the Summoning @soulus13 in case he can bring some light here 🙏 (if you find my random pings annoying please let me know, you've been just too helpful for us, so I think it's worth asking, but I don't want to bother you!) |
@aloubyansky ^^^ |
Thank you @deivid-rodriguez for the info.
I don't find it annoying at all. I'm going to do the same with some of our folks here! |
Your example above is a little wrong though (you used Nonetheless the results are the same:
|
Oh, thanks for correcting it. I tried a bunch of combinations and copied a wrong test case. Anyways, I could not make logic from the results. I did read the docs at https://maven.apache.org/ref/4.0.0-alpha-4/maven-artifact/apidocs/org/apache/maven/artifact/versioning/ComparableVersion.html, and found that we do implement the same qualifier priority in Dependabot. |
@deivid-rodriguez thanks for the analysis, we should have verified it before making noise about it. The confusing part is that an
However, the presence of the About taking into account
we need to make sure that if suddenly we release 2.7.7, it doesn't appear as the latest recommended release overall. |
Its really just adding a $ java -jar ~/.m2/repository/org/apache/maven/maven-artifact/3.8.4/maven-artifact-3.8.4.jar 2.13.5.SP1-x 2.13.5.Final-x
Display parameters as parsed by Maven (in canonical form and as a list of tokens) and comparison result:
1. 2.13.5.SP1-x -> 2.13.5.sp-1-x; tokens: [2, 13, 5, sp, [1, [x]]]
2.13.5.SP1-x < 2.13.5.Final-x
2. 2.13.5.Final-x -> 2.13.5-x; tokens: [2, 13, 5, [x]] Maven doesn't actually create a token for java -jar ~/.m2/repository/org/apache/maven/maven-artifact/3.8.4/maven-artifact-3.8.4.jar 2.13.5.SP 2.13.5.Final
Display parameters as parsed by Maven (in canonical form and as a list of tokens) and comparison result:
1. 2.13.5.SP -> 2.13.5.sp; tokens: [2, 13, 5, sp]
2.13.5.SP > 2.13.5.Final
2. 2.13.5.Final -> 2.13.5; tokens: [2, 13, 5] |
No worries about noise, it's a good learning exercise for us and maybe a way to get in touch with the upstream maven project! Regarding the priority here, I observed the same thing. And regarding considering |
@deivid-rodriguez, surely not bothering me 😃 And version parsing logic in maven when not staying withing the accepted versioning x.y[.z][-A] really changes. For this case a similar example is even listed in the maven wiki https://maven.apache.org/pom.html#Version_Order_Specification: |
Speaking of the author preferences when it comes to the latest recommendations, in Quarkus we have a service that provides this information per "stream", which is typically tied to For Quarkus we'd cover BOM versions and Maven plugins primarily. However, we have more sophisticated version update support than that. We can analyze which Quarkus extensions are found in the project and suggest the latest recommended combination of those for a given project. Could that be something dependabot would provide some kind of plugin for? |
Just my opinion here @aloubyansky, but I would think that dependabot would want to remain neutral to any potential Java-based frameworks and leave it to the package managers to do their work (i.e. Maven & Gradle)? |
Basically, it'd not just about latest version updates but taking into account compatibility between the upgraded components. |
Sure, just in case it could be interesting to provide version update suggestions based on compatibility info. |
I wonder if the same problem exists if using Gradle. I could probably do a little POC and see. |
Correct, despite the fact that today we re-implement Maven / Gradle parsing via Ruby/Regex (but poorly). We very much want to move away from that model toward Dependabot being the glue code and leave version resolving to the native package manager tools. Ideally Dependabot finds the manifests, passes them to the native package manager, then takes the output and turns it into a useful PR. However, that will take a bit of time, which is why issues like #1164 are still open. Practically speaking, that means that if today Dependabot's brittle Ruby/regexes are buggy compared to Maven/Gradle behavior, then we will fix those bugs. But if you're looking for us to extend those to support parsing additional fields, then we're unlikely to do that as we'd rather put the effort into shifting over to the native tooling. Similarly, if you want non-standard behavior or extensions to the native package manager behavior, then I suggest working with the upstream package manager as it'd be pointless to do that here if we plan to rip that all out eventually when we switch to the native tooling. |
It's exactly about delegating version updates to tools that can suggest not individual artifact updates but a set that is known to be related but only to those specialized tools. For example today I have a PoC that reads a multi module project, identifies Quarkus bits in it and suggests which properties, dependencies, BOM imports and plugins should be updated, removed or added pointing to pom.xml files where those changes should be applied. This could be integrated into a maven plugin or, i guess, a GitHub bot, which is why I wanted to share the idea. |
The discussion has wandered a little from the original bug report, so I'm unclear: I realize there's an implicit feature request for making some bits of this pluggable, but as I mentioned above, the best place is the native package manager for this, and then when we shift over to that we can discuss ways to make it possible to also pull in plugins for native tooling. Although TBH even once it's pluggable it'd probably still require running Dependabot standalone yourself, as we're unlikely to pull in custom plugins in the standard install we would run within the Dependabot service on GitHub. |
I think the original issue, although unresolved, is not dependabot's to resolve, based on how maven sees things. You can go ahead @jeffwidman and close it out. If something comes to light later on I can always re-open it. I really appreciate the promptness of everyone's responses! |
I agree. Thanks! |
I think, based on a bit more testing with Maven, we should probably look it This is what I tried (probably there's an easier way to check this but I wasn't sure how to ask
So it seems authors can override order if they want to and Doing this ourselves too has the added benefit that we don't need to do any sorting at all. |
I was curious about how Gradle works, so I created https://github.com/edeandrea/quarkus-gradle-dependabot-test and enabled dependabot. I use the same version ( First thing I noticed @aloubyansky is that it does not work based on the way we structure our projects. It does not look in
It seems that once I move things directly into So the documentation there should also include This original problem does seem to exist with Gradle as well. See https://github.com/edeandrea/quarkus-gradle-dependabot-test/network/updates/601855201
My guess is its for the same reason? |
@deivid-rodriguez We as authors don't really control this metadata apart from release scheduling.
Notice, there is no @edeandrea yes, i'm pretty sure the reason is the same. |
So that means we can't rely on |
Right. Dependabot shouldn't change anything in that regard though. |
I see, thanks for clarifying that. I found the docs here: https://maven.apache.org/ref/3.9.0/maven-repository-metadata/repository-metadata.html#versioning Indeed, it seems just chronological order. So, my conclusions so far, Dependabot is getting this right and maven is getting this wrong?? |
I wouldn't say that, otherwise shouldn't I see a PR in my repo updating from |
See dependabot/dependabot-core#6637 as to why dependabot didn't do this on its own.
Mmmm our conclusion was that in this case |
That's what maven thinks, but in reality SP1 is an uogrqde |
Oh, sure, but Dependabot needs rules, it can't read library author's minds. If library |
Right. I think what we deduced was that dependabot's rules are consistent with maven's. |
If we really think However, I still think there's a bug in Maven anyways because it's actually choosing So, there's two options:
|
Where are you getting that? Maven is choosing $ java -jar ~/.m2/repository/org/apache/maven/maven-artifact/3.8.4/maven-artifact-3.8.4.jar 2.13.5.SP1-redhat-00002 2.13.5.Final-redhat-00002
Display parameters as parsed by Maven (in canonical form and as a list of tokens) and comparison result:
1. 2.13.5.SP1-redhat-00002 -> 2.13.5.sp-1-redhat-2; tokens: [2, 13, 5, sp, [1, [redhat, [2]]]]
2.13.5.SP1-redhat-00002 < 2.13.5.Final-redhat-00002
2. 2.13.5.Final-redhat-00002 -> 2.13.5-redhat-2; tokens: [2, 13, 5, [redhat, [2]]] |
I tried to contact the Maven project about this and will post any updates I get, and link to a public discussion upstream about this if/when it's created. I'll close this ticket for now, since this does not seem a Dependabot issue (at the moment). We can reopen if it turns out we need to fix anything on our side. |
Maybe I did it wrong, but in #6637 (comment) I tried an idea to figure out what |
Few responses to this thread:
|
Also, it seems it is (old) code that is confused, maven-resolver (former aether) does it right:
This means that version sorting (old) code in maven (as tested in comment #6637 (comment)) and maven-resolver disagree as well, and this may explain some subtle differences. Created issue https://issues.apache.org/jira/browse/MNG-7690 |
Thanks a lot @cstamas! |
Thanks @cstamas, I subscribed to that ticket :) |
Hello @deivid-rodriguez did any progress get made with the Maven team on this? Seems to still be an issue with dependabot unfortunately (even though its not dependabot's actual problem). |
Sorry I never got back to you, I no longer maintain dependabot. Looking at https://issues.apache.org/jira/browse/MNG-7690, I see no further activity, so I guess the answer is no... |
Is there an existing issue for this?
Package ecosystem
maven
Package manager version
No response
Language version
No response
Manifest location and content before the Dependabot update
https://github.com/quarkusio/quarkus-super-heroes/blob/rhbq-2.13/event-statistics/pom.xml
https://github.com/quarkusio/quarkus-super-heroes/blob/rhbq-2.13/rest-fights/pom.xml
https://github.com/quarkusio/quarkus-super-heroes/blob/rhbq-2.13/rest-heroes/pom.xml
https://github.com/quarkusio/quarkus-super-heroes/blob/rhbq-2.13/rest-villains/pom.xml
dependabot.yml content
https://github.com/quarkusio/quarkus-super-heroes/blob/main/.github/dependabot.yml
Updated dependency
com.redhat.quarkus.platform:quarkus-bom
Current version in the
pom.xml
files is2.13.5.Final-redhat-00002
What you expected to see, versus what you actually saw
When I look at the log (https://github.com/quarkusio/quarkus-super-heroes/network/updates/601547269) it says
According to https://maven.repository.redhat.com/ga/com/redhat/quarkus/platform/quarkus-bom/maven-metadata.xml there is a newer version available:
2.13.5.SP1-redhat-00002
.Dependabot doesn't seem to be picking up this version.
Native package manager behavior
No response
Images of the diff or a link to the PR, issue, or logs
Latest dependabot log (as of February 10, 2023): https://github.com/quarkusio/quarkus-super-heroes/network/updates/601547269
Smallest manifest that reproduces the issue
The text was updated successfully, but these errors were encountered: