Improved maven indexer failure modes in low space situations #5655
Conversation
- indexing will disable itself if the index failed to extract due to no space left on the device (temp or cache) and notify the user - take cause and suppressed exceptions into account while scanning for "no space" messages - added a fallback to clean up remaining files on exception - simplified the code a bit
mbien
added
Maven
mbien
changed the title
| Exceptions.printStackTrace(ex); | ||
| } | ||
| LOGGER.log(Level.INFO, "Downloaded maven index file has size {0} (zipped). The usable space in {1} is {2}.", new Object[]{downloaded, tmpFolder, usableSpace}); | ||
| Path tmpFolder = Paths.get(System.getProperty("java.io.tmpdir")); |
There was a problem hiding this comment.
At a later time we might allow people to choose a different path for the temp files. If here the tmpFolder is initialized from tmpStorage, this would be easier to update and kept consistent. The variable might not yet initalized though or intialized to null.
There was a problem hiding this comment.
yes, this is right now only half of the implementation. Since we don't control the location of the gz itself yet. This is for the larger part: the extracted temp index, before it is moved to cache - so it does help a bit.
if we find a way to get this in: #4999 we then have almost full control over it. The last remaining part would be to fix a bug in maven-indexer which currently hardcodes the extraction of silo files to tmp - but this does also require 4999, since we use a EOL indexer lib now without a way to get fixes in.
Once everything is in place we could make it configurable in the UI. As workaround: users can probably start nb by overriding the tmp folder:
-J-Djava.io.tmpdir=/path/to/tmpdir
I am also in contact with maven devs via slack since I have some ideas how to potentially significantly shrink index size. (wouldn't it be nice to drop all artifacts which are older than x years and make this configurable in the UI? Some artifacts also copy their whole doc into the description field which we could probably drop too. However, this is all blocked by JDK 8, I don't want to work too much ahead when a simple veto can put that time into waste)
|
thanks for the review -> merging (and I agree we should make the path configurable as soon it is possible) |
(note: the numbers on the screenshot make no sense since I tested while using different paths to ram disks to simulate low storage, in reality one of the two would be very low)
regarding cancelling: It can happen that while the download or extraction is in progress other threads queue their queries and block on the mutex. When a user cancels the process it will simply restart until the "queue" is empty - this isn't solved by this PR.
https://issues.apache.org/jira/browse/NETBEANS-5489
https://issues.apache.org/jira/browse/NETBEANS-5239