Memory problem in Jabref #4323

bernhard-kleine · 2018-09-07T09:02:08Z

I have tested the latest master version from http://builds.jabref.org/master/ and the problem persists

JabRef 5.0-dev--snapshot--2018-09-06--docbookfive--ee248ae75
Windows 7 6.1 amd64
Java 1.8.0_181

Steps to reproduce:

Open two libraries and the select one entry and open it with crtl-e in the entry editor
Put something (like a doi in my case) into the clipboard
Switch to the other library
Try to open the entry editor by double clicking one entry.

Jabref chokes and the program can only be shut down from the task manager. It only increases available RAM up to 4.4 Gb (and even further, but I did not wait for it) without doing anything. A log file or error report is not possible.

xiaodongcentury · 2018-09-09T04:01:08Z

The same phenomenon appears in JabRef 5.0-dev--snapshot--2018-09-08 (Debian GNU/Linux).
terminal reads,

i search the error "java.lang.OutOfMemoryError: GC overhead limit exceeded", over the stackoverflow.com (https://stackoverflow.com/questions/1393486/error-java-lang-outofmemoryerror-gc-overhead-limit-exceeded), Mnementh gives a solution: A simple solution could be simply to give some more Heap-memory to the Java-Engine (-Xmx) but this only helps, if the application needs exactly as much memory, as the heap-limit before was set. while Adelin (https://stackoverflow.com/questions/11091516/exception-in-thread-main-java-lang-outofmemoryerror-gc-overhead-limit-exceede) just increase the memory that the virtual machine needs to compile the project.
I dont understand the app development, so i dont know what to do, may the information above be useful to resolve the problem.

Siedlerchr · 2018-09-09T12:48:46Z

Hi, Thanks for your reports, could you please try to tell when exactly the problem occurs?

If you want to debug this a bit further, you could try to use the YourKit java Profiler.
Launch JabRef, start Yourkit and attach it to the application. When the error occurs, take a Memory Snapshot and upload it somewhere for us, so we can analyze it in detail.
You can send the link to developers@jabref.org

https://www.yourkit.com/docs/java/help/memory_snapshots.jsp
https://www.yourkit.com/java/profiler/download/ (15 day free evaluation key)

xiaodongcentury · 2018-09-10T10:28:02Z

Thanks your advice, i try the latest version (JabRef 5.0-dev--snapshot--2018-09-09--master--4c7194a39), and the error is gone. Now everything looks fine, though the RSS is higher, or maybe just my opinon (BTW, i have 2000+ entries in my JabRef.).
Here is the task manage screen shot:

Oh, NO!!!
After i open a library, which contains 1000+ entries, then double click one entry, sadly, the JabRef was freeded.
the task manager screen shot:

and the terminal reads:

I can do nothing but kill JabRef.
Maybe the memory JabRef allocated is fixed, or limited, what about dynamic?

Siedlerchr · 2018-09-10T11:54:50Z

Hi, I found an error when opening a new library, which could be the problem here as well. I fixed it as part of another PR. https://builds.jabref.org/storeColumnSortOrder/
It seems like that on the Threading Error JabRef consumes all memory until it runs out of it. could be unix related, because on Windows the memory stays constant even when JabRef freezes

xiaodongcentury · 2018-09-10T13:26:50Z

Thanks! Now i'll try the stable version, JabRef 4.3.1, it seems like fine.

though the RSS is more higher, maybe.

xiaodongcentury · 2018-09-11T00:58:18Z

I'm sorry, man, the https://builds.jabref.org/storeColumnSortOrder/ version doesn't work. I cann't make the JabRef version screenshot( through "Option" > "About") in that the JabRef is freezed. Here's the terminal reads:

I can do nothing but kill JabRef.

Siedlerchr · 2018-09-13T11:56:16Z

@xiaodongcentury Could you please test again with the latest master version?
https://builds.jabref.org/master/
And then please try to call with the following options:
java -XX:+UseG1GC -XX:+UseStringDeduplication -XX:StringTableSize=1000003 -jar JabRef.jar

xiaodongcentury · 2018-09-13T13:09:12Z

no problem. I'll try it.
After i download the latest master version (2018-09-13), then java -jar xxxx.jar. At first, it seems like normal. Later, my debian OS freeze, ALL apps freeze, chromium, evince document viewer, task manager, terminal, emacs freeze. So, there is NO screenshot, and i manually shutdown debian OS.

xiaodongcentury · 2018-09-13T14:07:57Z

Once i reboot my OS, java -jar xxxx.jar (2018-09-13). Task manager reads: CPU, 100%,

So i kill Jabref.
Then try your given options, and the JabRef (2018-09-13) freezes, the same phenomenon as above, though the CPU is about 25%.

And a new "dbind-warning" appears, whether use your given option.
.
May the information above be useful to resolve the problem.
Then i google "dbind-warning", on a french Ubuntu website ( i use chromium 'translate to english'), maybe, https://forum.ubuntu-fi.org/index.php?topic=53591.0. According to nm's explanation: "accessibility bus are related to GNOME's accessibility support and communication between applications and the desktop system". The exact case I REALLY don't know, because i am new to Linux. And i "sudo apt install at-spi2-core", the terminal reads:

So i've no idea what can i do.

Siedlerchr · 2018-09-13T14:39:06Z

Hi, I recently had the same problem with it and found that this switch helps:
java -Djavafx.embed.singleThread=true -jar JabRef.jar

xiaodongcentury · 2018-09-13T15:14:15Z

OK, Thanks, I'll try your suggestion.

bernhard-kleine · 2018-09-14T13:15:40Z

I have tested the master snapshot of yesterday. (2018-09-13) And I encounter memory problems by a very small excerpt of my database which is included. I suspected an error with unescaped % signs in the abstract, but when I escaped the offending % signs manually using Notepad++, the strange behavior remained. I click on all entry in the file and nothing strange happened. Then I clicked on the entry NS86
and suddenly the RAM for Jabref goes from about 350.000 K to 1.500.000 K and I had to shut down Jabref via the taskmanager. I tested for longer since I have 16 Gb and seems to stop at 4.470.692 K of the RAM. But JabRef does not become aktiv again.

I ran the snapshot with the switch proposed, but it does not help.
testjabref.bib.txt

bernhard-kleine · 2018-09-14T13:52:08Z

after I let run java for a prolonged time I can no longer restart jabref 5dev since an java error occurs:

java.lang.ClassNotFoundException: org.jabref.JabRefMain
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at com.exe4j.runtime.LauncherEngine.launch(LauncherEngine.java:81)
at com.exe4j.runtime.WinLauncher.main(WinLauncher.java:94)
at com.install4j.runtime.launcher.WinLauncher.main(WinLauncher.java:25).

Please advise what to do.

tobiasdiez · 2018-09-14T14:01:27Z

Thanks @bernhard-kleine. This narrows down the problem. In fact,

author          = {Nambu, J R and Scheller, R H},

leads to the freeze while

author          = {Nambu, J. R. and Scheller, R. H.},

works fine (I just added dots after the first name initials). Moreover, removing the last dot in the working example also results in the freeze. Hence, I guess our integrity checks are the reason for the problem.

@bernhard-kleine: system restart?

bernhard-kleine · 2018-09-14T14:19:22Z

These entries were all imported using the PMID Medline search in Jabref. Now that the missing dots are there, I can not open the file which would make Jabref fail immediately. There are some 2660 entries. I cannot solve the problem by manually adding dots.

tobiasdiez · 2018-09-16T21:43:04Z

I debugged it and traced the problem to impl.org.controlsfx.ImplUtils#injectAsRootPane. For whatever reason, I cannot however step trough this method since the source code shown in the IDE is different from the bytecode. @Siedlerchr can you try to debug it and see where it freezes exactly?

Siedlerchr · 2018-09-17T13:59:13Z

@tobiasdiez I could trace it down to this line:

    if (originalParent != null) {
            getChildren(injectedParent, useReflection).add(0, originalParent);

which then seems to create the hanging:
When I then pause the thread I get the following stacktrace:
However, I can then debug through it fine.
Interestingly it has no effect if I have the group pane open or not.


Thread [JavaFX Application Thread] (Suspended)	
	CssStyleHelper.createStyleHelper(Node) line: 116	
	ValueTableCellFactory$1(Node).reapplyCss() line: 8983	
	ValueTableCellFactory$1(Node).impl_reapplyCSS() line: 8946	
	ValueTableCellFactory$1(Node).invalidatedScenes(Scene, SubScene) line: 854	
	ValueTableCellFactory$1(Node).setScenes(Scene, SubScene) line: 919	
	TableRow<T>(Parent).scenesChanged(Scene, SubScene, Scene, SubScene) line: 667	
	TableRow<T>(Node).invalidatedScenes(Scene, SubScene) line: 853	
	TableRow<T>(Node).setScenes(Scene, SubScene) line: 919	
	Group(Parent).scenesChanged(Scene, SubScene, Scene, SubScene) line: 667	
	Group(Node).invalidatedScenes(Scene, SubScene) line: 853	
	Group(Node).setScenes(Scene, SubScene) line: 919	
	VirtualFlow$ClippedContainer(Parent).scenesChanged(Scene, SubScene, Scene, SubScene) line: 667	
	VirtualFlow$ClippedContainer(Node).invalidatedScenes(Scene, SubScene) line: 853	
	VirtualFlow$ClippedContainer(Node).setScenes(Scene, SubScene) line: 919	
	VirtualFlow<T>(Parent).scenesChanged(Scene, SubScene, Scene, SubScene) line: 667	
	VirtualFlow<T>(Node).invalidatedScenes(Scene, SubScene) line: 853	
	VirtualFlow<T>(Node).setScenes(Scene, SubScene) line: 919	
	MainTable(Parent).scenesChanged(Scene, SubScene, Scene, SubScene) line: 667	
	MainTable(Node).invalidatedScenes(Scene, SubScene) line: 853	
	MainTable(Node).setScenes(Scene, SubScene) line: 919	
	ScrollPaneSkin$4(Parent).scenesChanged(Scene, SubScene, Scene, SubScene) line: 667	
	ScrollPaneSkin$4(Node).invalidatedScenes(Scene, SubScene) line: 853	
	ScrollPaneSkin$4(Node).setScenes(Scene, SubScene) line: 919	
	ScrollPaneSkin$3(Parent).scenesChanged(Scene, SubScene, Scene, SubScene) line: 667	
	ScrollPaneSkin$3(Node).invalidatedScenes(Scene, SubScene) line: 853	
	ScrollPaneSkin$3(Node).setScenes(Scene, SubScene) line: 919	
	ScrollPane(Parent).scenesChanged(Scene, SubScene, Scene, SubScene) line: 667	
	ScrollPane(Node).invalidatedScenes(Scene, SubScene) line: 853	
	ScrollPane(Node).setScenes(Scene, SubScene) line: 919	
	AnchorPane(Parent).scenesChanged(Scene, SubScene, Scene, SubScene) line: 667	
	AnchorPane(Node).invalidatedScenes(Scene, SubScene) line: 853	
	AnchorPane(Node).setScenes(Scene, SubScene) line: 919	
	SplitPaneSkin$Content(Parent).scenesChanged(Scene, SubScene, Scene, SubScene) line: 667	
	SplitPaneSkin$Content(Node).invalidatedScenes(Scene, SubScene) line: 853	
	SplitPaneSkin$Content(Node).setScenes(Scene, SubScene) line: 919	
	SplitPane(Parent).scenesChanged(Scene, SubScene, Scene, SubScene) line: 667	
	SplitPane(Node).invalidatedScenes(Scene, SubScene) line: 853	
	SplitPane(Node).setScenes(Scene, SubScene) line: 919	
	BasePanel(Parent).scenesChanged(Scene, SubScene, Scene, SubScene) line: 667	
	BasePanel(Node).invalidatedScenes(Scene, SubScene) line: 853	
	BasePanel(Node).setScenes(Scene, SubScene) line: 919	
	TabPaneSkin$TabContentRegion(Parent).scenesChanged(Scene, SubScene, Scene, SubScene) line: 667	
	TabPaneSkin$TabContentRegion(Node).invalidatedScenes(Scene, SubScene) line: 853	
	TabPaneSkin$TabContentRegion(Node).setScenes(Scene, SubScene) line: 919	
	DndTabPaneFactory$2(Parent).scenesChanged(Scene, SubScene, Scene, SubScene) line: 667	
	DndTabPaneFactory$2(Node).invalidatedScenes(Scene, SubScene) line: 853	
	DndTabPaneFactory$2(Node).setScenes(Scene, SubScene) line: 919	
	SplitPaneSkin$Content(Parent).scenesChanged(Scene, SubScene, Scene, SubScene) line: 667	
	SplitPaneSkin$Content(Node).invalidatedScenes(Scene, SubScene) line: 853	
	SplitPaneSkin$Content(Node).setScenes(Scene, SubScene) line: 919	
	SplitPane(Parent).scenesChanged(Scene, SubScene, Scene, SubScene) line: 667	
	SplitPane(Node).invalidatedScenes(Scene, SubScene) line: 853	
	SplitPane(Node).setScenes(Scene, SubScene) line: 919	
	JabRefFrame(Parent).scenesChanged(Scene, SubScene, Scene, SubScene) line: 667	
	JabRefFrame(Node).invalidatedScenes(Scene, SubScene) line: 853	
	JabRefFrame(Node).setScenes(Scene, SubScene) line: 919	
	Parent$1.onChanged(Change<Node>) line: 269	
	Parent$1(TrackableObservableList<T>).lambda$new$29(ListChangeListener$Change) line: 45	
	428168219.onChanged(ListChangeListener$Change) line: not available	
	ListListenerHelper$Generic<E>.fireValueChangedEvent(Change<? extends E>) line: 329	
	ListListenerHelper<E>.fireValueChangedEvent(ListListenerHelper<E>, Change<? extends E>) line: 73	
	Parent$1(ObservableListBase<E>).fireChange(Change<? extends E>) line: 233	
	ListChangeBuilder<E>.commit() line: 482	
	ListChangeBuilder<E>.endChange() line: 541	
	Parent$1(ObservableListBase<E>).endChange() line: 205 [local variables unavailable]	
	Parent$1(ModifiableObservableListBase<E>).add(int, E) line: 155	
	Parent$2(VetoableListDecorator<E>).add(int, E) line: 320	
	ImplUtils.injectAsRootPane(Scene, Parent, boolean) line: 71	
	Decorator.lambda$getDecorationPane$1(Consumer, Node, Scene) line: 199	
	227337281.accept(Object) line: not available	
	Decorator$1.invalidated(Observable) line: 222	
	ExpressionHelper$Generic<T>.fireValueChangedEvent() line: 349	
	ExpressionHelper<T>.fireValueChangedEvent(ExpressionHelper<T>) line: 81	
	ReadOnlyObjectWrapper$ReadOnlyPropertyImpl(ReadOnlyObjectPropertyBase<T>).fireValueChangedEvent() line: 74	
	Node$ReadOnlyObjectWrapperManualFire<T>(ReadOnlyObjectWrapper<T>).fireValueChangedEvent() line: 102	
	Node$ReadOnlyObjectWrapperManualFire<T>.fireSuperValueChangedEvent() line: 831	
	EditorTextField(Node).invalidatedScenes(Scene, SubScene) line: 881	
	EditorTextField(Node).setScenes(Scene, SubScene) line: 919	
	JournalEditor(Parent).scenesChanged(Scene, SubScene, Scene, SubScene) line: 667	
	JournalEditor(Node).invalidatedScenes(Scene, SubScene) line: 853	
	JournalEditor(Node).setScenes(Scene, SubScene) line: 919	
	GridPane(Parent).scenesChanged(Scene, SubScene, Scene, SubScene) line: 667	
	GridPane(Node).invalidatedScenes(Scene, SubScene) line: 853	
	GridPane(Node).setScenes(Scene, SubScene) line: 919	
	ScrollPaneSkin$4(Parent).scenesChanged(Scene, SubScene, Scene, SubScene) line: 667	
	ScrollPaneSkin$4(Node).invalidatedScenes(Scene, SubScene) line: 853	
	ScrollPaneSkin$4(Node).setScenes(Scene, SubScene) line: 919	
	ScrollPaneSkin$3(Parent).scenesChanged(Scene, SubScene, Scene, SubScene) line: 667	
	ScrollPaneSkin$3(Node).invalidatedScenes(Scene, SubScene) line: 853	
	ScrollPaneSkin$3(Node).setScenes(Scene, SubScene) line: 919	
	Parent$1.onChanged(Change<Node>) line: 269	
	Parent$1(TrackableObservableList<T>).lambda$new$29(ListChangeListener$Change) line: 45	
	428168219.onChanged(ListChangeListener$Change) line: not available	
	ListListenerHelper$Generic<E>.fireValueChangedEvent(Change<? extends E>) line: 329	
	ListListenerHelper<E>.fireValueChangedEvent(ListListenerHelper<E>, Change<? extends E>) line: 73	
	Parent$1(ObservableListBase<E>).fireChange(Change<? extends E>) line: 233	
	ListChangeBuilder<E>.commit() line: 482	
	ListChangeBuilder<E>.endChange() line: 541	
	Parent$1(ObservableListBase<E>).endChange() line: 205	
	Parent$1(ModifiableObservableListBase<E>).addAll(Collection<? extends E>) line: 102	
	Parent$2(VetoableListDecorator<E>).addAll(Collection<? extends E>) line: 237	
	Parent$2(VetoableListDecorator<E>).addAll(E...) line: 103	
	ScrollPaneSkin.initialize() line: 312	
	ScrollPaneSkin.<init>(ScrollPane) line: 125	
	ScrollPane.createDefaultSkin() line: 635	
	ScrollPane(Control).impl_processCSS(WritableValue<Boolean>) line: 872	
	TabPaneSkin$TabContentRegion(Parent).impl_processCSS(WritableValue<Boolean>) line: 1280	
	TabPane(Parent).impl_processCSS(WritableValue<Boolean>) line: 1280	
	TabPane(Control).impl_processCSS(WritableValue<Boolean>) line: 875	
	EntryEditor(Parent).impl_processCSS(WritableValue<Boolean>) line: 1280	
	SplitPaneSkin$Content(Parent).impl_processCSS(WritableValue<Boolean>) line: 1280	
	SplitPaneSkin$Content(Node).processCSS() line: 9056	
	SplitPane(Node).processCSS() line: 9049	
	BasePanel(Node).processCSS() line: 9049	
	TabPaneSkin$TabContentRegion(Node).processCSS() line: 9049	
	DndTabPaneFactory$2(Node).processCSS() line: 9049	
	SplitPaneSkin$Content(Node).processCSS() line: 9049	
	SplitPane(Node).processCSS() line: 9049	
	JabRefFrame(Node).processCSS() line: 9049	
	Scene.doCSSPass() line: 545	
	Scene.access$3600(Scene) line: 159	
	Scene$ScenePulseListener.pulse() line: 2392	
	Toolkit.lambda$runPulse$29(TKPulseListener) line: 398	
	1557081070.run() line: not available	
	AccessController.doPrivileged(PrivilegedAction<T>, AccessControlContext) line: not available [native method]	
	QuantumToolkit(Toolkit).runPulse(TKPulseListener, AccessControlContext) line: 397	
	QuantumToolkit(Toolkit).firePulse() line: 424	
	QuantumToolkit.pulse(boolean) line: 518	
	QuantumToolkit.pulse() line: 498	
	QuantumToolkit.pulseFromQueue() line: 491	
	QuantumToolkit.lambda$runToolkit$403() line: 319	
	1338856507.run() line: not available	
	InvokeLaterDispatcher$Future.run() line: 95	
	WinApplication._runLoop(Runnable) line: not available [native method]	
	WinApplication.lambda$null$147(int, Runnable) line: 177	
	1014328909.run() line: not available	
	Thread.run() line: 748

Fixes #4323 and probably fixes #4294 as well. The reason for the freezes were the pull-request https://bitbucket.org/controlsfx/controlsfx/pull-requests/710. These changes mess up the layout for some reason. The fix is to inject a DecorationPane ourselves directly from the beginning. Although this fix started as a workaround for this bug in controlsfx, I think, we should keep it permanently for performance reasons.

Siedlerchr · 2018-09-18T08:45:08Z

I can confirm that the latest development version with the fix from @tobiasdiez now works fine! the memory footprint is much lower now and no more freezes in entry editor.

bernhard-kleine · 2018-09-18T09:02:48Z

I confirm that the import reported in #4345 is now flawlessly.
Thanks a lot!

tobiasdiez added the status: waiting-for-feedback The submitter or other users need to provide more information about the issue label Sep 13, 2018

tobiasdiez removed the status: waiting-for-feedback The submitter or other users need to provide more information about the issue label Sep 14, 2018

bernhard-kleine mentioned this issue Sep 15, 2018

Pubmed import should have quality check multiple times. #4345

Closed

tobiasdiez mentioned this issue Sep 18, 2018

Fix freezes in entry editor #4351

Merged

Siedlerchr closed this as completed in #4351 Sep 18, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Memory problem in Jabref #4323

Memory problem in Jabref #4323

bernhard-kleine commented Sep 7, 2018 •

edited

Loading

xiaodongcentury commented Sep 9, 2018

Siedlerchr commented Sep 9, 2018

xiaodongcentury commented Sep 10, 2018

Siedlerchr commented Sep 10, 2018

xiaodongcentury commented Sep 10, 2018

xiaodongcentury commented Sep 11, 2018

Siedlerchr commented Sep 13, 2018

xiaodongcentury commented Sep 13, 2018

xiaodongcentury commented Sep 13, 2018

Siedlerchr commented Sep 13, 2018 •

edited

Loading

xiaodongcentury commented Sep 13, 2018

bernhard-kleine commented Sep 14, 2018 •

edited

Loading

bernhard-kleine commented Sep 14, 2018 •

edited

Loading

tobiasdiez commented Sep 14, 2018 •

edited

Loading

bernhard-kleine commented Sep 14, 2018 •

edited

Loading

tobiasdiez commented Sep 16, 2018

Siedlerchr commented Sep 17, 2018

Siedlerchr commented Sep 18, 2018

bernhard-kleine commented Sep 18, 2018

Memory problem in Jabref #4323

Memory problem in Jabref #4323

Comments

bernhard-kleine commented Sep 7, 2018 • edited Loading

xiaodongcentury commented Sep 9, 2018

Siedlerchr commented Sep 9, 2018

xiaodongcentury commented Sep 10, 2018

Siedlerchr commented Sep 10, 2018

xiaodongcentury commented Sep 10, 2018

xiaodongcentury commented Sep 11, 2018

Siedlerchr commented Sep 13, 2018

xiaodongcentury commented Sep 13, 2018

xiaodongcentury commented Sep 13, 2018

Siedlerchr commented Sep 13, 2018 • edited Loading

xiaodongcentury commented Sep 13, 2018

bernhard-kleine commented Sep 14, 2018 • edited Loading

bernhard-kleine commented Sep 14, 2018 • edited Loading

tobiasdiez commented Sep 14, 2018 • edited Loading

bernhard-kleine commented Sep 14, 2018 • edited Loading

tobiasdiez commented Sep 16, 2018

Siedlerchr commented Sep 17, 2018

Siedlerchr commented Sep 18, 2018

bernhard-kleine commented Sep 18, 2018

bernhard-kleine commented Sep 7, 2018 •

edited

Loading

Siedlerchr commented Sep 13, 2018 •

edited

Loading

bernhard-kleine commented Sep 14, 2018 •

edited

Loading

bernhard-kleine commented Sep 14, 2018 •

edited

Loading

tobiasdiez commented Sep 14, 2018 •

edited

Loading

bernhard-kleine commented Sep 14, 2018 •

edited

Loading