High CPU/memory consumption after a short while, zombie process remains after quitting with "Quit" - caused by author autocomplete preference

[ytzemih]

JabRef version

Latest development branch build (please note build date below)

Operating system

GNU / Linux

Details on version and operating system

JabRef 5.10--2023-05-25--e021686 Linux 5.19.0-42-generic amd64

Checked with the latest development build

• I made a backup of my libraries before testing the latest development version.
• I have tested the latest development version and the problem persists

Steps to reproduce the behaviour

Since my update from JR 5.9 to 5.10, I'm experiencing that my JR instance (usually with 3 to 4 bigger libraries open) starts to consume 2 of the 4 CPU cores (at 100%) and a lot of memory, after a short while (10-15 minutes). I got aware of that by recognising the CPU fan starting to run high. Moreover, a zombie process remains after exiting with "Quit" from the menu keeping the CPU busy as described. The only help is to kill the process (with top, e.g.).

1. Start /opt/jabref/bin/JabRef (from the console), make sure autoindexing is on
2. Open one or more larger libraries
3. Make some (random) edits in the library (I don't know which exactly but after a while the problem will show up)
4. After a while I can see JR remaining at the first place in top:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
.... .... 20 0 92,5g 1,6g 244056 S 200,7 10,2 7:16.49 JabRef

5. Use quit to exit JR

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
.... .... 20 0 92,5g 1,5g 244372 S 200,3 9,9 11:30.92 JabRef

6. I have to kill JR, otherwise it won't stop.

I don't know whether or how this issue might be related to some of the issues listed there.

Note: I have now switched back to JR 5.9. Opening the same libraries (with autoindexing switched on) and making edits, the problems seems not to occur. So, it seems not to be my specific configuration.

[ThiloteE]

Is indexing still running when cpu / ram maxes out? The first run after having upgraded JabRef to a new version usually reindexes your whole library. It is recommended to wait until it is finished, before starting to work. Also, is JabRef's RAM usage lower than your devices max available RAM capabilities? How many entries are we talking about here? Have you tried to follow https://docs.jabref.org/faq#q-i-have-a-huge-library.-what-can-i-do-to-mitigate-performance-issues?

If you cannot point to a trigger what exactly causes the undesired behavour, a fix is unlikely. If you are knowledable about coding (or have a friend that has IT background) and are willing and have the time to go deeper, you could set up a local workspace and do some debugging.

[ThiloteE]

Finding the regression window would also help. Try to find the exact commit that broke JabRef.

[ytzemih]

Hi @ThiloteE , thanks a lot for the questions and the FAQ hint. Let me quickly answer your questions as far as possible:

1. Indexing has finished. At least no process is running as far as I can see from clicking the Indexing icon in the top right. Although, one process seems to have stopped halfway through. But that hasn't changed in many starts of JR. So, none is running, no "cancel" button is active.
2. I'm aware of the after-update indexing process. I might nevertheless have interacted with JR before it stopped. Hmm.
3. JR uses about 10-20% of my RAM, that doesn't seem to be an issue. But under normal use it uses around 1-3%, speaking from my gut.
4. One file has about 4000 entries, the other around 200; most entries have PDFs linked with them.

Ok, I'll not be able to engage in coding, but I can try to find the regression window.

Also, is there a way to delete the old index and force JR to build up a new one?

[Siedlerchr]

You can rebuilt the index under Tools -> rebuild fulltext search index
Generally, Jabref waits for background to stop in time at exiting but if not it will try to force quit them after a timeout.

I would also recommend trying to reset the preferences and check if this has any effect

[wujastyk]

+1

JabRef 5.10--2023-06-12--92dcad3
Linux 5.15.0-73-generic amd64
Java 20.0.1
JavaFX 20+19

• "Background tasks are done"
• "Full text index" not selected.

Jabref continues to suck cycles in the background even after program exit. Has to be killed.

[mfguasti]

Similar problem here. Consumes lots of CPU and remains even if jabref is closed.
This behaviour begins when a word search is made.
JabRef 5.9--2023-01-08--76253f1a7
Linux 6.2.0-25-generic amd64
Java 19.0.1
JavaFX 19+11

[mfguasti]

Perhaps I should add that after installation of jabref_5.9_amd64.deb with QApt in kubuntu 22.10, it worked fine four or five times. But then, after that, it consumes CPU every time a word search is done. Have disabled index searching and other features to no avail.
This also happens systematically in other two computers one with kubuntu 22.10 (Huawei matebook) and another with kubuntu 22.04 (Lenovo)

[wujastyk]

still a problem
JabRef 5.10--2023-07-12--fd82bef
Linux 5.15.0-76-generic amd64
Java 21-internal
JavaFX 20+19

[Siedlerchr]

Do you have anything in the log files? JabRef writes log files to the file system https://docs.jabref.org/faq/linux#where-can-i-find-jabrefs-log-files

[mfguasti]

Thank you for your reply.
log.txt

Just opened jabref, high cpu consumption. closed app, jabref 'orphan' remained (see image). Attached is log.txt file, another log.txt.lock is empty.

[wujastyk]

JabRef 5.10--2023-07-21--1332e29
Linux 5.15.0-76-generic amd64
Java 21-internal
JavaFX 20+19

Just been working with Jabref, starting and quitting several times, and now, after quitting, Top shows me this:

[ThiloteE]

Hey, the other day I repaired my ancient computer from 2012 and learned a lot about hardware, which allowed me to squeeze every little bit of performance out of this old machine.

May I ask, if you could provide your hardware details, so that we can get better estimate the scope of the problem?
For example via

• inxi -Fx
• sudo lshw -short

Source: https://www.binarytides.com/linux-commands-hardware-info/

Edit: obviously, the real problem are the zombie processes, not the hardware! T.T

[wujastyk]

lshw and inxi outputs.zip

[ThiloteE]

@wujastyk your hardware is pretty decent. Clearly not from 2012 and definitely powerful enough for (non-buggy) JabRef and normal office work. I guess we somehow need to debug this. I might have a look tomorrow or one of these days and will try to reproduce on my linux machines. Maybe I find something.

[mfguasti]

lshw&inxi.txt

Here are also my files. Thanks for your help.

[shmilee]

+1
I have one file with 4000 entries too. And indexing, cpu/memory usage are completely intolerable.
Do we have a performance optimization plan?

This is my large bib file,(6M): AAA-papers.bib

[ThiloteE]

I tried to heavily open and close JabRef development version 5.10 from 203-07-25 (not doing anything else) and I do not have zombie processes or high cpu on my old machine from 2012 running Linux FedoraKDE 38, but it was only a small database of maybe 30 entries. I could imagine a database with thousands of entries to behave different or alternatively it must be triggered from something you do while working within JabRef.

[mfguasti]

I wonder if my database can be of use to try. I attach it.
Using it a few times and particularly doing a search tends to trigger the zombie high CPU.
Once it begins to do it, it keeps on doing it almost every time
phys.zip

[wujastyk]

My large bib is here, if it helps.

[HoussemNasri]

Hi everyone,

Unfortunately, I can't reproduce the behavior on my machine nor can any of the other maintainers. Our best bet is that someone having the problem could provide us with performance profiling information so we have something to debug.

We need a threadump of one of the zombie processes. Also, CPU/Memory samples for when the CPU usage spikes. You can start the sampling process after opening JabRef and stop it after reproducing the high CPU usage behavior. Everything mentioned above can be done with VisualVM. Feel free to include any additional information that VisualVM can generate.

[mfguasti]

I have downloaded visualvm 2.1.6 and can start its binary program.

What is not clear to me is what to do next.
Do I just start jabref on its own as usual and something will populate in this visualvm ?

[mfguasti]

Thank you for taking the time to look into this problem.
It is dreadful, I have to close jabref and open it again to work a little, and then again, it runs like mad and have to close it.

[HoussemNasri]

Hi @mfguasti,

Thank you for the quick response.

To capture a threadump, you need to launch JabRef and try to reproduce the zombie process behavior. Then go to visualVM and Right-click on JabRef node (the zombie process) in the Applications window and choose Thread Dump.

To capture CPU samples, you need to

1. Launch JabRef
2. Follow this guide to start capturing CPU samples
3. Go to JabRef and try to reproduce the high CPU consumption behavior
4. Once reproduced, go to VisualVM and stop the sampling
5. Save the result

Capturing memory samples is very similar to CPU, same as the steps above just in step 2 click "Memory" instead of "CPU" to start sampling in VisualVM.

[mfguasti]

Hi @HoussemNasri ,
thread-copy.txt
jabref-obj.csv
jabref-mem.nps.tar.gz

With great difficulty I managed to get the above files having the jabref zombie running. I am afraid that I am completely unfamiliar with java and with visualvm. It seems to me that you require a more competent collaborator. I could not sample the CPU, visualvm gave me a messsage that something was missing.

I hope this limited information helps.

[HoussemNasri]

Hi @mfguasti,

Thank you for your cooperation.

As an initial analysis, it seems the high CPU consumption is caused by the author's name suggestions when using the autocomplete feature. For some reason, it allocates multiple threads and continues running even after JabRef is closed.

To reproduce:

1. Enable the autocomplete feature from preferences
2. Open 2 big libraries, something with more than 1000 entries
3. Select one library and type something in the search bar very fast
4. Select the other library and type something in the search bar very fast

[HoussemNasri]

Hello everyone, please try this build and see if the problem persist.

[mfguasti]

I have just downloaded the debian file from your 'build' link. I will try it now.
I should mention that today I worked with jabref having turned the autocomplete option off, and it behaved ok.

[mfguasti]

JabRef 5.10-PullRequest10159.1233--2023-08-13--7de82a7
Linux 6.2.0-27-generic amd64
Java 21-internal
JavaFX 20+19

Turned the autocomplete feature again. So far so good!
@HoussemNasri You are brilliant!

[wujastyk]

Yes, I could reproduce the problem, after several tries, with autocompletion on, using

JabRef 5.10--2023-08-05--faa6288
Linux 6.2.0-26-generic amd64
Java 21-internal
JavaFX 20+19

Then I installed the new build,

JabRef 5.10-PullRequest10159.1233--2023-08-13--7de82a7
Linux 6.2.0-26-generic amd64
Java 21-internal
JavaFX 20+19

and the behaviour is good, so far. JabRef is not remaining in memory with high CPU, after closing down.

There are still some quirks with autocomplete, though. Maybe related, maybe need their own Git issue.

1. Autocomplete with "full firstname" works in the author field, but not in the editor field. In the editor field, the abbreviiated firstname is given.
2. The selction buttons for abbreviation options are round "radio" buttons, not square "checkbox" buttons. But their behaviour is as checkboxes. They are not mutually exclusive, as one expects with radio buttons. I stumbled on this accidentally.

[HoussemNasri]

1. Autocomplete with "full firstname" works in the author field, but not in the editor field. In the editor field, the abbreviiated firstname is given.

This one I'm not sure I understand, feel free to open a new issue for it.

2. The selction buttons for abbreviation options are round "radio" buttons, not square "checkbox" buttons. But their behaviour is as checkboxes. They are not mutually exclusive, as one expects with radio buttons. I stumbled on this accidentally.

Yeah, I noticed that too, I created an issue here #10161

[ytzemih]

First of all, thanks to all responding to this intricate issue and trying to figure out possible causes. After a while, yesterday, I had again time to update JR, in this case to:

JabRef 5.11--2023-09-06--ddbc736
Linux 6.2.0-32-generic amd64
Java 21-internal
JavaFX 20+19

I've had autocompletion switched off (for years now), I'm not using this feature. But I've had indexing switched on, using four files, one incl. about 4500 entries, and I'm using regexp search quite regularly.

Unlike hinted at, JR 5.11 does not seem to store logs in .cache/jabref/logs and I couldn't do anything about it, though the log4j output is flushed into the console where I can see stuff like

2023-09-06 22:57:51 [JavaFX-Launcher] sun.util.logging.internal.LoggingProviderImpl$JULWrapper.log()
WARN: Unsupported JavaFX configuration: classes were loaded from 'module org.jabref.merged.module', isAutomatic: false, isOpen: true
2023-09-06 22:57:54 [JavaFX Application Thread] sun.util.logging.internal.LoggingProviderImpl$JULWrapper.log()
WARN: Resource "" not found.

which might have to do with the logger settings.

Anway, indeed, the high CPU consumption had shown up again after a some quarter of an hour:

PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                       
26714 xxx     25   5  108,5g   3,4g 255792 S 101,7  22,4  59:02.72 JabRef 

I then followed the suggestions to switch off indexing (autocomplete was already switched off), to completely reset my configuration (prefs.xml), and make sure that the mentioned features are switched off.

The result: The problem seems to have gone for me.

1. Sorting and searching seems to work fine.
2. However, regarding performance: Typing a comment while the search field is non-empty renders typing quite slow in the larger database. After clearing the search field, typing again is fast as usual. It is interesting to see that there is an (at least from the user point unexpected) interaction between search and using the entry editor. Anyway, that point should have been addressed with Fix high CPU usage because of author name autocomplete #10159

I know, this information is not satisfactory for bugfixing, but I hope it helps to simplify the hunt.

[koppor]

@ytzemih Thank you for the updates. We are working on a new search at #8963. Can you try to subscribe to that PR? We are working on getting this done, but will take some months to complete.

[ytzemih]

Thanks, @koppor for pointing me to #8963, which I've subscribed now.

Just for the protocol: The problem has shown up again also with 5.11, after having JR running for around half an hour (despite having completely deleted my JR settings and with auto-indexing and -completion switched off). At least after closing JR, my top shows that the process is fully gone.

But, nevermind, I'm looking forward to the new search implementation.

[ThiloteE]

@ytzemih What version of JabRef is this? Are you compiling JabRef from source by any chance? When I use the newest development version, I do not see these error messages in the commandline log. Be aware, JabRef currently uses a customized version of JavaFX, so there are particular steps to follow, if you compile from source on linux.
Also, typing into fields is too slow very much sounds like issue #8977. We know of a workaround that involves sorting the maintable from low to high. Any other sort slows down JabRef.

[ytzemih]

@ThiloteE No, I'm using the binary from https://builds.jabref.org/main/jabref_5.11_amd64.deb (currently still from Sep 6 or 7). So, no compilation. Also, I am acting under the assumption that the JR deb just uses the Ubuntu JavaFX package. Do you think, that two JFX versions are interfering here?

Thanks, I've been following #8977 a little bit. Works for me as well. My pragmatic solution to the typing performance problem is to clear the search field after my search, and return to entry editor and typing is swift again. It's not a big deal, once you know the cause.