Elasticsearch container failures

[atb1r21]

Over the last few months I have been testing a migration of the docker-compose deployment from RHEL 7 to RHEL 9.

I am finding that over a period of time when I run the elasticsearch container, eventually the container starts to fail health checks.

The only solution I have found to far is to clear the 'elasticsearch' directory and recreate it from scratch to get the service back online.

I was wondering if this was a result of running on RHEL 9 and could be a bug, as I have not observed it on RHEL 7.

The service generally runs consistently aside from a reboot once a month for a patching cycle.

I wanted to sanity check with the developers if this has been observed on your side. Generally the docker logs do not points to a critical system error regarding the elasticsearch service.

I have started again today and am happy to provide you with logs for the container if the problem recurs.

[hanars]

We have never seen this issue, but we also never left elasticsearch running for a month at a time without restarting it, and we were not using RHEL, so its not the most comparable situation unfortunately.

To manage your expectations, we will most likely not be able to help debug this issue for you, as we do not have the ability to create a system that mirrors your own in order to isolate the issue. If you are able to identify why this is happening and describe a fix that would help (or ideally provide a pull request with the needed change) we would be happy to integrate that fix into the main version of seqr. If the issue does continue to happen we can take a look at your logs for you, but if the issue is not apparent in the logs there will not be much we can do