Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redis cache grows unilimmited #25487

Closed
toxix opened this issue Nov 6, 2019 · 4 comments
Closed

Redis cache grows unilimmited #25487

toxix opened this issue Nov 6, 2019 · 4 comments
Assignees
Labels
Fixed in 2.4.x The issue has been fixed in 2.4-develop branch Issue: Format is valid Gate 1 Passed. Automatic verification of issue format passed

Comments

@toxix
Copy link
Member

toxix commented Nov 6, 2019

Preconditions (*)

  1. Magento 2.3.3
  2. Redis Cache enabled
  3. Production mode

Steps to reproduce (*)

  1. Run Magento in Production with 15 Stores and organic trafic
  2. Redis caching enabled

Expected result (*)

  1. Redis cache will stabilize in size.
  2. Cache key expiery is set for all things that do have an expiery

Actual result (*)

  1. Redis is growing in size to unlimited (>30GB)
  2. Cache key expirey is not set
> info keyspace
# Keyspace
db0:keys=45775,expires=41711,avg_ttl=79567831
db1:keys=49089,expires=27837,avg_ttl=16355872
db2:keys=706,expires=706,avg_ttl=696476

Further information

The only way to prevent our production system server from dying was to set a hard limit for Redis memory consumption. The relying on the memory limit of redis has some performance impact and there are always things cleared that don't need to and vice versa. This should be handled by the application with setting an resonable ttl.

fix

This is actualy an known bug and already long time fixed in an dependency of magento core composer file. See also colinmollenhour/Cm_Cache_Backend_Redis@bc63e72
But this is not included in current magento version Last update of this core depedency was in September 2018: 32058c7

Applying this change as a hotfix, actualy stabalize our production redis at 3,5GB ram usage, instead of growing over 30GB. And also boost the performance of the system quiete a bit.

Hot Fix

Add the following requirement to your project composer.json file:

    "colinmollenhour/cache-backend-redis": "1.11.0 as 1.10.6"

Fix

I will File a PR with an updated composer dependency to the magento core and reference this issue.

@magento-engcom-team magento-engcom-team added the Issue: Format is valid Gate 1 Passed. Automatic verification of issue format passed label Nov 6, 2019
@m2-assistant
Copy link

m2-assistant bot commented Nov 6, 2019

Hi @toxix. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

  • Summary of the issue
  • Information on your environment
  • Steps to reproduce
  • Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento give me 2.3-develop instance - upcoming 2.3.x release

For more details, please, review the Magento Contributor Assistant documentation.

@toxix do you confirm that you were able to reproduce the issue on vanilla Magento instance following steps to reproduce?

  • yes
  • no

toxix added a commit to toxix/magento2 that referenced this issue Nov 6, 2019
I experienced a situation on production system with 15 stores and
organic trafic, where the redis cache grows to over 30GB. It looks like
magento is putting a lot of entries into the redis cache without any
expiary. I needed to limit redis memory to prevent the server from
dying, not how it should be. This had some performance influences.

For this issue there is actualy a fix in place which just need to be
pulled in. This commit is do this. With the fix in place the redis
stabalized at arround 3GB. Over all performance is was also increased
with this change.

Referces:
[1] colinmollenhour/Cm_Cache_Backend_Redis@bc63e72
[2] magento@32058c7
[3] magento#25487
@ghost ghost assigned toxix Nov 6, 2019
@PascalBrouwers
Copy link
Contributor

@toxix Having the same problem but only at certain periods it will grow unlimited. I will see if your fix works too.

@kilis
Copy link

kilis commented Dec 16, 2019

Same is also on Magento version 2.1.7. Did install only that fix, but nothing has really changed.

@magento-engcom-team
Copy link
Contributor

@magento-engcom-team magento-engcom-team added the Fixed in 2.4.x The issue has been fixed in 2.4-develop branch label Feb 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Fixed in 2.4.x The issue has been fixed in 2.4-develop branch Issue: Format is valid Gate 1 Passed. Automatic verification of issue format passed
Projects
None yet
Development

No branches or pull requests

4 participants