Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for redis-cluster #23869

Closed
pat-s opened this issue Apr 2, 2023 · 13 comments · Fixed by #23975
Closed

Support for redis-cluster #23869

pat-s opened this issue Apr 2, 2023 · 13 comments · Fixed by #23975
Labels
Milestone

Comments

@pat-s
Copy link
Member

pat-s commented Apr 2, 2023

Feature Description

Using Gitea with redis-cluster seems not possible as of right now.
When attempting so, the following error is thrown by Gitea

gitea 2023/04/01 23:31:36 ...ers/web/auth/auth.go:312:handleSignInFull() [E] [6428bed8] RegenerateSession: regenerate session: CROSSSLOT Keys in request don't hash to the same slot

This is a unique behavior of redis-cluster compared to a single redis installation and must be accounted for on the client side as described in https://dzone.com/articles/resolved-crossslot-keys-error-in-redis-cluster-mod.

When creating keys that could be used by multi-key operations, use hashtags ({…}) to force the keys into the same hash slot. When the key contains a “{…}” pattern, only the substring between the braces, “{” and “}”, is considered to calculate the hash slot.
For example, the keys {user1}:mykey1 and {user1}:mykey2 are hashed to the same hash-slot because only the string inside the braces “{” and “}”, that is, “user1”, is used to compute the hash-slot.

I can't infer if this would be a simple addition or a breaking change but wanted to drop the information about redis-cluster anyhow as I couldn't find a prior discussion about it.

Screenshots

No response

@pat-s pat-s added type/proposal The new feature has not been accepted yet but needs to be discussed first. type/feature Completely new functionality. Can only be merged if feature freeze is not active. labels Apr 2, 2023
@lunny
Copy link
Member

lunny commented Apr 2, 2023

Could you paste your configuration and redis cluster information here?

@pat-s
Copy link
Member Author

pat-s commented Apr 2, 2023

I just used the default config of the bitnami chart. Important: I used the latest v6 redis (chart: release 7.6.4) as the v7 has some compatibility issues with the underlying Debian image and runs into another issue (unrelated to Gitea).

@wxiaoguang
Copy link
Contributor

wxiaoguang commented Apr 2, 2023

You can modify your redis conn string in your Gitea config , add a "prefix" with slot, then p.c.Rename(ctx, poldsid, psid) in code won't report the CROSSSLOT error. I guess: redis://..../?...&prefix={session}:

@lunny
Copy link
Member

lunny commented Apr 2, 2023

See the possible configuration connection string https://github.com/go-gitea/gitea/blob/main/modules/nosql/redis.go#L18-L22

@wxiaoguang
Copy link
Contributor

The "prefix" option is not documented, but cluster needs it. Make it prefix={session}: , then all sessions keys become {session}:random-session-id, it could make the redis cluster happy.

@silverwind
Copy link
Member

silverwind commented Apr 3, 2023

Tangentially related: There is also redis-sentinel, a kind of "proxy" daemon that may need special support inside our code. I think it can be used in addition to redis-cluster. It is the premier HA option for redis.

@wxiaoguang
Copy link
Contributor

To fix the problem, either:

  1. Let users do "?prefix={session}:" trick
  • OR -
  1. Do not use Rename in redis session provider , just Get and Set, then no cross-cluster problem.

@pat-s
Copy link
Member Author

pat-s commented Apr 7, 2023

I came across this while testing redis-cluster support for the helm chart. Happy to try again once there is a built-in fix for this which does not require manual modifications. This way we can have a "clean" start when adding support for redis-cluster.

@wxiaoguang
Copy link
Contributor

Can you try this PR: Use Get/Set instead of Rename when Regenerate session id #23975 ?

@pat-s
Copy link
Member Author

pat-s commented Apr 7, 2023

Not as a PR as I would need an image to test it with the Helm chart.

@wxiaoguang
Copy link
Contributor

Approve and merge, then get a dev image?

@lunny lunny added the type/bug label Apr 7, 2023
@lunny lunny added this to the 1.19.1 milestone Apr 7, 2023
@lunny lunny removed type/proposal The new feature has not been accepted yet but needs to be discussed first. type/feature Completely new functionality. Can only be merged if feature freeze is not active. labels Apr 7, 2023
@lunny
Copy link
Member

lunny commented Apr 7, 2023

It should be a bug rather than a feature because we have already supports redis cluster.

6543 pushed a commit that referenced this issue Apr 7, 2023
Do not use Rename here, because the old sid and new sid may be in
different redis cluster slot.

Fix #23869
GiteaBot pushed a commit to GiteaBot/gitea that referenced this issue Apr 7, 2023
)

Do not use Rename here, because the old sid and new sid may be in
different redis cluster slot.

Fix go-gitea#23869
jolheiser pushed a commit that referenced this issue Apr 7, 2023
…3983)

Backport #23975 by @wxiaoguang

Do not use Rename here, because the old sid and new sid may be in
different redis cluster slot.

Fix #23869

Co-authored-by: wxiaoguang <[email protected]>
@pat-s
Copy link
Member Author

pat-s commented Apr 13, 2023

This PR together with #24114 resulted in a successful connection to a redis-cluster instance 🎉

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 29, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants