feat: "onlyOnNoPrefix" - detect browser locale when no prefix

[dword-design]

Currently, the browser detection overrides an existing locale in the route. For example when I'm calling /en/foo, it will redirect to /de/foo although there is already a locale in the route. This PR adjusts the priority.

/foo -> /de/foo
/de/foo -> /de/foo
/en/foo -> /en/foo

[divine]

What's the reason you have removed STRATEGIES.NO_PREFIX?

This shouldn't be working within that strategy at all.

[dword-design]

I thought it‘s not needed anymore because the check is already at the top. But maybe it‘s not redundant.🤔

[rchl]

Many tests are failing and also this change doesn't quite make sense to me since if you will prioritize route's locale instead of detecting the browser locale then that will make browser locale detection pretty useless (it will probably only work for non-existant routes).

Anyway, are you aware of detectBrowserLanguage.onlyOnRoot option? I think this is what you might want to look into.

[dword-design]

The browser detection will trigger if the route does not include a locale. So /foo and / still use the browser detection. That‘s why there is the null check.

Could be that the tests work again when the prefix check is back in. I could check that later.

[rchl]

The browser detection will trigger if the route does not include a locale. So /foo and / still use the browser detection.

I think you are misunderstanding how getLocaleFromRoute works. It doesn't check the locale in the path, it checks the internally assigned route locale. And that is pretty much always set for valid routes, even if the route path doesn't have a prefix.

[dword-design]

@rchl That's partially right. It indeed checks the locale route via a regex, but also checks the name of the route and this leads to problems in my solution if the strategy is not prefix. I adjusted the implementation to use the regex, which fixes the tests, and I added two tests for this PR.

[rchl]

• This is a breaking change and needs to be guarded by another option. Otherwise, it will for example affect alwaysRedirect behavior in an unexpected way (I think we don't have tests that cover that case).
• Related to above, we need tests for navigating with i18n_redirected cookie set to verify that this behavior doesn't trigger with alwaysRedirect set. And that it does trigger with yet-to-be-added option added. (You can look at fallbacks to default locale with invalid locale cookie test for inspiration).
• The onlyOnRoot should probably "win" over this behavior when set, which it is not doing right now (that also pertains to missing option for this behavior).

[rchl]

And a general question:
What's the motivation for introducing such behavior?
Is there any "prior art" related to this change (are there other libraries that handle browser locale detection in such a way)?
Is it motivated by some SEO "best practices"?

I'd like to know why you think this change makes sense.

[dword-design]

Alright let's have a look. Firstly, I added your suggested code changes.

Breaking change:
For me this still feels like a bugfix because it does not make sense that a route that includes an explicit locale is redirected by default. But this is up to the maintainers to decide that I guess, so I could also go with an option.

alwaysRedirect:
Yes I'd also say that this should have precedence. I added a test and an implementation. The implementation feels a bit improvised, I think someone more into the code can maybe suggest a better way to do this.

onlyOnRoot:
Do you have a concrete use case where the current implementation does not do the expected behavior? I thought a bit about it and didn't find something.

Motivation:
Yes, so first of all there is the much-linked Google multilingual recommendations page:
https://support.google.com/webmasters/answer/182192
This states that cookie-based approaches are discouraged. Without a cookie, it's currently not possible to navigate multiple languages, so there we are at this ticket.

Also, there are many websites that as far as I see it go with my approach. For example (without cookies):
https://evernote.com
https://www.jenskommnick.de (Joomla-based I think)
And I think Drupal also does it like this. I don't have an example though.

There are more, but maybe that's fine for the start.

I'm actually wondering why it's not the default behavior to be able to navigate between languages without cookies but still using browser detection when not providing a language code. It's probably the use case for all my websites. But requirements are different I guess 😄.

[rchl]

For me this still feels like a bugfix because it does not make sense that a route that includes an explicit locale is redirected by default. But this is up to the maintainers to decide that I guess, so I could also go with an option.

Given the number of people using this module (or number of people using sites using this module), it's bound to be a breaking change for some. Better play it safe as we did with onlyOnRoot option.

That said, the plan is to make the recently introduced onlyOnRoot option enabled by default in the next major version.

onlyOnRoot:
Do you have a concrete use case where the current implementation does not do the expected behavior? I thought a bit about it and didn't find something.

Well, first we have to decide what would the option for this behavior look like. Adding another option would likely be confusing as then it's not clear how they interact with each other and make it harder to decide on the correct options to go with.

The ideal would probably be to have onlyOnRoot and this new option under the same (renamed) option but we can't rename options without making breaking change. So maybe just have add a new option and I'd have to clean that up in the next major version...

Also, there are many websites that as far as I see it go with my approach. For example (without cookies):
evernote.com

Evernote one doesn't seem to redirect at all. Visiting root page (or one of the sub-pages) doesn't redirect me to the Polish version even though I've set Polish as my preferred language.

jenskommnick.de (Joomla-based I think)

This one seems to redirect me to the English version, even when having German as my preferred language...

I'm actually wondering why it's not the default behavior to be able to navigate between languages without cookies but still using browser detection when not providing a language code. It's probably the use case for all my websites. But requirements are different I guess 😄.

Why not default? Probably some bad decision when creating this module (which was before I've taken over). But as I've said, the plan is to enable onlyOnRoot by default which should remedy this problem in most cases.

Now here is something that I'm concerned about this change...

The onlyOnRoot option was added after many back-and-forth comments. The logic introduced in your change was also considered. The main problem that onlyOnRoot was addressing was SEO related -- the fact that crawlers got redirected, making it impossible for them to crawl all locales.

With onlyOnRoot enabled, that problem is mostly remedied as only the root path (/) will get redirected. But with your change, which is basically a more lax version of onlyOnRoot, the SEO issues are likely to be more severe than with onlyOnRoot because when using PREFIX_EXCEPT_DEFAULT strategy (which is the default one), the default locale is not prefixed and thus all URLs for that locale are potentially gonna get redirected.

So I'm not sure if I really want this change in and also I wonder why onlyOnRoot doesn't work for you. Sure, /foo -> /de/foo redirect won't happen in that case but if you care about SEO, you should avoid redirects as much as possible, as the google's article says.

[dword-design]

@rchl Thanks for your elaboration! I'd say we should jump to requirements before we (or I) continue developing. So, here is the set of requirements that I have in mind. If there is a crack in my thoughts, go ahead. The browser locale is de and the default locale is en for simplicity's sake. The strategy is prefix.

1. Cookies aren't used at all because crawlers are stateless and do not use cookies.
2. The user opens the website via /. He is redirected to /de because cookies are not enabled (see 1).
3. There is a backlink from somewhere else, but the external source does not know the language of the user. So he links to /foo. When the user opens /foo, he is redirected to /de/foo.
4. The user opens /foo and his browser language is fr. He is redirected to /fr/foo.

What I actually do not like so much is that root is always redirected because in some cases there are problems when pasting the url into a form and the form validates the existence of the website. In these cases the redirect is considered a problem (but this would also a problem with onlyOnRoot).

I had some thoughts of not using a prefix and still showing the website in the browser's language, but this should always break crawling, or am I wrong?

Maybe you can comment on the requirements and maybe we could conclude some "cook book" like doc from it, which would make selecting the right config easier 😃.

P.S.: It's interesting that the redirect does not work for the two examples. It could be that the language detection works differently. In my case it both redirects to /de.

[rchl]

4. The user opens /foo and his browser language is fr. He is redirected to /foo/en.

This wouldn't happen because en doesn't have a prefix if it's the default locale.
Also, I guess you meant /en/foo and not /foo/en.

What I actually do not like so much is that root is always redirected because in some cases there are problems when pasting the url into a form and the form validates the existence of the website. In these cases the redirect is considered a problem (but this would also a problem with onlyOnRoot).

So maybe consider using the cookie to help in those real-user cases. It shouldn't affect the crawlers since crawlers ignore cookies.

I had some thoughts of not using a prefix and still showing the website in the browser's language, but this should always break crawling, or am I wrong?

Yes, that wouldn't be crawler-friendly. There is a NO_PREFIX strategy for enabling that but you should only use that if you don't care about crawler having access to all locales.

P.S.: It's interesting that the redirect does not work for the two examples. It could be that the language detection works differently. In my case it both redirects to /de.

Which examples?

Maybe you should create a repo that reproduces your situation so that we are on the same page?

[MikeScheurwater]

Hey @rchl, the behaviour @dword-design is trying to implement is a function I need as well. My website uses multiple languages and my idea is to send users links to pages without a locale (eg. /privacy). The behaviour I'm expecting is that the user opens the link and then is redirected to his/hers browser-language. At the moment this function doesn't work (well). When I have a defaultLocale and navigate to /privacy while my browser is set to another language (Dutch), i'm always redirected to the English version.

[rchl]

@MikeScheurwater please report a separate issue with details (your configuration and ideally repo that reproduces). From your description, it doesn't seem like your issue is directly related. It looks like it's maybe a configuration issue.

[dword-design]

Hey @rchl,

4. The user opens /foo and his browser language is fr. He is redirected to /foo/en.

This wouldn't happen because en doesn't have a prefix if it's the default locale.

It does have if the strategy is prefix. I added this to my above post.

Also, I guess you meant /en/foo and not /foo/en.

Yes I fixed this

What I actually do not like so much is that root is always redirected because in some cases there are problems when pasting the url into a form and the form validates the existence of the website. In these cases the redirect is considered a problem (but this would also a problem with onlyOnRoot).

So maybe consider using the cookie to help in those real-user cases. It shouldn't affect the crawlers since crawlers ignore cookies.

Yes this sounds like an option for this, thanks.

I had some thoughts of not using a prefix and still showing the website in the browser's language, but this should always break crawling, or am I wrong?

Yes, that wouldn't be crawler-friendly. There is a NO_PREFIX strategy for enabling that but you should only use that if you don't care about crawler having access to all locales.

P.S.: It's interesting that the redirect does not work for the two examples. It could be that the language detection works differently. In my case it both redirects to /de.

Which examples?

I meant the two websites I posted as a reference.

Maybe you should create a repo that reproduces your situation so that we are on the same page?

That's a good idea. There you go:
https://github.com/dword-design/i18n-module-locale-priority
Looking for your feedback.

[rchl]

That's a good idea. There you go:
dword-design/i18n-module-locale-priority
Looking for your feedback.

i18n is not set up in that repo.

[dword-design]

@rchl No it's a prototype for the concept of this PR. What have you expected?

[rchl]

I'm not sure what I've expected as I've lost the track of the conversation. :)

[rchl]

I've pushed a change that puts this functionality behind an option and added some extra tests.

It's good on my side, feel free to check it out before I merge it.

[dword-design]

Jup works for me. Thanks for your time and merging! 😊