automatically change language if cookied #3968

peterbe · 2021-06-04T03:42:10Z

This is something we've wanted to do for a long time. I've thought about it for a long time too. #3949 is just one of many similar issues filed. A lot of people change the locale, but keep stumbling into different locales which are different from what they prefer. For example, a user in Spain prefers to read in en-US because they're comfortable with that and/or they know it's more up-to-date. But a Google search in their country for "mdn array foreach" links to the /es/docs/Web/... URL. Or, someone prefers zh-CN but there's a blog post or tweet with an absolute URL for en-US.

According to the GA event we have on the "Change language" form (document footer), only ~15% use that where they don't have a preferredlocale cookie already. In the last 30 days, ~107,000 times people have used the "Change language" button. And ~97,000 times people have clicked on the "Switch to English". So roughly 200k times people have had to change the language. With this PR, it effectively does that language switch automatically for them.

It would have been nice to do this on the server, with a 302 redirect but then we'd need to make the CDN cache hashing depend on this cookie and the worst thing is that in the Lambda@Edge code (where we'd do that redirect), it can't know if the translation is available (unless it's for redirecting to en-US which can always be assumed to exist now that slugs are never translated).

Fixes mdn#3949

schalkneethling · 2021-06-07T10:54:18Z

This is so cool Peter and works really well except for one use case 🙃.
It breaks the "View in English" link if you have selected a different cookie-backed locale.

So, if I change my language to say Spanish and then navigate around, get to a page that looks to be out of date, and click the "View in English" link, it will just keep redirecting me back to es. I would have to change my preferred locale using the language switcher in order to get to the English version.

Could we make it so that the "View in English" clears the cookie value or, somehow bypasses it?

schalkneethling · 2021-06-07T10:46:15Z

client/src/ui/molecules/language-menu/index.tsx

+    ) {
+      const newPathname = translateURL(pathname, locale, cookieLocale);
+      // Just to be absolutely paranoidly certain it's not going to redirect
+      // to the URL you're already don, we're doing this extra check.


Suggested change

// to the URL you're already don, we're doing this extra check.

// to the URL you're already on, we're doing this extra check.

peterbe · 2021-06-07T11:18:58Z

This is so cool Peter and works really well except for one use case 🙃.
It breaks the "View in English" link if you have selected a different cookie-backed locale.

So, if I change my language to say Spanish and then navigate around, get to a page that looks to be out of date, and click the "View in English" link, it will just keep redirecting me back to es. I would have to change my preferred locale using the language switcher in order to get to the English version.

Could we make it so that the "View in English" clears the cookie value or, somehow bypasses it?

We could definitely do that. Although I always get nervous about trying to inject a onClick on a link. Because it would break if you right-click and "Open in a new Tab". So to solve that, we'd need to put a query string on the link.

OR, we consider deleting that link. Seriously. I.e. that it always, and only, has "Change language" in the top bar (right of breadcrumbs).
Now, all those people in Spain who use google.es, who often stumble into non-English MDN URLs, they'd already be saved by this new redirect functionality. I.e. you only have to change to English (US) once, and no need to press that "View in English" button ever again. (or at least in your current browser).

peterbe · 2021-06-07T11:19:29Z

I'm changing this to a draft because I clearly forgot to address the fact that the "View in English" button broke.

schalkneethling · 2021-06-08T04:54:30Z

OR, we consider deleting that link. Seriously. I.e. that it always, and only, has "Change language" in the top bar (right of breadcrumbs).
Now, all those people in Spain who use google.es, who often stumble into non-English MDN URLs, they'd already be saved by this new redirect functionality. I.e. you only have to change to English (US) once, and no need to press that "View in English" button ever again. (or at least in your current browser).

I reckon this is a worthwhile experiment. Having the "View in English" link was an experiment in itself to get a sense of how often people would use it. We now have some numbers and I reckon this is the login next step. Roll this out, have a look at the numbers again. If you see that now, there are a lot more events where people switch from X to English using the language selector, then perhaps that could be a signal that we need to bring back the link.

I suspect that with this redirect logic, we are going to see people come into a document, change to their actually preferred language, and then be done with it until they are in a different browser or they clear cookies.

KartikSoneji · 2021-07-14T00:12:51Z

I'm changing this to a draft because I clearly forgot to address the fact that the "View in English" button broke.

I have three suggestions that keep the View in English link:

This seems like the perfect usecase for sessionStorage.
Change the link to a button and when the user clicks View in English, the page sets sessionStorage.localeOverride = "en-US", and the redirection code checks sessionStorage.localeOverride before redirecting.
The page can even clear the override by adding an event listener for beforeunload.
Use a localeOverride=en-US query parameter.
As an added benefit, it will allow linking to a specific locale of a page, without which automatic redirects would break text fragments (#:~:text= links).
(Side note, I couldn't find an MDN entry for text fragments)
The redirection code checks the localeOverride query parameter before redirecting.

Change the link to a button that always opens in a new tab.
This fits with the use-case of using the English version as a reference along with the translated content.
Eg: (View in English )

let enUsTab = window.open(window.location.href);
enUsTab.sessionStorage.localeOverride = "en-US";

enUsTab.location.reload();
// or
enUsTab.location = new URL(window.location.pathname.replace(/[\w\-]+/, "en-US"), window.location);

Personally, I think the link will be useful because the authoritative source is always in English, and translations might not capture the meaning completely or be inaccurate or incomplete.
But I only use MDN in English, so maybe someone who uses a different locale can weight in?

peterbe · 2021-07-14T20:04:22Z

@KartikSoneji Of all those wonderful suggestions, I would favor a simple '#localeOverride=en-US` or something.
I often, when I view a translated page, right-click and "Open in a new tab" (or window) so that I can compare the two pages side-by-side and I very much suspect this is how many translators use it too. So if it was a button, you'd lose that functionality.

What we can do is something like this:

<a href="#localOverride=en-US" onClick={event => {
  changeLanguage('en-US');
}}>View in English</a>

Then, it would change over to English instantly without a page load. And if they used right-click and "Open in new tab" the page would stay in English.
And since we're allowing the click-through, the Back button would effectively make the URL go from http://URL#localOverride=en-US back to http://URL and it's what you'd expect to happen.

You inspire me to give this a try!

KartikSoneji · 2021-07-14T21:28:32Z

Of course! That is brilliantly simple.

peterbe · 2021-07-14T21:40:17Z

Now it works really well.

People who prefer a certain locale always get their page in that locale if they arrive on MDN with a URL that doesn't have the locale they prefer. And this change is not based on optimism; it only does the automatic (pushState client-side only!) redirect if the translation exists.
If you, for example, have a cookie that says you prefer zh-CN you can still click "View in English" and see it in English. And you can click left-click that or right-click and "Open in new tab".
People who haven't got a preference cookie don't get that #localeOverride in their URL when they click "View in English"

The screencast hopefully demonstrates all of this:

https://user-images.githubusercontent.com/26739/125696581-79f-90d0-48ac-821a-3e3452f41981.mov

@KartikSoneji Would you mind testing it out too. Do you know how to enable the translated content in a locale yari instance?

KartikSoneji

#localeOverride is working well, but it breaks all self-links on the English version of the page.
A simple fix is to remember if the locale was overridden even if the hash changes.
We can still retain the forward/back navigation with history.replaceState and listening to the popstate event. As an added bonus, it allows us to remove the hash after load, which I personally feel looks cleaner.

KartikSoneji · 2021-07-14T22:32:07Z

client/src/preferred-locale.ts

+  let value = document.cookie
+    .split("; ")
+    .find((row) => row.startsWith(`${PREFERRED_LOCALE_COOKIE_NAME}=`));
+  if (value && value.includes("=")) {


Nitpick, but we can use the new optional chaining operator here (yay!).

Suggested change

if (value && value.includes("=")) {

if (value?.includes("=")) {

KartikSoneji · 2021-07-14T22:48:42Z

client/src/ui/molecules/language-toggle/index.tsx

+
+  React.useEffect(() => {
+    const cookieValue = getPreferredCookieLocale(document);
+    if (cookieValue && cookieValue.toLowerCase() !== "en-us") {


Suggested change

if (cookieValue && cookieValue.toLowerCase() !== "en-us") {

if (cookieValue?.toLowerCase() !== "en-us") {

KartikSoneji · 2021-07-15T00:01:18Z

client/src/ui/molecules/language-menu/index.tsx

@@ -19,13 +21,36 @@ export function LanguageMenu({
  native: string;
 }) {
  const ga = useGA();
-  const { pathname } = useLocation();
+  const { hash, pathname } = useLocation();
  const navigate = useNavigate();
  const [preferredLocale, setPreferredLocale] = React.useState(locale);



Suggested change

const [preventLocaleOverride, setPreventLocaleOverride] =

React.useState(false);

React.useEffect(() => {

window.addEventListener("popstate", (event) => {

if (event.state?.resetLocaleOverride) {

setPreventLocaleOverride(false);

}

});

}, []);

KartikSoneji · 2021-07-15T00:03:57Z

client/src/ui/molecules/language-menu/index.tsx

+  React.useEffect(() => {
+    const cookieLocale = getPreferredCookieLocale(document);
+    if (
+      locale &&
+      cookieLocale &&
+      locale.toLowerCase() !== cookieLocale.toLowerCase() &&
+      // If the URL is something like `#localeOverride` we omit this
+      // automatic "redirect" because the user has most likely clicked
+      // a link that means that want to "peek" at a locale that is
+      // different from what their cookie prefers.
+      !hash.toLowerCase().includes(LOCALE_OVERRIDE_HASH.toLowerCase()) &&
+      translations
+        .map((t) => t.locale.toLowerCase())
+        .includes(cookieLocale.toLowerCase())
+    ) {
+      const newPathname = translateURL(pathname, locale, cookieLocale);
+      // Just to be absolutely paranoidly certain it's not going to redirect
+      // to the URL you're already don, we're doing this extra check.
+      if (newPathname !== pathname) {
+        navigate(newPathname);
+      }
+    }
+  }, [locale, hash, pathname, navigate, translations]);


Suggested change

React.useEffect(() => {

const cookieLocale = getPreferredCookieLocale(document);

if (

locale &&

cookieLocale &&

locale.toLowerCase() !== cookieLocale.toLowerCase() &&

// If the URL is something like `#localeOverride` we omit this

// automatic "redirect" because the user has most likely clicked

// a link that means that want to "peek" at a locale that is

// different from what their cookie prefers.

!hash.toLowerCase().includes(LOCALE_OVERRIDE_HASH.toLowerCase()) &&

translations

.map((t) => t.locale.toLowerCase())

.includes(cookieLocale.toLowerCase())

) {

const newPathname = translateURL(pathname, locale, cookieLocale);

// Just to be absolutely paranoidly certain it's not going to redirect

// to the URL you're already don, we're doing this extra check.

if (newPathname !== pathname) {

navigate(newPathname);

}

}

}, [locale, hash, pathname, navigate, translations]);

React.useEffect(() => {

const cookieLocale = getPreferredCookieLocale(document);

const hasLocaleOverrideHash = hash

.toLowerCase()

.includes(LOCALE_OVERRIDE_HASH.toLowerCase());

if (

!preventLocaleOverride &&

cookieLocale &&

locale?.toLowerCase() !== cookieLocale.toLowerCase() &&

// If the URL is something like `#localeOverride` we omit this

// automatic "redirect" because the user has most likely clicked

// a link that means that want to "peek" at a locale that is

// different from what their cookie prefers.

!hasLocaleOverrideHash &&

translations

.map((t) => t.locale.toLowerCase())

.includes(cookieLocale.toLowerCase())

) {

const newPathname = translateURL(pathname, locale, cookieLocale);

// Just to be absolutely paranoidly certain it's not going to redirect

// to the URL you're already don, we're doing this extra check.

if (newPathname !== pathname) {

navigate(newPathname);

}

}

// Remember that the locale was overriden even if the hash changes,

// otherwise it breaks self-links.

// Remove the hash so the url looks cleaner.

if (hasLocaleOverrideHash) {

setPreventLocaleOverride(true);

window.history.pushState(

{ resetLocaleOverride: true },

"",

window.location.href.replace(/#.*$/, "")

);

}

}, [preventLocaleOverride, locale, hash, pathname, navigate, translations]);

This should fix the self-links, and preserve forward/back navigation.

KartikSoneji · 2021-07-15T09:24:06Z

@peterbe
Here is a diff, you can apply this with git apply or patch.

diff --git a/client/src/ui/molecules/language-menu/index.tsx b/client/src/ui/molecules/language-menu/index.tsx
index 4ee9078c..b1a48e96 100644
--- a/client/src/ui/molecules/language-menu/index.tsx
+++ b/client/src/ui/molecules/language-menu/index.tsx
@@ -24,21 +24,35 @@ export function LanguageMenu({
   const { hash, pathname } = useLocation();
   const navigate = useNavigate();
   const [preferredLocale, setPreferredLocale] = React.useState(locale);
+  const [preventLocaleOverride, setPreventLocaleOverride] =
+    React.useState(false);
+
+  React.useEffect(() => {
+    window.addEventListener("popstate", (event) => {
+      if (event.state?.resetLocaleOverride) {
+        setPreventLocaleOverride(false);
+      }
+    });
+  }, []);
 
   // This effect makes you automatically navigate to the locale your cookie
   // prefers if the current page's locale isn't what you prefer and the
   // locale you prefer is one of the valid translations.
   React.useEffect(() => {
     const cookieLocale = getPreferredCookieLocale(document);
+    const hasLocaleOverrideHash = hash
+      .toLowerCase()
+      .includes(LOCALE_OVERRIDE_HASH.toLowerCase());
+
     if (
-      locale &&
+      !preventLocaleOverride &&
       cookieLocale &&
-      locale.toLowerCase() !== cookieLocale.toLowerCase() &&
+      locale?.toLowerCase() !== cookieLocale.toLowerCase() &&
       // If the URL is something like `#localeOverride` we omit this
       // automatic "redirect" because the user has most likely clicked
       // a link that means that want to "peek" at a locale that is
       // different from what their cookie prefers.
-      !hash.toLowerCase().includes(LOCALE_OVERRIDE_HASH.toLowerCase()) &&
+      !hasLocaleOverrideHash &&
       translations
         .map((t) => t.locale.toLowerCase())
         .includes(cookieLocale.toLowerCase())
@@ -50,7 +64,19 @@ export function LanguageMenu({
         navigate(newPathname);
       }
     }
-  }, [locale, hash, pathname, navigate, translations]);
+
+    // Remember that the locale was overriden even if the hash changes,
+    // otherwise it breaks self-links.
+    // Remove the hash so the url looks cleaner.
+    if (hasLocaleOverrideHash) {
+      setPreventLocaleOverride(true);
+      window.history.pushState(
+        { resetLocaleOverride: true },
+        "",
+        window.location.href.replace(/#.*$/, "")
+      );
+    }
+  }, [preventLocaleOverride, locale, hash, pathname, navigate, translations]);
 
   return (
     <form

KartikSoneji · 2021-07-17T19:41:14Z

Fixes #2724

The platform doesn't seem to remember my language preference #2724

Might resolve:

peterbe · 2021-07-27T13:35:31Z

@KartikSoneji Not a day goes by where I don't feel sad that I haven't had a chance to enjoy your feedback. This tab's still open and I haven't forgotten. Just got to put out fires and planned work first.

KartikSoneji · 2021-07-28T23:45:43Z

Of course, I completely understand.

NiedziolkaMichal · 2022-03-03T11:31:10Z

Would it be possible to get back on this? It would be a great improvement to the new UI.

schalkneethling · 2022-03-03T12:30:46Z

Would it be possible to get back on this? It would be a great improvement to the new UI.

This is definitely on our list of items to pick up again as soon as possible. Thank you for the nudge @NiedziolkaMichal

github-actions · 2022-06-20T17:03:35Z

This pull request has merge conflicts that must be resolved before it can be merged.

automatically change language if cookied

24beeea

Fixes mdn#3949

peterbe requested a review from schalkneethling June 4, 2021 12:36

schalkneethling suggested changes Jun 7, 2021

View reviewed changes

peterbe marked this pull request as draft June 7, 2021 11:19

Merge branch 'main' into 3949-automatically-change-language-if-cookied

b2e9d61

localeOverride

5306b7f

peterbe marked this pull request as ready for review July 14, 2021 21:40

KartikSoneji suggested changes Jul 15, 2021

View reviewed changes

tidying

ce93ddb

github-actions bot added the 🐌 idle Issues and PRs without recent activity. Flagged for maintainer follow-up. label Dec 8, 2021

github-actions bot removed the 🐌 idle Issues and PRs without recent activity. Flagged for maintainer follow-up. label Mar 3, 2022

schalkneethling mentioned this pull request Mar 10, 2022

Suggestion: Persist language selection #5498

Closed

schalkneethling mentioned this pull request Mar 21, 2022

Issue with "Send feedback about MDN Web Docs": Store selected language. #5461

Closed

schalkneethling added the 🧑‍🤝‍🧑 community contributions by our wonderful community label Apr 22, 2022

Merge branch 'main' into 3949-automatically-change-language-if-cookied

a077868

caugner self-assigned this May 10, 2022

github-actions bot added the merge conflicts 🚧 Please rebase onto or merge the latest main. label Jun 20, 2022

github-actions bot added the 🐌 idle Issues and PRs without recent activity. Flagged for maintainer follow-up. label Jul 27, 2022

caugner marked this pull request as draft December 8, 2022 15:58

github-actions bot removed the 🐌 idle Issues and PRs without recent activity. Flagged for maintainer follow-up. label Dec 8, 2022

github-actions bot added the idle label Jan 8, 2023

github-actions bot removed the idle label Jun 19, 2023

github-actions bot added the idle label Jul 20, 2023

caugner removed their assignment Jul 21, 2023

github-actions bot removed the idle label Jul 21, 2023

github-actions bot added the idle label Aug 21, 2023

github-actions bot removed the idle label Aug 16, 2024

github-actions bot added the idle label Sep 16, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

automatically change language if cookied #3968

automatically change language if cookied #3968

peterbe commented Jun 4, 2021

schalkneethling commented Jun 7, 2021

schalkneethling Jun 7, 2021

peterbe commented Jun 7, 2021

peterbe commented Jun 7, 2021

schalkneethling commented Jun 8, 2021

KartikSoneji commented Jul 14, 2021

peterbe commented Jul 14, 2021

KartikSoneji commented Jul 14, 2021

peterbe commented Jul 14, 2021

KartikSoneji left a comment

KartikSoneji Jul 14, 2021

KartikSoneji Jul 14, 2021

KartikSoneji Jul 15, 2021

KartikSoneji Jul 15, 2021

KartikSoneji Jul 15, 2021

KartikSoneji commented Jul 15, 2021

KartikSoneji commented Jul 17, 2021

peterbe commented Jul 27, 2021

KartikSoneji commented Jul 28, 2021

NiedziolkaMichal commented Mar 3, 2022

schalkneethling commented Mar 3, 2022

github-actions bot commented Jun 20, 2022

	// to the URL you're already don, we're doing this extra check.
	// to the URL you're already on, we're doing this extra check.

	if (value && value.includes("=")) {
	if (value?.includes("=")) {

	if (cookieValue && cookieValue.toLowerCase() !== "en-us") {
	if (cookieValue?.toLowerCase() !== "en-us") {

automatically change language if cookied #3968

Are you sure you want to change the base?

automatically change language if cookied #3968

Conversation

peterbe commented Jun 4, 2021

schalkneethling commented Jun 7, 2021

schalkneethling Jun 7, 2021

Choose a reason for hiding this comment

peterbe commented Jun 7, 2021

peterbe commented Jun 7, 2021

schalkneethling commented Jun 8, 2021

KartikSoneji commented Jul 14, 2021

peterbe commented Jul 14, 2021

KartikSoneji commented Jul 14, 2021

peterbe commented Jul 14, 2021

KartikSoneji left a comment

Choose a reason for hiding this comment

KartikSoneji Jul 14, 2021

Choose a reason for hiding this comment

KartikSoneji Jul 14, 2021

Choose a reason for hiding this comment

KartikSoneji Jul 15, 2021

Choose a reason for hiding this comment

KartikSoneji Jul 15, 2021

Choose a reason for hiding this comment

KartikSoneji Jul 15, 2021

Choose a reason for hiding this comment

KartikSoneji commented Jul 15, 2021

KartikSoneji commented Jul 17, 2021

peterbe commented Jul 27, 2021

KartikSoneji commented Jul 28, 2021

NiedziolkaMichal commented Mar 3, 2022

schalkneethling commented Mar 3, 2022

github-actions bot commented Jun 20, 2022