Improve ability to load messages on the client side, automatic tree-shaking of messages & lazy loading

[amannn]

(repurposed from an older issue that was solely about lazy loading)

Messages can either be consumed in Server- or Client Components.

The former is recommended in the docs, as it has both performance benefits as well as a simpler model in general since there is only a single environment where code executes. However, for highly interactive apps, using messages on the client side is totally fine too. The situation gets a bit hairy when you need to split messages based on usage in Client Components (e.g. by page, or a mix of RSC / RCC).

Ideally, we'd provide more help with handing off translations to Client Components.

Ideas:

1. User-defined namespaces
2. Manually composed namespaces
3. Compiler-based approach

(1) User-defined namespaces

This is already possible (and shown in the docs), but unfortunately not so maintainable. For single components it can be fine however, so when you have many small interactive leaf components, this is quite ok.

(2) Manually composed namespaces

This worked well for the Pages Router and was documented, but unfortunately doesn't work with RSC.

Apollo has a similar situation with static fragments: apollographql/apollo-client-nextjs#27

Since namespaces are only known in Client Components, they should be able to initiate the splitting, but they mustn't have received all possible messages at this point.

This can probably be realized if you expose an endpoint where translations can be fetched from. I've experimented with using Server Actions, but they can't be used during the initial server render in Client Components. SSR would be required though.

(3) Compiler-based approach

This is explored in #1 (see #1 (comment)). Similar to Relay, if the collection of namespaces happens at build time, we could provide this list to Server Components. The experimentation here hasn't progressed far yet, this is a bit further in the future.

---

This situation shows again, that staying in RSC will always be the simplest option. So if that's a choice for your app it's, certainly the best option.

[amannn]

Maybe messages can be rendered exclusively on the server: #1 (comment)

[cdaringe]

Lazy loading is an option, but perhaps ideal is per_locale client side chunks. Rather than a common JS chunk for a page/component, instead locale-based static/* assets may be pre-built for each locale, meaning thin assets and no racing downloads or text flashes!

[mismosmi]

one could use next/dynamic for this:

One would need to wrap each locale file for each component in a NextIntlProvider. Then the component rendering that NextIntlProvider can be lazy loaded usind next/dynamic. This way next will still include the messages in the SSR'd html.

[amannn]

next/dynamic can be used to lazy load client code for a React component. But how would you integrate that with fetching the messages for a particular locale? I think with React 17 you'd still need useEffect for fetching messages lazily. Being able to pass additional messages to the provider would definitely be an interesting addition though.

I hope that future versions of React with Suspense for data fetching will help out here.

[mismosmi]

you could to something like

const i18n = {
  'en-US': dynamic(() => import('./en-US')),
  'en-CA': dynamic(() => import('./en-CA')),
  ...
}

function MyComponent() {
  const { locale } = useRouter()

  const IntlProvider = i18n[locale]

  return <IntlProvider><InnerComponent /></IntlProvider>
}

where en-US.js contains something like

export default function EnUsIntl({ children }) {
  return <NextIntlProvider locale="en-US" messages={{ ... }}>{children}</NextIntlProvider>
}

[mismosmi]

Wrap that pattern in a couple of HOCs and you end up with

// en-US.js
makeI18n({
  MyComponent: {
    title: "bla",
  }
})

// index.js
export default withI18n(() => {
  const t = useTranslation("MyComponent")
  ...
}, { 'en-US': dynamic(() => import('./en-US')), ... })

[amannn]

But that would require you to load messages for unused locales as well, right? Also I think the messages would only be loaded as soon as the module code is evaluated, so you'll probably have a null pointer if the component renders too soon.

[mismosmi]

to my understanding, dynamic does not load anything until the dynamic component is rendered so only the active locale providers would be loaded.

The messages would therefore be loaded whenever the component renders for the first time (as they are part of the lazy loaded modules) and that first time is either during the SSR step or after immediately after client side navigation (I'm not sure though if next/dynamic deals with react's suspense the same way React.lazy does)

[amannn]

dynamic does not load anything until the dynamic component is rendered

I also think that's true, but it assumes that what you're wrapping with dynamic is a component. It will either return null or a loading placeholder until the module is loaded. So it's different from suspense in that it has to render immediately and doesn't suspend until all lazy loaded modules are available. That's at least my understanding.

[mismosmi]

Yeah you might have to manually handle the loading state. Blank should be fine for most cases though as nextjs

• renders dynamic components during SSR so SEO is fine
• is pretty smart about pre-loading stuff (which might mean it will pre load all locales in most cases, I'd have to expirement with it to figure that out)

[amannn]

During the Next.js Conf this week, the Vercel team has shown a brief demo on how data could be fetched at a component-level at arbitrary levels in the tree. This approach would come in really handy for lazy loading messages, potentially from a provided API route.

This approach would also be compatible with the now suggested approach for splitting messages by namespace:

next-intl/packages/example/src/pages/index.tsx

Lines 24 to 29 in a168021

	// The namespaces can be generated based on used components. `PageLayout` in
	// turn requires messages for `Navigation` and therefore a recursive list of
	// namespaces is created dynamically, where the owner of a component doesn't
	// have to know which nested components are rendered. Note that this approach
	// is limited to components which are not lazy loaded.
	Index.messages = ['Index', ...PageLayout.messages];

Components would still define their namespaces, but lazy loaded components would be wrapped in a provider that fetches the relevant messages and adds them to the provider.

An open question is however how this would integrate with server/client components. The easiest case is if only server components need messages, but I'm not sure yet how the transition to client components could work if they need messages. Potentially also client-only components could use the new data fetching wrapper and trigger the nearest suspense boundary.

[mismosmi]

I assume it would be possible to use a server component to fetch some i18n data and then a isomorphic component to provide it as context

// locale/Common.server.tsx
export const CommonMessages = ({ children }) => {
  const { locale } = useRouter()
  const messages = JSON.parse(fs.readFileSync(`./${locale}.json`))
  return <I18nProvider messages={{ common: messages }}>{children}</I18nProvider>
}

[mismosmi]

Then you might wrap any component that needs the 'common' messages within this component and it would be passed the right messages like provideI18n(CommonMessages, ...)(MyComponent).
Then one would only have to take care that each set of messages is only transmitted once (first time the message provider is rendered), that should be doable though.

[mismosmi]

How the I18nProvider works depends a bit on how the react team will implement the context api for server components

[amannn]

Yep, exactly – that's similar to what I had in mind as well. I first thought about having an endpoint that provides messages, but if we can transfer context from a server to a client component then using a server side component could be really interesting. Thanks for your thoughts!

[bryanprimus]

do we already have a guide on how to implement this inside the app dir?

[mismosmi]

Uuuuh good question, this might actually get interesting again now…

[amannn]

@bryanltobing In Server Components, there's no need for lazy loading messages, since the performance will not be impacted by loading the translations for your whole app. Are you interested in using translations in Client Components?

The current best practice is described here: https://next-intl-docs.vercel.app/docs/next-13/server-components#using-translations-in-client-components

Apart from that, if you need "real lazy loading" this can currently be implemented in user land by fetching messages from a Client Component and then passing them to NextIntlClientProvider.

Related to this, there was a little bit of movement in regard to automatic tree-shaking for Client Components recently: #1 (comment)

That's all we have for now!

[bryanprimus]

my main concern is in the server component. Let's say I have 1 en.json file containing more than 2000 translations.
I want to use translation In RootLayout and the only translation copy I need is for the Nav namespace

en.json

{
  "Nav": {
    "home": "Home",
    "about": "About",
  }
  "Home": {
    "Welcome": "Welcome"
  },
  ...otherTranslationsCopy
  
}

Does it fetch the whole translation when it only needs Nav in this case? I'm worried about the server's performance.
I think this is also something that happens in the client component and ends up with a bloated bundle because it fetches the whole translation.

[amannn]

The messages that are loaded server side depend on what you load in i18n.ts. On the client side it depends on what you pass to NextIntlClientProvider. You can optimize both places as necessary if you're worried that too many messages are provided.

[kneza23]

We want to use useTranslations hook on both Client and Server components.
Passing down props from Server to Client is not ideal as we can have multiple levels down, so it really can become a pain to do that.

What is the main downside if we setup our Next.js app both with (i18n.ts, next.config.js) Server config and (NextIntlClientProvider on root layout) Client config?

Both pieces of code target external API to fetch translations and use cache and next: { revalidate: 60 } so that we do not fetch every time.

Is this a big performance hit? Can you please explain what are the downsides of doing this, instead of using just Server Components for translations or do the wrapping with NextIntlClientProvider for each Client Component that needs some translation namespace.

[amannn]

That's a great question @kneza23!

I'm currently working on a major docs update and have added a page that should hopefully answer this: Internationalization of Server & Client Components in Next.js 13.

Feel free to have a look! If you have any feedback, please let me know in #351

[kneza23]

Docs update looks good, keep up the great work ,but sadly Internationalization of Server & Client Components in Next.js 13 still does not answer my question what to do if you have translations in both Server and Client components.

[amannn]

Hmm, which aspects remain unanswered for you?

The page is structured into three sections:

1. Passing translations to Client Components (recommended, if possible)
2. Using interactive state in translations (discusses staying in server land if possible, but with a note about using the provider)
3. Highly dynamic apps (discusses passing all messages to the client side)

The performance aspects are discussed at the very top, depending on which route you take you're making a tradeoff. How big the tradeoff is (and if it's justified anyway) really depends on your situation, so you should measure this yourself if your app is very performance sensitive.

Let me know if this helps!

[kneza23]

Sry if i did not make myself more clear.

I will try to explain.

We have an app that can be considered Highly dynamic apps from your docs, but we also have some server components sprinkled here and there that do not have access then the translations, so we need to have both environments capable of accessing the translations.

I don't see that example in the docs. Only the part where you advise to wrap a client component in Provider and then pick the namespace needed for children to use it.

But our components are 80% client side, so that solution also does not make sense in our situation.

Also prop drilling from server to client can be really cumbersome in that case.

So my solution is to have just one NextIntlClientProvider at the root layout for ALL the Client components and also have i18n.ts, next.config.js Server config for ALL the Server components, so every component regarding of the type/enviroment (Server/Client) can get the translations with useTranslations.

So my question is. What are the repercussions of this, and downsides, as i'm not familiar with technical implementation details of the library?

I cant find it in the docs, there is no example where we can make it so we get access to translations on both Client and Server without 1. option (prop drilling) 2. option (have lots of different NextIntlClientProvider files for each client, and 3. option is just all Client.

Sry for long post :)

[kneza23]

Some code snippets

This is our root layout for Client components:

export default async function RootLayout({
  children,
  params: { locale },
}: Props) {
  let messages;
  try {
    messages = await fetchTranslationsFromSomeAPI(locale);
  } catch (error) {
    notFound();
  }

  return (
    <html lang={locale}>
      <body>
        <NextIntlClientProvider locale={locale} messages={messages}>
          <Providers>{children}</Providers>
        </NextIntlClientProvider>
      </body>
    </html>
  );
}

This is our next.config.js

const withNextIntl = require("next-intl/plugin")(
  // This is the default (also the `src` folder is supported out of the box)
  "./i18n.ts"
);

this is our 18n.ts

export default getRequestConfig(async ({ locale }) => ({
  messages: await fetchTranslationsFromSomeAPI(locale),
}));

our middleware.tsx with next-auth combined

const intlMiddleware = createIntlMiddleware({
  locales,
  defaultLocale: "en",
});

const authMiddleware = withAuth(
  (req) => intlMiddleware(req),
  {
    callbacks: {
      authorized: ({ req, token }) => {
        return token != null;
      },
    },
    pages: {
      signIn: "/signin",
    },
  }
);

export default function middleware(req: NextRequest) {
  const publicPathnameRegex = RegExp(
    `^(/(${locales.join("|")}))?(${publicPages.join("|")})?/?$`,
    "i"
  );
  const isPublicPage = publicPathnameRegex.test(req.nextUrl.pathname);

  if (isPublicPage) {
    return intlMiddleware(req);
  } else {
    return (authMiddleware as any)(req);
  }
}

export const config = {
  // Skip all paths that should not be internationalized
  matcher: ["/((?!api|_next|.*\\..*).*)"],
};

Fetch requests are cached, and revalidated after 60 sec.

So with this, every component has access to translations.

[amannn]

Right, I understand! Your code looks fine to me.

My main point with the page I've linked to is to explain the tradeoffs for different techniques that allow handling translations in Client Components. I've reworked the page again a bit, hopefully to illustrate the order of recommendation a bit clearer, but also to use less negative wording for providing messages to Client Components, since there are legitimate use cases for this. I guess that sounded too scary before and wasn't really appropriate.

I've also added a small paragraph to the global configuration page that briefly touches on how i18n.ts works internally:

The configuration object is created once for each request by internally using React's cache. The first component to use internationalization will call the function defined with getRequestConfig.

Does that help? Once Server Components support is out of beta, the tab order on this page will be reversed to be i18n.ts-first.

To quickly answer your question from above, the "additional" usage via i18n.ts in your app is totally fine. If fetchTranslationsFromSomeAPI is integrated with React cache, then the request will be deduplicated anyway. I'm just trying to align the docs here, so this is clearer for the next person with this question in mind.

If you could help to provide feedback if there's still something missing in the docs, that would be really helpful. Thanks!

Side note: I'm considering adding useMessages that could help to pass messages originally received via i18n.ts to NextIntlClientProvider since there are legitimate cases for this and to take away any worry if fetching messages twice is a performance concern. Do you think that would be helpful in your case?

[kneza23]

Tnx for feedback, useMessages would be useful 👍

[amannn]

I just stumbled over the TanStack Query nextjs-suspense-streaming example (tweet).

Based on the implementation I'm wondering if we could:

1. Expose an endpoint that returns requested messages by namespace
2. Same as the example above, the endpoint can be called during render and we can use suspense to wait for data to be available
3. Use the messages during the server-side render
4. Prime a cache of the messages for the client side
5. On the client side, everything is already fetched and we can use the messages

Assumption to be validated: Calling an endpoint on the server from the server doesn't introduce significant delay

The advantage would be that we don't need any compiler magic.

While it can be an advantage to use this for lazy-loaded client components, it would introduce a waterfall if we have to fetch messages for every component one by one. The RSC model fits much better here.

These are just raw thoughts, leaving them here for later reference.

[mismosmi]

Check out server actions.
Those will be handled as an endpoint when called on the client but on the server they're just regular async functions.

[amannn]

Thanks for chiming in! I'd be really interested in using server actions, but it seems like currently they have a restriction that they mustn't be called during a render on the server side.

E.g. this throws:

'use client';
import {use} from 'react';
import {getValue} from './actions';

function Component() {
  use(getValue());
}

[mismosmi]

Mhh I see. They're in beta anyway.

But might be worth waiting for, seems like a really good fit 🙃

[amannn]

That's very true! 🙂

[Yhozen]

@kneza23 do you perceive any significant performance drawback when getting messages from a API?

[salos1982]

HI, I also have suggestion for this topic. I would like to have possibility to merge server messages int one. Now it overwrites previous messages and I have to copy of messages in the page.

My example: Root layout

export default async function RootLayout({
  children,
  params: { locale },
}: Props) {
  let messages = await getMessage();

  return (
    <html lang={locale}>
      <body>
        <NextIntlClientProvider locale={locale} messages={pick(messages, "footer_header")}>
          <Header/>
          {children}
          <Footer/>
        </NextIntlClientProvider>
      </body>
    </html>
  );
}

Page.tsx

export default async function ContentPage(props: NextPageProps) {
  const locale = params.locale as string;
  return (
      <NextIntlClientProvider locale={locale} messages={pick(messages, "content")}>
        <main>
          <ContentData />
        </main>
      </NextIntlClientProvider>
)
}

I want to have possibility to merge namespaces. So content component can use translations from "footer_header" namespace. For the moment I need to add namespace "footer_header" for content page messages. And I have 2 copies of "footer_header" namespace on the page.

[kneza23]

@kneza23 do you perceive any significant performance drawback when getting messages from a API?

Did not profiled it or measure the performance but i think it is fine because we use revalidate to cache it so it does not fetch it every time

[amannn]

I'll close this issue in favor of #1 as there's a lot of overlap.