A new approach for JavaScript I18N and translator comments

[swissspidy]

I am kicking off this discussion because it came up again as part of the work leveraging React Compiler (#66361), but this is not actually a new problem and has been on my mind for a long time.

For JavaScript i18n we use the same gettext-style API like in PHP, using the __() family of functions. Additional information for polyglots is often added with /* translators */ comments. On platforms like translate.wordpress.org, these are then extracted from the minified JS files using tools like WP-CLI. While this approach works well in PHP, it is not suitable for the modern JS landscape with minifiers, bundlers, and compilers.

An issue early on in the development of Gutenberg was that /* translators */ comments were removed from minified output. Luckily, webpack and Babel had options to preserve comments. Newer tools like React Compiler, esbuild, swc, etc., do not support preserving comments.

Adding /* translators */ comments is fragile because they are often not added directly preceding the __() function call, but often on the line above or even further away. It's an easy mistake to make. In such a case, after minification the comment is often somewhere completely different, causing the string extraction tool to not pick it up. I recently fixed a lot of these issues in #66510.

If we want to use state-of-the-art tooling, we need a better solution for translator comments.

Approach A: use source files for string extraction

If we only extract translator comments from the source files, it doesn't matter if they are missing from the minified files.

Pros:

• This would give the best context for translators, as a minified JS file is pointless as a source
• No code changes to GB or @wordpress/i18n required

Cons:

• Tricky to do for Gutenberg
• Impossible to do for third-party plugins in the plugin directory, as their JS source code is not on SVN (and shouldn't need to be)

Because of the last point, I simply don't think this would be viable.

Approach B: Provide a slightly new JS API for translator comments

We could offer something like __( 'Translate me', 'default', 'I am a translator comment' ) or __( { message: 'Translate me', domain: 'default', comment: 'I am a translator comment' } )

Pros:

• Minimally invasive changes
• Preserves backward compatibility
• Works for GB and third-party plugins

Cons:

• Requires many code changes (though a codemod could be created)

Approach c: Adopt a completely new JS i18n tooling

Something like MessageFormat / FormatJS would not only provide a better solution for translator comments, but also for supporting things like gendered translations.

Pros:

• Future proof (gender support possible)

Cons:

• Requires significant code changes
• Giant lift for the whole ecosystem

---

Personally, right now I think approach B with __( { message: 'Translate me', domain: 'default', comment: 'I am a translator comment' } ) is very reasonable and achievable.

Curious to hear other thoughts.

Pinging relevant i18n folks:

@tyxla @SergeyBiryukov @ocean90 @jsnajdr

[tyxla]

Thanks for opening the discussion, @swissspidy!

Personally, I think Approach B offers the best compromise. After all, this will only be a problem for projects that adopt React 19 and the React Compiler. It could be documented as a breaking change, and a requirement when migrating to using React 19 / React Compiler. Approach A would break many systems in ways that will be difficult to recover, while Approach C will require unsustainable migrations.

Problems like this are exactly why we've been aiming for early adoption of React 19 / React Compiler in Gutenberg, and only when such problems are ironed out, with a proper migration path, we'd open them to the rest of the ecosystem and encourage the migration.

From your Approach B examples, though, I'd lean towards __( 'Translate me', 'default', 'I am a translator comment' ) as that would allow for syntax consistency, should we decide to support the same on the PHP side.

The only thing I'd be cautious about is that if we have multiple uses of the same string with the same textdomain, they would use the same translation, even if they have different comments. I think that makes sense, because if folks want different translations, they could just use the context localization functions. This variety in behavior and localization needs is something that we'll also need to document well going further.

[swissspidy]

Personally, I think Approach B offers the best compromise. After all, this will only be a problem for projects that adopt React 19 and the React Compiler.

Note that it's not only about React Compiler. Even when you simply use something like esbuild you will run into this problem, as comments are not preserved. I bet many plugins/themes already run into this today without realizing.

From your Approach B examples, though, I'd lean towards __( 'Translate me', 'default', 'I am a translator comment' ) as that would allow for syntax consistency, should we decide to support the same on the PHP side.

The only problem there is that this gets complex quickly when using something like _nx():

sprintf(
	_nx(
	'%s+ Million', // singular
	'%s+ Million', // plural
	numberOfMillions, // number
	'Active plugin installations', // context
	'%s: Number of millions.' // comment
	),
	numberOfMillions
);

I don't see any requirement for us to do the same in PHP, except if we want to aim for consistency. Actual comments feel more readable in PHP, the difference between context and comment is clearer.

The only thing I'd be cautious about is that if we have multiple uses of the same string with the same textdomain, they would use the same translation, even if they have different comments.

This is already the case today, FWIW. A bit annoying for translators because then the same string has multiple almost identical comments. It's something we can fix with CI though.

[tyxla]

Note that it's not only about React Compiler. Even when you simply use something like esbuild you will run into this problem, as comments are not preserved. I bet many plugins/themes already run into this today without realizing.

Yes, I agree. I recall tons of hacks we needed to do to preserve those. A similar issue is where build tools mangle & obfuscate function names, but we intentionally ask them to skip the localization function names.

The only problem there is that this gets complex quickly when using something like _nx():

Good point, but from my experience as a GTE for Bulgarian, I've found it very rare to actually need both a context and a translator comment.