Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build translations from mdbook #84

Open
wants to merge 18 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions i18n-helpers/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ pulldown-cmark = { version = "0.9.2", default-features = false }
pulldown-cmark-to-cmark = "11.0.0"
regex = "1.9.4"
semver = "1.0.16"
serde = "1.0.130"
serde_json = "1.0.91"

[dev-dependencies]
Expand Down
105 changes: 105 additions & 0 deletions i18n-helpers/src/bin/mdbook-i18n.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
use anyhow::{anyhow, Result};
use mdbook::renderer::RenderContext;
use serde::Deserialize;
use std::collections::BTreeMap;
use std::io;

/// Parameters for the i18n renderer.
///
/// They are set in the `output.i18n` section of the book's `book.toml` file.
#[derive(Deserialize)]
struct I18nConfiguration {
/// A map of language codes to language names.
///
/// ## Example
///
/// ```toml
/// [output.i18n.languages]
/// "en" = "English"
/// "es" = "Spanish (Español)"
/// "ko" = "Korean (한국어)"
/// "pt-BR" = "Brazilian Portuguese (Português do Brasil)"
/// ```
#[serde(default)]
languages: BTreeMap<String, String>,
/// Default language code. It will not be translated.
default_language: Option<String>,

/// Whether to translate all languages or just the selected language in `mdbook.config.book.language`, defaults to false.
#[serde(default)]
translate_all_languages: bool,
/// Whether to move the translations to their renderer's directory, defaults to false.
///
/// By default, translations' output will live in `book/i18n/<language>/<renderer>`.
/// For all renderers in this list, we will move individual translations to `book/<renderer>/<language>`.
#[serde(default)]
move_translations_directories: Vec<String>,
Comment on lines +34 to +36
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're saying here that you'll break the contract defined by mdbook.

I don't want the new renderer to touch files in the output directories of other renderers.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's very convenient with mdbook serve to view the output like it will be on the final site. It doesn't need to be enabled otherwise and by default does nothing

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're saying here that you'll break the contract defined by mdbook.

I don't want the new renderer to touch files in the output directories of other renderers.

It's very convenient with mdbook serve to view the output like it will be on the final site. It doesn't need to be enabled otherwise and by default does nothing

I don't think you addressed my concern: if the code is writing outside of the output directly, then the code is doing something wrong.

I don't doubt that it is convenient: we should just write everything inside of the designated output directory.

Maybe you're not aware, but I often use mdbook build -d /tmp/foo and similar to generate output in non-standard directories. So this renderer must respect the output setting.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default, we don't break the contract but if we enable this, we do. That's also what we're already doing in comprehensive rust, so I don't really see a problem as it is opt-in only.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's also what we're already doing in comprehensive rust, so I don't really see a problem as it is opt-in only.

I hope our mdbook plugins are not breaking this?

Are you talking about the code in publish.yml, then that is quite different: the code there relies on the contract of mdbook which says that it will put things into directory so-and-so. After calling mdbook build, we are of course free to move things around and mess with the book/ directory — we own it then!

This is the main difference between implementing this functionality as a wrapper script (as suggested in #18!) and writing it as a mdbook plugin. As a plugin, you are subjected to the rules of the surrounding system.

It might be a subtle difference, but I believe it's an important one. By following the principle carefully on every level of the stack, we will be able to build bigger "stacks" of software which behaves in a predictable manner when seen as a whole.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script does this:

for language in languages:
    cmd(f"MDBOOK_BOOK__LANGUAGE={language} mdbook build . -d i18n-helpers/{language}")
    for renderer_to_move in move_translations_directories:
        cmd(f"mv i18n-helpers/{language}/{renderer_to_move} {renderer_to_move}/{language}")

Note that the move happens after the mdbook build command ran. Which respects this statement:

After calling mdbook build, we are of course free to move things around and mess with the book/ directory — we own it then!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script does this

Sorry, I don't think I know what script you're referring to here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This renderer I mean

}

fn main() -> Result<()> {
let mut stdin = io::stdin();

// Get the configs
sakex marked this conversation as resolved.
Show resolved Hide resolved
let ctx = RenderContext::from_json(&mut stdin)?;
let i18n_config: I18nConfiguration = ctx
.config
.get_deserialized_opt("output.i18n")?
.ok_or_else(|| anyhow!("No output.i18n config in book.toml"))?;

if !i18n_config.translate_all_languages {
return Ok(());
}

let mut mdbook = mdbook::MDBook::load(&ctx.root)?;
// Overwrite with current values from stdin. This is necessary because mdbook will add data to the config.
mdbook.book = ctx.book.clone();
mdbook.config = ctx.config.clone();
mdbook.root = ctx.root.clone();
sakex marked this conversation as resolved.
Show resolved Hide resolved

let book_config = mdbook
.config
.get_mut("output.i18n")
.ok_or_else(|| anyhow!("No output.i18n config in book.toml"))?;
// Set translate_all_languages to false for nested builds to prevent infinite recursion.
book_config
.as_table_mut()
.ok_or_else(|| anyhow!("output.i18n config in book.toml is not a table"))?
.insert(String::from("translate_all_languages"), false.into());

let output_directory = ctx.destination;
let default_language = &i18n_config.default_language;

for language in i18n_config.languages.keys() {
// Skip current language and default language.
if Some(language) == ctx.config.book.language.as_ref() {
continue;
}
if default_language.as_ref() == Some(language) {
continue;
}
let translation_path = output_directory.join(language);

// Book doesn't implement clone, so we just mutate in place.
mdbook.config.book.language = Some(language.clone());
mdbook.config.book.multilingual = true;
mdbook.config.build.build_dir = translation_path;
mdbook.build()?;
for renderer in &i18n_config.move_translations_directories {
std::fs::create_dir_all(
output_directory
.parent()
.ok_or_else(|| anyhow!("Failed to retrieve parent directory"))?
.join(renderer),
)?;
std::fs::rename(
output_directory.join(language).join(renderer),
output_directory
.parent()
.ok_or_else(|| anyhow!("Failed to retrieve parent directory"))?
.join(renderer)
.join(language),
)?;
}
}
Ok(())
}