Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(build): copy images that are only in default locale to l10n folder #7917

Merged
merged 11 commits into from
Jul 17, 2023
23 changes: 7 additions & 16 deletions build/check-images.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@ export function checkImageReferences(
options,
{ url, rawContent }
) {
caugner marked this conversation as resolved.
Show resolved Hide resolved
const filePaths = new Set();
// filePaths is a map of basename to full path
const filePaths = new Map<string, string>();
caugner marked this conversation as resolved.
Show resolved Hide resolved

const checkImages = options.flawLevels.get("images") !== FLAW_LEVELS.IGNORE;

Expand Down Expand Up @@ -144,9 +145,7 @@ export function checkImageReferences(
// insensitively.

// What follows uses the same algorithm as FileAttachment.findByURLWithFallback
// but only adds a filePath if it exists for the DEFAULT_LOCALE
const filePath = FileAttachment.findByURL(finalSrc);
let enUSFallback = false;
let filePath = FileAttachment.findByURL(finalSrc);
caugner marked this conversation as resolved.
Show resolved Hide resolved
if (
!filePath &&
doc.locale !== DEFAULT_LOCALE &&
Expand All @@ -156,23 +155,15 @@ export function checkImageReferences(
new RegExp(`^/${doc.locale}/`, "i"),
`/${DEFAULT_LOCALE}/`
);
if (FileAttachment.findByURL(enUSFinalSrc)) {
// Use the en-US src instead
finalSrc = enUSFinalSrc;
// Note that this `<img src="...">` value can work if you use the
// en-US equivalent URL instead.
enUSFallback = true;
}
// try to find the attachment in the default locale
filePath = FileAttachment.findByURL(enUSFinalSrc);
}
if (filePath) {
filePaths.add(filePath);
filePaths.set(path.basename(filePath), filePath);
}

if (checkImages) {
if (enUSFallback) {
// If it worked by switching to the en-US src, don't do anything more.
// Do nothing! I.e. don't try to perfect the spelling.
} else if (!filePath) {
if (!filePath) {
// E.g. <img src="doesnotexist.png"
addImageFlaw(img, src, {
explanation:
Expand Down
4 changes: 2 additions & 2 deletions build/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -228,10 +228,10 @@ async function buildDocuments(
fs.writeFileSync(liveSamplePath, html);
}

for (const filePath of fileAttachments) {
for (const [basename, filePath] of fileAttachments) {
caugner marked this conversation as resolved.
Show resolved Hide resolved
// We *could* use symlinks instead. But, there's no point :)
// Yes, a symlink is less disk I/O but it's nominal.
fs.copyFileSync(filePath, path.join(outPath, path.basename(filePath)));
fs.copyFileSync(filePath, path.join(outPath, basename));
}

// Collect active documents' slugs to be used in sitemap building and
Expand Down
28 changes: 24 additions & 4 deletions build/index.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import fs from "node:fs";
import path from "node:path";

import chalk from "chalk";
Expand All @@ -12,11 +13,11 @@ import {
} from "../kumascript/src/errors.js";

import { Doc, WebFeature, WebFeatureStatus } from "../libs/types/document.js";
import { Document, execGit } from "../content/index.js";
import { Document, execGit, slugToFolder } from "../content/index.js";
import { CONTENT_ROOT, REPOSITORY_URLS } from "../libs/env/index.js";
import * as kumascript from "../kumascript/index.js";

import { FLAW_LEVELS } from "../libs/constants/index.js";
import { DEFAULT_LOCALE, FLAW_LEVELS } from "../libs/constants/index.js";
import { extractSections } from "./extract-sections.js";
import { extractSidebar } from "./extract-sidebar.js";
import { extractSummary } from "./extract-summary.js";
Expand Down Expand Up @@ -159,7 +160,7 @@ function injectSource(doc, document, metadata) {
export interface BuiltDocument {
doc: Doc;
liveSamples: any;
fileAttachments: any;
fileAttachments: Map<string, string>;
caugner marked this conversation as resolved.
Show resolved Hide resolved
source?: {
github_url: string;
};
Expand Down Expand Up @@ -394,9 +395,28 @@ export async function buildDocument(
// it returns which images it checked. But we'll need to complement any
// other images in the folder.
getAdjacentFileAttachments(path.dirname(document.fileInfo.path)).forEach(
(fp) => fileAttachments.add(fp)
(fp) => fileAttachments.set(path.basename(fp), fp)
);

if (doc.locale !== DEFAULT_LOCALE) {
// If it's not the default locale, we need to add the images
// from the default locale too.
const defaultLocaleDir = path.join(
CONTENT_ROOT,
DEFAULT_LOCALE.toLowerCase(),
slugToFolder(metadata.slug)
);

if (fs.existsSync(defaultLocaleDir)) {
getAdjacentFileAttachments(defaultLocaleDir).forEach((fp) => {
const basename = path.basename(fp);
if (!fileAttachments.has(basename)) {
fileAttachments.set(basename, fp);
}
});
}
}

// Check the img tags for possible flaws and possible build-time rewrites
checkImageWidths(doc, $, options, document);

Expand Down
12 changes: 10 additions & 2 deletions testing/tests/index.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -214,7 +214,7 @@ test("French translation using English front-matter bits", () => {
expect(bcd.value.query).toBe("javascript.builtins.Array.toLocaleString");
});

test("content built zh-CN page for hreflang tag testing", () => {
test("content built zh-CN page for hreflang tag and copying image testing", () => {
const builtFolder = path.join(buildRoot, "zh-cn", "docs", "web", "foo");
const jsonFile = path.join(builtFolder, "index.json");
expect(fs.existsSync(jsonFile)).toBeTruthy();
Expand Down Expand Up @@ -243,6 +243,11 @@ test("content built zh-CN page for hreflang tag testing", () => {
expect($('meta[property="og:title"]').attr("content")).toBe(
"<foo>: 测试网页 | MDN"
);

// The image should be in the built folder,
// even though it's not referenced in the translated content.
const imageFile = path.join(builtFolder, "screenshot.png");
expect(fs.existsSync(imageFile)).toBeTruthy();
});

test("content built zh-TW page with en-US fallback image", () => {
Expand Down Expand Up @@ -273,8 +278,11 @@ test("content built zh-TW page with en-US fallback image", () => {
"<foo>: 測試網頁 | MDN"
);
expect($("#content img").attr("src")).toBe(
"/en-US/docs/Web/Foo/screenshot.png"
"/zh-TW/docs/Web/Foo/screenshot.png"
);

const imageFile = path.join(builtFolder, "screenshot.png");
expect(fs.existsSync(imageFile)).toBeTruthy();
});

test("content built French Embeddable page", () => {
Expand Down
11 changes: 11 additions & 0 deletions testing/translated-content/files/zh-cn/web/foo/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,14 @@ translation_of: Web/Foo

This is a test page for hreflang tag testing. When zh-TW locale is also
available, this page should not have duplicate hreflang tags.

<!--
copying images test, see
https://github.com/mdn/yari/issues/5652
-->

This page is also used for test that copying all the images from the default
locale to the translated locale.

The images in default locale should still be copied to translated locale, even
if we have not reference the images that are only in the default locale.