Skip to content
This repository has been archived by the owner on Feb 22, 2023. It is now read-only.

Simplify get-translations.js and add error handling and fallbacks #2192

Merged
merged 3 commits into from
Feb 16, 2023

Conversation

dhruvkb
Copy link
Member

@dhruvkb dhruvkb commented Feb 16, 2023

Description

This PR improves upon #2188 in the following ways

  • breaks the code down into async functions, reducing the use of promises
  • breaks the large get-translations.js file into smaller files that work independently
  • adds error handling at critical points
  • adds fallback to separate parallel downloads when bulk download fails
  • adds a flag --require-complete that can be passed to halt builds when the translations are incompletely downloaded.

All in all, this PR adds significant resilience to the multi-pronged approach developed in #2188.

Testing Instructions

  1. Check out the PR.
  2. Try different combinations of the following:
  • Make credentials
    • valid
    • invalid
  • rate limits in src/locales/scripts/axios.js
    • reasonable (~50)
    • unreasonable (~1000)
  • --require-complete flag
    • set
    • unset

Checklist

  • My pull request has a descriptive title (not a vague title like
    Update index.md).
  • My pull request targets the default branch of the repository (main) or
    a parent feature branch.
  • My commit messages follow best practices.
  • My code follows the established code style of the repository.
  • I added or updated tests for the changes I made (if applicable).
  • I added or updated documentation (if applicable).
  • I tried running the project locally and verified that there are no visible
    errors.

Developer Certificate of Origin

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.


Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

@dhruvkb dhruvkb requested review from zackkrida and obulat February 16, 2023 09:06
@dhruvkb dhruvkb requested a review from a team as a code owner February 16, 2023 09:06
@github-actions
Copy link

github-actions bot commented Feb 16, 2023

Storybook and Tailwind configuration previews: Ready

Storybook: https://wordpress.github.io/openverse-frontend/_preview/2192
Tailwind: https://wordpress.github.io/openverse-frontend/_preview/2192/tailwind

Please note that GitHub pages takes a little time to deploy newly pushed code, if the links above don't work or you see old versions, wait 5 minutes and try again.

You can check the GitHub pages deployment action list to see the current status of the deployments.

@openverse-bot openverse-bot added the 🚦 status: awaiting triage Has not been triaged & therefore, not ready for work label Feb 16, 2023
@dhruvkb dhruvkb changed the title Organise .gitignore for i18n files Simplify get-translations.js and add error handling and fallbacks Feb 16, 2023
@dhruvkb dhruvkb added 🟧 priority: high Stalls work on the project or its dependents ✨ goal: improvement Improvement to an existing user-facing feature 🤖 aspect: dx Concerns developers' experience with the codebase and removed 🚦 status: awaiting triage Has not been triaged & therefore, not ready for work labels Feb 16, 2023
@github-actions
Copy link

github-actions bot commented Feb 16, 2023

Size Change: 0 B

Total Size: 908 kB

ℹ️ View Unchanged
Filename Size
./.nuxt/dist/client/237.js 273 B
./.nuxt/dist/client/237.modern.js 277 B
./.nuxt/dist/client/238.js 1.85 kB
./.nuxt/dist/client/app.js 152 kB
./.nuxt/dist/client/app.modern.js 125 kB
./.nuxt/dist/client/commons/app.js 87.7 kB
./.nuxt/dist/client/commons/app.modern.js 78.2 kB
./.nuxt/dist/client/components/loading-icon.js 748 B
./.nuxt/dist/client/components/loading-icon.modern.js 749 B
./.nuxt/dist/client/components/table-sort-icon.js 508 B
./.nuxt/dist/client/components/table-sort-icon.modern.js 513 B
./.nuxt/dist/client/components/v-all-results-grid.js 7.54 kB
./.nuxt/dist/client/components/v-all-results-grid.modern.js 5.05 kB
./.nuxt/dist/client/components/v-audio-cell.js 384 B
./.nuxt/dist/client/components/v-audio-cell.modern.js 393 B
./.nuxt/dist/client/components/v-audio-details.js 2.54 kB
./.nuxt/dist/client/components/v-audio-details.modern.js 1.79 kB
./.nuxt/dist/client/components/v-audio-track-skeleton.js 1.01 kB
./.nuxt/dist/client/components/v-audio-track-skeleton.modern.js 1.02 kB
./.nuxt/dist/client/components/v-audio-track.js 5.27 kB
./.nuxt/dist/client/components/v-audio-track.modern.js 5.22 kB
./.nuxt/dist/client/components/v-back-to-search-results-link.js 538 B
./.nuxt/dist/client/components/v-back-to-search-results-link.modern.js 543 B
./.nuxt/dist/client/components/v-bone.js 685 B
./.nuxt/dist/client/components/v-bone.modern.js 689 B
./.nuxt/dist/client/components/v-box-layout.js 1.23 kB
./.nuxt/dist/client/components/v-box-layout.modern.js 1.23 kB
./.nuxt/dist/client/components/v-content-link.js 1.11 kB
./.nuxt/dist/client/components/v-content-link.modern.js 1.09 kB
./.nuxt/dist/client/components/v-content-page.js 466 B
./.nuxt/dist/client/components/v-content-page.modern.js 471 B
./.nuxt/dist/client/components/v-content-report-button.js 774 B
./.nuxt/dist/client/components/v-content-report-button.modern.js 781 B
./.nuxt/dist/client/components/v-content-report-form.js 6.1 kB
./.nuxt/dist/client/components/v-content-report-form.modern.js 3.58 kB
./.nuxt/dist/client/components/v-content-report-popover.js 1.23 kB
./.nuxt/dist/client/components/v-content-report-popover.modern.js 4.23 kB
./.nuxt/dist/client/components/v-copy-button.js 3.99 kB
./.nuxt/dist/client/components/v-copy-button.modern.js 3.99 kB
./.nuxt/dist/client/components/v-copy-license.js 1 kB
./.nuxt/dist/client/components/v-copy-license.modern.js 1 kB
./.nuxt/dist/client/components/v-copy-license/components/v-error-image/components/v-media-reuse/components/v-search-grid/d219393b.js 9.88 kB
./.nuxt/dist/client/components/v-copy-license/components/v-error-image/components/v-media-reuse/components/v-search-grid/d219393b.modern.js 9.85 kB
./.nuxt/dist/client/components/v-dmca-notice.js 744 B
./.nuxt/dist/client/components/v-dmca-notice.modern.js 751 B
./.nuxt/dist/client/components/v-error-image.js 1.69 kB
./.nuxt/dist/client/components/v-error-image.modern.js 1.68 kB
./.nuxt/dist/client/components/v-error-section.js 373 B
./.nuxt/dist/client/components/v-error-section.modern.js 376 B
./.nuxt/dist/client/components/v-external-search-form.js 2.06 kB
./.nuxt/dist/client/components/v-external-search-form.modern.js 2.05 kB
./.nuxt/dist/client/components/v-external-source-list.js 901 B
./.nuxt/dist/client/components/v-external-source-list.modern.js 899 B
./.nuxt/dist/client/components/v-full-layout.js 1.59 kB
./.nuxt/dist/client/components/v-full-layout.modern.js 1.59 kB
./.nuxt/dist/client/components/v-grid-skeleton.js 1.62 kB
./.nuxt/dist/client/components/v-grid-skeleton.modern.js 1.62 kB
./.nuxt/dist/client/components/v-home-gallery.js 4.8 kB
./.nuxt/dist/client/components/v-home-gallery.modern.js 4.77 kB
./.nuxt/dist/client/components/v-homepage-content.js 1.72 kB
./.nuxt/dist/client/components/v-homepage-content.modern.js 1.69 kB
./.nuxt/dist/client/components/v-image-carousel.js 4.77 kB
./.nuxt/dist/client/components/v-image-carousel.modern.js 4.73 kB
./.nuxt/dist/client/components/v-image-cell-square.js 1.02 kB
./.nuxt/dist/client/components/v-image-cell-square.modern.js 1.02 kB
./.nuxt/dist/client/components/v-image-cell.js 1.45 kB
./.nuxt/dist/client/components/v-image-cell.modern.js 1.44 kB
./.nuxt/dist/client/components/v-image-details.js 2.15 kB
./.nuxt/dist/client/components/v-image-details.modern.js 1.42 kB
./.nuxt/dist/client/components/v-image-grid.js 4.93 kB
./.nuxt/dist/client/components/v-image-grid.modern.js 2.47 kB
./.nuxt/dist/client/components/v-license-tab-panel.js 521 B
./.nuxt/dist/client/components/v-license-tab-panel.modern.js 525 B
./.nuxt/dist/client/components/v-load-more.js 3.16 kB
./.nuxt/dist/client/components/v-load-more.modern.js 684 B
./.nuxt/dist/client/components/v-media-license.js 819 B
./.nuxt/dist/client/components/v-media-license.modern.js 827 B
./.nuxt/dist/client/components/v-media-reuse.js 1.62 kB
./.nuxt/dist/client/components/v-media-reuse.modern.js 1.61 kB
./.nuxt/dist/client/components/v-media-tag.js 428 B
./.nuxt/dist/client/components/v-media-tag.modern.js 434 B
./.nuxt/dist/client/components/v-no-results.js 752 B
./.nuxt/dist/client/components/v-no-results.modern.js 751 B
./.nuxt/dist/client/components/v-old-homepage-content.js 1.88 kB
./.nuxt/dist/client/components/v-old-homepage-content.modern.js 1.85 kB
./.nuxt/dist/client/components/v-radio.js 1.51 kB
./.nuxt/dist/client/components/v-radio.modern.js 1.47 kB
./.nuxt/dist/client/components/v-related-audio.js 1.25 kB
./.nuxt/dist/client/components/v-related-audio.modern.js 1.25 kB
./.nuxt/dist/client/components/v-related-images.js 1.05 kB
./.nuxt/dist/client/components/v-related-images.modern.js 3.03 kB
./.nuxt/dist/client/components/v-report-desc-form.js 965 B
./.nuxt/dist/client/components/v-report-desc-form.modern.js 968 B
./.nuxt/dist/client/components/v-row-layout.js 1.7 kB
./.nuxt/dist/client/components/v-row-layout.modern.js 1.71 kB
./.nuxt/dist/client/components/v-scroll-button.js 813 B
./.nuxt/dist/client/components/v-scroll-button.modern.js 819 B
./.nuxt/dist/client/components/v-search-grid.js 5.87 kB
./.nuxt/dist/client/components/v-search-grid.modern.js 5.8 kB
./.nuxt/dist/client/components/v-search-results-title.js 596 B
./.nuxt/dist/client/components/v-search-results-title.modern.js 599 B
./.nuxt/dist/client/components/v-search-type-radio.js 791 B
./.nuxt/dist/client/components/v-search-type-radio.modern.js 767 B
./.nuxt/dist/client/components/v-server-timeout.js 299 B
./.nuxt/dist/client/components/v-server-timeout.modern.js 304 B
./.nuxt/dist/client/components/v-sketch-fab-viewer.js 3.37 kB
./.nuxt/dist/client/components/v-sketch-fab-viewer.modern.js 894 B
./.nuxt/dist/client/components/v-skip-to-content-container.js 887 B
./.nuxt/dist/client/components/v-skip-to-content-container.modern.js 896 B
./.nuxt/dist/client/components/v-snackbar.js 1.18 kB
./.nuxt/dist/client/components/v-snackbar.modern.js 1.19 kB
./.nuxt/dist/client/components/v-sources-table.js 16.6 kB
./.nuxt/dist/client/components/v-sources-table.modern.js 16.6 kB
./.nuxt/dist/client/components/v-warning-suppressor.js 298 B
./.nuxt/dist/client/components/v-warning-suppressor.modern.js 303 B
./.nuxt/dist/client/pages/about.js 1.52 kB
./.nuxt/dist/client/pages/about.modern.js 1.53 kB
./.nuxt/dist/client/pages/audio/_id/index.js 8.01 kB
./.nuxt/dist/client/pages/audio/_id/index.modern.js 4.87 kB
./.nuxt/dist/client/pages/external-sources.js 1.54 kB
./.nuxt/dist/client/pages/external-sources.modern.js 1.54 kB
./.nuxt/dist/client/pages/feedback.js 1.32 kB
./.nuxt/dist/client/pages/feedback.modern.js 1.32 kB
./.nuxt/dist/client/pages/image/_id/index.js 9.32 kB
./.nuxt/dist/client/pages/image/_id/index.modern.js 7.45 kB
./.nuxt/dist/client/pages/image/_id/report.js 3.59 kB
./.nuxt/dist/client/pages/image/_id/report.modern.js 4.24 kB
./.nuxt/dist/client/pages/index.js 8.65 kB
./.nuxt/dist/client/pages/index.modern.js 8.51 kB
./.nuxt/dist/client/pages/preferences.js 1.23 kB
./.nuxt/dist/client/pages/preferences.modern.js 1.22 kB
./.nuxt/dist/client/pages/privacy.js 995 B
./.nuxt/dist/client/pages/privacy.modern.js 995 B
./.nuxt/dist/client/pages/search-help.js 1.62 kB
./.nuxt/dist/client/pages/search-help.modern.js 1.62 kB
./.nuxt/dist/client/pages/search.js 5.18 kB
./.nuxt/dist/client/pages/search.modern.js 2.67 kB
./.nuxt/dist/client/pages/search/audio.js 6.13 kB
./.nuxt/dist/client/pages/search/audio.modern.js 3.62 kB
./.nuxt/dist/client/pages/search/image.js 596 B
./.nuxt/dist/client/pages/search/image.modern.js 2.73 kB
./.nuxt/dist/client/pages/search/index.js 480 B
./.nuxt/dist/client/pages/search/index.modern.js 485 B
./.nuxt/dist/client/pages/search/model-3d.js 242 B
./.nuxt/dist/client/pages/search/model-3d.modern.js 245 B
./.nuxt/dist/client/pages/search/search-page.types.js 266 B
./.nuxt/dist/client/pages/search/search-page.types.modern.js 271 B
./.nuxt/dist/client/pages/search/video.js 240 B
./.nuxt/dist/client/pages/search/video.modern.js 243 B
./.nuxt/dist/client/pages/sources.js 1.56 kB
./.nuxt/dist/client/pages/sources.modern.js 1.56 kB
./.nuxt/dist/client/runtime.js 2.74 kB
./.nuxt/dist/client/runtime.modern.js 2.74 kB
./.nuxt/dist/client/vendors/app.js 63.8 kB
./.nuxt/dist/client/vendors/app.modern.js 63.1 kB

compressed-size-action

Copy link
Member

@zackkrida zackkrida left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll review more deeply for code quality, but when I switch locales I'm seeing english, and noticed items in the translation files look like this:

      "filters.audio-categories.sound_effect\u0004Sound effects": {
        "0": "Efectos de sonido"
      },

Where the en.json equivalent is:

    "audio-categories": {
      "title": "Audio category",
      "audiobook": "Audiobook",
      "music": "Music",
      "news": "News",
      "podcast": "Podcast",
      "pronunciation": "Pronunciation",
      "sound_effect": "Sound effects",
      "sound": "Sound effects"
    },

@dhruvkb
Copy link
Member Author

dhruvkb commented Feb 16, 2023

I'll debug that immediately. Update: I was missing a step. Fixed in 195bfb3.

@dhruvkb dhruvkb requested a review from zackkrida February 16, 2023 17:19
@github-actions
Copy link

Playwright Failure Test Results

It looks like some of the Playwright tests failed. You can download the Playwright trace
output for both Storybook and Nuxt Playwright tests that have failed at the bottom of this
page, under the "Artifacts" section:

https://github.com/WordPress/openverse-frontend/actions/runs/4197079922

Read more about how to use this artifact here: https://github.com/WordPress/openverse-frontend/blob/main/test/playwright/README.md#debugging

Copy link
Member

@zackkrida zackkrida left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I've tested all cases and it's working well. The logging is all quite clear and useful.

@dhruvkb dhruvkb merged commit 13a95ff into bulk_i18n Feb 16, 2023
@dhruvkb dhruvkb deleted the bulk_i18n_improv branch February 16, 2023 19:08
github-actions bot pushed a commit that referenced this pull request Feb 16, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
🤖 aspect: dx Concerns developers' experience with the codebase ✨ goal: improvement Improvement to an existing user-facing feature 🟧 priority: high Stalls work on the project or its dependents
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants