Skip to content
This repository has been archived by the owner on Feb 22, 2023. It is now read-only.

Reduce api token logging #1823

Merged
merged 2 commits into from
Sep 26, 2022
Merged

Reduce api token logging #1823

merged 2 commits into from
Sep 26, 2022

Conversation

zackkrida
Copy link
Member

Description

We've had a pretty large increase to our log storage costs in production. This PR tries to remove some of the logs from the API token process. I'm opening to keeping a subset of these or whatever the minimum amount of information we need to debug problems with this is.

Testing Instructions

Checklist

  • My pull request has a descriptive title (not a vague title like Update index.md).
  • My pull request targets the default branch of the repository (main) or a parent feature branch.
  • My commit messages follow best practices.
  • My code follows the established code style of the repository.
  • I added or updated tests for the changes I made (if applicable).
  • I added or updated documentation (if applicable).
  • I tried running the project locally and verified that there are no visible errors.

Developer Certificate of Origin

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.


Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

@zackkrida zackkrida requested a review from a team as a code owner September 16, 2022 19:59
@github-actions
Copy link

github-actions bot commented Sep 16, 2022

Storybook and Tailwind configuration previews: Ready

Storybook: https://wordpress.github.io/openverse-frontend/_preview/1823
Tailwind: https://wordpress.github.io/openverse-frontend/_preview/1823/tailwind

Please note that GitHub pages takes a little time to deploy newly pushed code, if the links above don't work or you see old versions, wait 5 minutes and try again.

You can check the GitHub pages deployment action list to see the current status of the deployments.

@openverse-bot openverse-bot added the 🚦 status: awaiting triage Has not been triaged & therefore, not ready for work label Sep 16, 2022
@github-actions
Copy link

github-actions bot commented Sep 16, 2022

Size Change: +5.71 kB (+1%)

Total Size: 798 kB

Filename Size Change
./.nuxt/dist/client/230.js 0 B -273 B (removed) 🏆
./.nuxt/dist/client/230.modern.js 0 B -276 B (removed) 🏆
./.nuxt/dist/client/231.js 0 B -1.85 kB (removed) 🏆
./.nuxt/dist/client/app.js 120 kB +3.8 kB (+3%)
./.nuxt/dist/client/app.modern.js 111 kB +2.89 kB (+3%)
./.nuxt/dist/client/commons/app.js 88.2 kB +24 B (0%)
./.nuxt/dist/client/commons/app.modern.js 77 kB -41 B (0%)
./.nuxt/dist/client/components/v-copy-license/components/v-error-image/components/v-media-reuse/components/v-search-grid/09090664.js 9.64 kB +105 B (+1%)
./.nuxt/dist/client/components/v-copy-license/components/v-error-image/components/v-media-reuse/components/v-search-grid/09090664.modern.js 9.62 kB +112 B (+1%)
./.nuxt/dist/client/components/v-desktop-page-menu.js 0 B -527 B (removed) 🏆
./.nuxt/dist/client/components/v-desktop-page-menu.modern.js 0 B -529 B (removed) 🏆
./.nuxt/dist/client/components/v-mobile-page-menu.js 0 B -545 B (removed) 🏆
./.nuxt/dist/client/components/v-mobile-page-menu.modern.js 0 B -545 B (removed) 🏆
./.nuxt/dist/client/components/v-search-grid.modern.js 5.39 kB -11 B (0%)
./.nuxt/dist/client/components/v-sources-table.js 14.8 kB +87 B (+1%)
./.nuxt/dist/client/components/v-sources-table.modern.js 14.8 kB +85 B (+1%)
./.nuxt/dist/client/pages/index.js 5.05 kB -182 B (-3%)
./.nuxt/dist/client/pages/index.modern.js 4.94 kB -180 B (-4%)
./.nuxt/dist/client/pages/search.js 2.73 kB +15 B (+1%)
./.nuxt/dist/client/pages/search.modern.js 2.58 kB +12 B (0%)
./.nuxt/dist/client/runtime.js 2.66 kB -55 B (-2%)
./.nuxt/dist/client/runtime.modern.js 2.66 kB -56 B (-2%)
./.nuxt/dist/client/vendors/app.modern.js 47 kB -168 B (0%)
./.nuxt/dist/client/221.js 272 B +272 B (new file) 🆕
./.nuxt/dist/client/221.modern.js 276 B +276 B (new file) 🆕
./.nuxt/dist/client/222.js 1.85 kB +1.85 kB (new file) 🆕
./.nuxt/dist/client/components/v-content-settings-button.js 690 B +690 B (new file) 🆕
./.nuxt/dist/client/components/v-content-settings-button.modern.js 695 B +695 B (new file) 🆕
ℹ️ View Unchanged
Filename Size Change
./.nuxt/dist/client/components/loading-icon.js 747 B 0 B
./.nuxt/dist/client/components/loading-icon.modern.js 753 B +2 B (0%)
./.nuxt/dist/client/components/table-sort-icon.js 508 B -1 B (0%)
./.nuxt/dist/client/components/table-sort-icon.modern.js 512 B 0 B
./.nuxt/dist/client/components/v-all-results-grid.js 2.78 kB +2 B (0%)
./.nuxt/dist/client/components/v-all-results-grid.modern.js 2.75 kB -3 B (0%)
./.nuxt/dist/client/components/v-all-results-grid/pages/search/audio/pages/search/index.js 3.02 kB 0 B
./.nuxt/dist/client/components/v-all-results-grid/pages/search/audio/pages/search/index.modern.js 2.92 kB -1 B (0%)
./.nuxt/dist/client/components/v-audio-cell.js 357 B 0 B
./.nuxt/dist/client/components/v-audio-cell.modern.js 360 B -1 B (0%)
./.nuxt/dist/client/components/v-audio-details.js 1.81 kB -2 B (0%)
./.nuxt/dist/client/components/v-audio-details.modern.js 1.79 kB -2 B (0%)
./.nuxt/dist/client/components/v-audio-track-skeleton.js 1.01 kB 0 B
./.nuxt/dist/client/components/v-audio-track-skeleton.modern.js 1.02 kB +1 B (0%)
./.nuxt/dist/client/components/v-audio-track.js 5.11 kB +5 B (0%)
./.nuxt/dist/client/components/v-audio-track.modern.js 5.05 kB -3 B (0%)
./.nuxt/dist/client/components/v-back-to-search-results-link.js 571 B 0 B
./.nuxt/dist/client/components/v-back-to-search-results-link.modern.js 574 B +1 B (0%)
./.nuxt/dist/client/components/v-bone.js 684 B -2 B (0%)
./.nuxt/dist/client/components/v-bone.modern.js 690 B +2 B (0%)
./.nuxt/dist/client/components/v-box-layout.js 1.21 kB +3 B (0%)
./.nuxt/dist/client/components/v-box-layout.modern.js 1.2 kB +2 B (0%)
./.nuxt/dist/client/components/v-content-link.js 1.13 kB +1 B (0%)
./.nuxt/dist/client/components/v-content-link.modern.js 1.11 kB 0 B
./.nuxt/dist/client/components/v-content-page.js 466 B 0 B
./.nuxt/dist/client/components/v-content-page.modern.js 470 B 0 B
./.nuxt/dist/client/components/v-content-report-button.js 765 B -1 B (0%)
./.nuxt/dist/client/components/v-content-report-button.modern.js 772 B 0 B
./.nuxt/dist/client/components/v-content-report-form.js 3.77 kB +3 B (0%)
./.nuxt/dist/client/components/v-content-report-form.modern.js 3.58 kB +1 B (0%)
./.nuxt/dist/client/components/v-content-report-popover.js 4.45 kB +1 B (0%)
./.nuxt/dist/client/components/v-content-report-popover.modern.js 4.26 kB 0 B
./.nuxt/dist/client/components/v-copy-button.js 3.97 kB +6 B (0%)
./.nuxt/dist/client/components/v-copy-button.modern.js 3.98 kB -1 B (0%)
./.nuxt/dist/client/components/v-copy-license.js 2.95 kB -3 B (0%)
./.nuxt/dist/client/components/v-copy-license.modern.js 2.93 kB +1 B (0%)
./.nuxt/dist/client/components/v-dmca-notice.js 749 B +6 B (+1%)
./.nuxt/dist/client/components/v-dmca-notice.modern.js 754 B 0 B
./.nuxt/dist/client/components/v-error-image.js 1.69 kB +2 B (0%)
./.nuxt/dist/client/components/v-error-image.modern.js 1.68 kB -2 B (0%)
./.nuxt/dist/client/components/v-error-section.js 371 B 0 B
./.nuxt/dist/client/components/v-error-section.modern.js 376 B 0 B
./.nuxt/dist/client/components/v-full-layout.js 1.48 kB +1 B (0%)
./.nuxt/dist/client/components/v-full-layout.modern.js 1.48 kB +1 B (0%)
./.nuxt/dist/client/components/v-grid-skeleton.js 1.61 kB -1 B (0%)
./.nuxt/dist/client/components/v-grid-skeleton.modern.js 1.61 kB 0 B
./.nuxt/dist/client/components/v-image-cell-square.js 1.01 kB -1 B (0%)
./.nuxt/dist/client/components/v-image-cell-square.modern.js 1.02 kB 0 B
./.nuxt/dist/client/components/v-image-cell.js 1.42 kB -1 B (0%)
./.nuxt/dist/client/components/v-image-cell.modern.js 1.42 kB +1 B (0%)
./.nuxt/dist/client/components/v-image-details.js 1.44 kB -1 B (0%)
./.nuxt/dist/client/components/v-image-details.modern.js 1.43 kB +1 B (0%)
./.nuxt/dist/client/components/v-image-grid.js 2.53 kB +5 B (0%)
./.nuxt/dist/client/components/v-image-grid.modern.js 2.42 kB +3 B (0%)
./.nuxt/dist/client/components/v-license-tab-panel.js 921 B -1 B (0%)
./.nuxt/dist/client/components/v-license-tab-panel.modern.js 917 B 0 B
./.nuxt/dist/client/components/v-load-more.js 787 B -1 B (0%)
./.nuxt/dist/client/components/v-load-more.modern.js 684 B 0 B
./.nuxt/dist/client/components/v-media-license.js 798 B +1 B (0%)
./.nuxt/dist/client/components/v-media-license.modern.js 808 B 0 B
./.nuxt/dist/client/components/v-media-reuse.js 1.08 kB 0 B
./.nuxt/dist/client/components/v-media-reuse.modern.js 1.08 kB -4 B (0%)
./.nuxt/dist/client/components/v-media-reuse/pages/image/_id.js 2.95 kB -2 B (0%)
./.nuxt/dist/client/components/v-media-reuse/pages/image/_id.modern.js 2.93 kB +2 B (0%)
./.nuxt/dist/client/components/v-media-tag.js 430 B +1 B (0%)
./.nuxt/dist/client/components/v-media-tag.modern.js 434 B 0 B
./.nuxt/dist/client/components/v-meta-search-form.js 3.1 kB -5 B (0%)
./.nuxt/dist/client/components/v-meta-search-form.modern.js 3.07 kB -6 B (0%)
./.nuxt/dist/client/components/v-meta-source-list.js 2.56 kB -1 B (0%)
./.nuxt/dist/client/components/v-meta-source-list.modern.js 2.54 kB -2 B (0%)
./.nuxt/dist/client/components/v-no-results.js 2.76 kB -1 B (0%)
./.nuxt/dist/client/components/v-no-results.modern.js 2.73 kB -3 B (0%)
./.nuxt/dist/client/components/v-radio.js 1.51 kB -2 B (0%)
./.nuxt/dist/client/components/v-radio.modern.js 1.47 kB 0 B
./.nuxt/dist/client/components/v-related-audio.js 1.23 kB -3 B (0%)
./.nuxt/dist/client/components/v-related-audio.modern.js 1.24 kB 0 B
./.nuxt/dist/client/components/v-related-images.js 3.09 kB +1 B (0%)
./.nuxt/dist/client/components/v-related-images.modern.js 2.98 kB +4 B (0%)
./.nuxt/dist/client/components/v-report-desc-form.js 965 B -1 B (0%)
./.nuxt/dist/client/components/v-report-desc-form.modern.js 965 B 0 B
./.nuxt/dist/client/components/v-row-layout.js 1.68 kB 0 B
./.nuxt/dist/client/components/v-row-layout.modern.js 1.69 kB +1 B (0%)
./.nuxt/dist/client/components/v-scroll-button.js 813 B +8 B (+1%)
./.nuxt/dist/client/components/v-scroll-button.modern.js 819 B +9 B (+1%)
./.nuxt/dist/client/components/v-search-grid.js 5.44 kB -5 B (0%)
./.nuxt/dist/client/components/v-search-results-title.js 657 B 0 B
./.nuxt/dist/client/components/v-search-results-title.modern.js 652 B 0 B
./.nuxt/dist/client/components/v-search-type-radio.js 829 B -1 B (0%)
./.nuxt/dist/client/components/v-search-type-radio.modern.js 820 B +1 B (0%)
./.nuxt/dist/client/components/v-server-timeout.js 298 B 0 B
./.nuxt/dist/client/components/v-server-timeout.modern.js 303 B 0 B
./.nuxt/dist/client/components/v-sketch-fab-viewer.js 996 B 0 B
./.nuxt/dist/client/components/v-sketch-fab-viewer.modern.js 894 B -1 B (0%)
./.nuxt/dist/client/components/v-skip-to-content-container.js 889 B -1 B (0%)
./.nuxt/dist/client/components/v-skip-to-content-container.modern.js 894 B +1 B (0%)
./.nuxt/dist/client/components/v-snackbar.js 1.18 kB 0 B
./.nuxt/dist/client/components/v-snackbar.modern.js 1.19 kB -1 B (0%)
./.nuxt/dist/client/components/v-tab-panel.js 689 B 0 B
./.nuxt/dist/client/components/v-tab-panel.modern.js 684 B +1 B (0%)
./.nuxt/dist/client/components/v-tab.js 1.73 kB +1 B (0%)
./.nuxt/dist/client/components/v-tab.modern.js 1.72 kB +8 B (0%)
./.nuxt/dist/client/components/v-tabs.js 779 B 0 B
./.nuxt/dist/client/components/v-tabs.modern.js 764 B 0 B
./.nuxt/dist/client/components/v-warning-suppressor.js 298 B 0 B
./.nuxt/dist/client/components/v-warning-suppressor.modern.js 303 B 0 B
./.nuxt/dist/client/pages/about.js 1.05 kB +1 B (0%)
./.nuxt/dist/client/pages/about.modern.js 1.06 kB +1 B (0%)
./.nuxt/dist/client/pages/audio/_id.js 6.63 kB -1 B (0%)
./.nuxt/dist/client/pages/audio/_id.modern.js 6.46 kB -7 B (0%)
./.nuxt/dist/client/pages/feedback.js 3.23 kB +1 B (0%)
./.nuxt/dist/client/pages/feedback.modern.js 3.21 kB +4 B (0%)
./.nuxt/dist/client/pages/image/_id.js 6.99 kB +2 B (0%)
./.nuxt/dist/client/pages/image/_id.modern.js 6.76 kB +3 B (0%)
./.nuxt/dist/client/pages/meta-search.js 1.67 kB +3 B (0%)
./.nuxt/dist/client/pages/meta-search.modern.js 1.67 kB -3 B (0%)
./.nuxt/dist/client/pages/preferences.js 1.21 kB +1 B (0%)
./.nuxt/dist/client/pages/preferences.modern.js 1.21 kB 0 B
./.nuxt/dist/client/pages/search-help.js 1.55 kB 0 B
./.nuxt/dist/client/pages/search-help.modern.js 1.55 kB 0 B
./.nuxt/dist/client/pages/search/audio.js 1.19 kB -3 B (0%)
./.nuxt/dist/client/pages/search/audio.modern.js 1.19 kB -1 B (0%)
./.nuxt/dist/client/pages/search/image.js 2.78 kB +3 B (0%)
./.nuxt/dist/client/pages/search/image.modern.js 2.67 kB +5 B (0%)
./.nuxt/dist/client/pages/search/index.js 2.99 kB +3 B (0%)
./.nuxt/dist/client/pages/search/index.modern.js 2.96 kB -2 B (0%)
./.nuxt/dist/client/pages/search/model-3d.js 243 B 0 B
./.nuxt/dist/client/pages/search/model-3d.modern.js 247 B +1 B (0%)
./.nuxt/dist/client/pages/search/search-page.types.js 266 B 0 B
./.nuxt/dist/client/pages/search/search-page.types.modern.js 271 B 0 B
./.nuxt/dist/client/pages/search/video.js 240 B +1 B (0%)
./.nuxt/dist/client/pages/search/video.modern.js 244 B 0 B
./.nuxt/dist/client/pages/sources.js 1.44 kB +1 B (0%)
./.nuxt/dist/client/pages/sources.modern.js 1.45 kB -1 B (0%)
./.nuxt/dist/client/vendors/app.js 47.8 kB -2 B (0%)

compressed-size-action

Copy link
Member

@dhruvkb dhruvkb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that the implementation currently works, I'm ok with removing the logging.

But would it be a better solution to make the log function a no-op on the production server-side? That allows us to keep the detailed logging in dev untouched, just in case.

export const getLogger = (level: 'log' | 'warn' | 'error') =>
isProd && isClient
? () => {
// do nothing
}
: console[level]

@zackkrida
Copy link
Member Author

@dhruvkb I'm not sure; they also clutter local development considerably too. Perhaps I should comment them out rather than delete for easy restoration if there's ever issues with the token implementation?

Copy link
Member

@dhruvkb dhruvkb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to delete them if they aren't useful in dev either.

Copy link
Contributor

@obulat obulat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! This has been making the dev output almost unusable, but we might want to add at least some indication of whether the token is added to requests or not.

@@ -176,12 +161,6 @@ const apiToken: Plugin = async (context, inject) => {
let openverseApiToken: string | undefined
try {
openverseApiToken = await getApiAccessToken(context)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should leave these lines so that it's clear if we are using the token when fetching or not?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure. We're already sending the error('Unable to retrieve API token, clearing existing token', e) when things fail, which i think would be enough information to debug this token code in the event of a problem.

@zackkrida
Copy link
Member Author

@sarayourfriend as the original author would you mind looking at this before I merge?

@sarayourfriend
Copy link
Contributor

What was the increase in log storage costs? We had zero logs before on the frontend so I'm not surprised if it went up several thousand percent, but I would like to know what the overall cost of these logs is.

Can we remove the logs that happen on every request but keep the ones around the token refresh? Those should only happen every 12 hours per instance.

Copy link
Contributor

@sarayourfriend sarayourfriend left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@zackkrida zackkrida merged commit 967b001 into main Sep 26, 2022
@zackkrida zackkrida deleted the reduce-logging branch September 26, 2022 16:43
github-actions bot pushed a commit that referenced this pull request Sep 26, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
🚦 status: awaiting triage Has not been triaged & therefore, not ready for work
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants