Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core): queued storage (Web) for logger #12791

Merged
merged 1 commit into from
Jan 10, 2024
Merged

Conversation

HuiSF
Copy link
Member

@HuiSF HuiSF commented Jan 3, 2024

Description of changes

Add implementation of a queued storage for the upcoming logger. Backed by the IndexedDB on Web.

Issue #, if available

Description of how you validated changes

Checklist

  • PR description included
  • yarn test passes
  • Tests are changed or added
  • Relevant documentation is changed or added (and PR referenced)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@HuiSF HuiSF requested review from a team as code owners January 3, 2024 20:20
@HuiSF HuiSF force-pushed the hui/feat/core/queued-storage branch 3 times, most recently from 10f7253 to 912924f Compare January 4, 2024 20:49
packages/core/src/utils/queuedStorage/types.ts Outdated Show resolved Hide resolved
mockGetAll.mockClear();
});

test('peek() returns specified number of queued items', async () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Looks like you're intentionally mixing it and test, would it be better to add describe instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it and test are identical functionally as it is an alias of test, I think they fit in in this context to save two describe() wrappers... wdyt?

@HuiSF HuiSF force-pushed the hui/feat/core/queued-storage branch from 912924f to 38e0351 Compare January 5, 2024 18:21
Copy link
Contributor

@jimblanc jimblanc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@HuiSF HuiSF force-pushed the hui/feat/core/queued-storage branch 2 times, most recently from e89ef1f to f359840 Compare January 9, 2024 00:58
Copy link
Member

@AllanZhengYP AllanZhengYP left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks good to me!


const openDBPromise = new Promise<IDBDatabase>((resolve, reject) => {
const { indexedDB } = window;
const openRequest = indexedDB.open(DATABASE_NAME, 1);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super nit: is 1 required?

const { indexedDB } = window;
const openRequest = indexedDB.open(DATABASE_NAME, 1);

openRequest.onupgradeneeded = () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: some doc string around when onupgradeneeded is triggered. From untrained eye, I'd expect upgrade is not needed when indexedDB version is 1?

Copy link
Member

@AllanZhengYP AllanZhengYP left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks good to me!


await promisifyIDBRequest(request);

for (const item of request.result) {
Copy link
Member

@ashwinkumar6 ashwinkumar6 Jan 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit, QQ
Are we fetching only the items recently changed and append the size or are we fetching all items in db and then re-assign the size

I'm guessing it's the former. so ideally would request.result just have a single item when we're adding

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's all items, on app reload.

await promisifyIDBRequest(request);

for (const item of request.result) {
currentBytesSize += (item as QueuedItem).bytesSize;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit QQ
Would openRequest.onsuccess get called when items are deleted as well. In which case this would append that size as well

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

openRequest.onsuccess is being called once when opening the db.

@@ -0,0 +1,380 @@
// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
// SPDX-License-Identifier: Apache-2.0

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit QQ
when adding or deleting items are we also testing if the total currentBytesSize is getting updated as expected

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 will add a new test for this separately.

@HuiSF HuiSF force-pushed the hui/feat/core/queued-storage branch from f359840 to 5596896 Compare January 10, 2024 19:07
@HuiSF HuiSF merged commit ff20db7 into main Jan 10, 2024
30 checks passed
@HuiSF HuiSF deleted the hui/feat/core/queued-storage branch January 10, 2024 19:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants