Ensure database storage extension uses to track all storages does not grow unnecessarily #17598

karrtikr · 2021-10-01T23:28:33Z

…duplicate entries

karrtikr

Two factors which caused the issue:

karrtikr · 2021-10-02T00:05:32Z

src/client/common/persistentState.ts

@@ -89,29 +111,44 @@ export class PersistentStateFactory implements IPersistentStateFactory, IExtensi
        defaultValue?: T,
        expiryDurationMs?: number,
    ): IPersistentState<T> {
-        if (!this.workspaceKeysStorage.value.includes({ key, defaultValue })) {


{ key, defaultValue } is a different object than what's stored in this.workspaceKeysStorage.value. Even though the values are the same, this check always returned false and hence storage kept on growing.

karrtikr · 2021-10-02T00:07:00Z

src/client/common/persistentState.ts

+     * It is only cached for the particular arguments passed, so the argument type is simplified here.
+     */
+    @cache(-1, true)
+    private async addKeyToStorage<T>(keyStorageType: keysStorageType, key: string, defaultValue?: T) {


This is done in background, so it may still happen that this is again called for the same key and default value pair before the first call was finished. So cache the promise returned using decorator.

This case cannot be unit tested as easily as decorators are disabled when executing tests.

… grow unnecessarily (microsoft#17598) * Ensure central storage to track keys for other storage never contain duplicate entries * News entry * Add tests * Add more test cases * Fix storage for function

… grow unnecessarily (#17598) * Ensure central storage to track keys for other storage never contain duplicate entries * News entry * Add tests * Add more test cases * Fix storage for function

* Use correct python executable when running code using Python execution factory (#17614) * pin to debugpy verion 1.4.3 * Pin debugpy (#17619) * pin to debugpy verion 1.4.3 * Pin to particular version of debugpy. * Ensure database storage extension uses to track all storages does not grow unnecessarily (#17598) * Ensure central storage to track keys for other storage never contain duplicate entries * News entry * Add tests * Add more test cases * Fix storage for function * Fix handling of empty directories in debugpy wheels when combining them. (#17620) Add filename logging when combining wheels. Update debugpy to 1.5.0. * Update news items for cherry picked. (#17624) * Pin wrapt to 1.12.1 (#17630) (#17631) * Use conda-forge when installing into conda envs (#17629) (#17632) Co-authored-by: Brett Cannon <brcan@microsoft.com> * Ensure keys whose default value type is an array are not duplicated (#17627) (#17634) * Update version and changelog for release (#17647) * Update release date in change log * Add thanks to change log. * Fix for formatting notebook cells. (#17650) * Fix for formatting notebook cells. * Ensure we check language id. * Hide walkthrough if on the web (#17638) * Log commands run by the discovery component in the output channel (#17670) * Log commands run by the discovery component in the output channel * Remove outdated comments * Ensure commands run are not logged twice in Python output channel (#17697) * Ensure commands run are not logged twice in Python output channel * News entry * Hide UI elements that are not applicable when using `github.dev` (#17698) * Hide UI elements that are not applicable when using VS Code Web * Fix indentaiton * News entry * Disable welcome view for testing * Localize strings on `github.dev` using VSCode FS API (#17711) * Change localization in the extension to be async and use the VS Code APIs * News entry * Modify error thrown * Move localization into separate module * Update news entry * Oops * Refactor so code is not duplicated * Fix tests * Oopsp * Fix lint * Provide IntelliSense status information when using `github.dev` (#17658) * Provide IntelliSense status information when using github.dev * News entry * Update wording * PR reviews * Include proposed APIs for browser config * Include common module for browser config * Change how we detect * Change wording * Have same browser config as extension config * Revert "Have same browser config as extension config" This reverts commit 515003a. * Revert "Change wording" This reverts commit dd64f24. * Revert "Change how we detect" This reverts commit ae56079. * Revert "Include common module for browser config" This reverts commit bf1815c. * Do not import from misc module * Detect how we check if property exists * Do not localize strings * Localize strings using new localize module * Remove outdated comment * Localize 'learn more' text (#17714) * Update version and change log for point release (#17717) Co-authored-by: Kartik Raj <karraj@microsoft.com> Co-authored-by: Pavel Minaev <pminaev@microsoft.com> Co-authored-by: Don Jayamanne <don.jayamanne@outlook.com> Co-authored-by: Brett Cannon <brcan@microsoft.com> Co-authored-by: Luciana Abud <45497113+luabud@users.noreply.github.com>

* Use correct python executable when running code using Python execution factory (microsoft/vscode-python#17614) * pin to debugpy verion 1.4.3 * Pin debugpy (microsoft/vscode-python#17619) * pin to debugpy verion 1.4.3 * Pin to particular version of debugpy. * Ensure database storage extension uses to track all storages does not grow unnecessarily (microsoft/vscode-python#17598) * Ensure central storage to track keys for other storage never contain duplicate entries * News entry * Add tests * Add more test cases * Fix storage for function * Fix handling of empty directories in debugpy wheels when combining them. (microsoft/vscode-python#17620) Add filename logging when combining wheels. Update debugpy to 1.5.0. * Update news items for cherry picked. (microsoft/vscode-python#17624) * Pin wrapt to 1.12.1 (microsoft/vscode-python#17630) (microsoft/vscode-python#17631) * Use conda-forge when installing into conda envs (microsoft/vscode-python#17629) (microsoft/vscode-python#17632) Co-authored-by: Brett Cannon <brcan@microsoft.com> * Ensure keys whose default value type is an array are not duplicated (microsoft/vscode-python#17627) (microsoft/vscode-python#17634) * Update version and changelog for release (microsoft/vscode-python#17647) * Update release date in change log * Add thanks to change log. * Fix for formatting notebook cells. (microsoft/vscode-python#17650) * Fix for formatting notebook cells. * Ensure we check language id. * Hide walkthrough if on the web (microsoft/vscode-python#17638) * Log commands run by the discovery component in the output channel (microsoft/vscode-python#17670) * Log commands run by the discovery component in the output channel * Remove outdated comments * Ensure commands run are not logged twice in Python output channel (microsoft/vscode-python#17697) * Ensure commands run are not logged twice in Python output channel * News entry * Hide UI elements that are not applicable when using `github.dev` (microsoft/vscode-python#17698) * Hide UI elements that are not applicable when using VS Code Web * Fix indentaiton * News entry * Disable welcome view for testing * Localize strings on `github.dev` using VSCode FS API (microsoft/vscode-python#17711) * Change localization in the extension to be async and use the VS Code APIs * News entry * Modify error thrown * Move localization into separate module * Update news entry * Oops * Refactor so code is not duplicated * Fix tests * Oopsp * Fix lint * Provide IntelliSense status information when using `github.dev` (microsoft/vscode-python#17658) * Provide IntelliSense status information when using github.dev * News entry * Update wording * PR reviews * Include proposed APIs for browser config * Include common module for browser config * Change how we detect * Change wording * Have same browser config as extension config * Revert "Have same browser config as extension config" This reverts commit 515003aee2fa4a44105f22c4749f1fa721be6bcf. * Revert "Change wording" This reverts commit dd64f24d4c4c8cb763d9fa3aa5a2eb5808b5af9b. * Revert "Change how we detect" This reverts commit ae5607994f30b54158315c72a46583e7cf286fb2. * Revert "Include common module for browser config" This reverts commit bf1815c9b8aaa9091c37d54cf010ff3bd46c8ab4. * Do not import from misc module * Detect how we check if property exists * Do not localize strings * Localize strings using new localize module * Remove outdated comment * Localize 'learn more' text (microsoft/vscode-python#17714) * Update version and change log for point release (microsoft/vscode-python#17717) Co-authored-by: Kartik Raj <karraj@microsoft.com> Co-authored-by: Pavel Minaev <pminaev@microsoft.com> Co-authored-by: Don Jayamanne <don.jayamanne@outlook.com> Co-authored-by: Brett Cannon <brcan@microsoft.com> Co-authored-by: Luciana Abud <45497113+luabud@users.noreply.github.com>

Ensure central storage to track keys for other storage never contain …

7ccea49

…duplicate entries

github-actions bot assigned karrtikr Oct 1, 2021

Kartik Raj added 2 commits October 1, 2021 16:32

News entry

cd55d29

Add tests

8fa8617

karrtikr changed the title ~~Ensure central storage to track keys for other storage never contains duplicate entries~~ Ensure database storage extension uses to track all storages does not grow unnecessarily Oct 2, 2021

karrtikr requested a review from karthiknadig October 2, 2021 00:04

karrtikr commented Oct 2, 2021

View reviewed changes

karrtikr mentioned this pull request Oct 2, 2021

Extension enters duplicate data into local globalStorage sqlite db #17488

Closed

Kartik Raj added 2 commits October 1, 2021 17:23

Add more test cases

eccdbd3

Fix storage for function

77fd2ec

karthiknadig approved these changes Oct 4, 2021

View reviewed changes

karrtikr merged commit 77d31bb into microsoft:main Oct 4, 2021

karrtikr deleted the storage branch October 4, 2021 22:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ensure database storage extension uses to track all storages does not grow unnecessarily #17598

Ensure database storage extension uses to track all storages does not grow unnecessarily #17598

karrtikr commented Oct 1, 2021

karrtikr left a comment

karrtikr Oct 2, 2021

karrtikr Oct 2, 2021

karrtikr Oct 2, 2021

Ensure database storage extension uses to track all storages does not grow unnecessarily #17598

Ensure database storage extension uses to track all storages does not grow unnecessarily #17598

Conversation

karrtikr commented Oct 1, 2021

karrtikr left a comment

Choose a reason for hiding this comment

karrtikr Oct 2, 2021

Choose a reason for hiding this comment

karrtikr Oct 2, 2021

Choose a reason for hiding this comment

karrtikr Oct 2, 2021

Choose a reason for hiding this comment