Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Idea for copy on write #5031

Merged
merged 12 commits into from
May 1, 2023
Merged

Conversation

rchiodo
Copy link
Collaborator

@rchiodo rchiodo commented Apr 28, 2023

Wrote a simple test, but also used this in pyrx to change how fix-all works.

I put back the change events for the program/sourceFile. It was the easiest way to capture the events (otherwise I believe all the changes would be a full document replace).

@rchiodo
Copy link
Collaborator Author

rchiodo commented Apr 28, 2023

Perf data for this is very promising.

Old way of clone for the microsoft/pylance-release#4263:

  • 2200ms

New way with copy on write:

  • 100ms

I believe this is functionally equivalent. In the fix all command, instead of cloning the service, we enter edit mode. During edit mode, all changes to a source file create a new SourceFileInfo and SourceFile. From then on, all changes are tracked.

Once done with the changes, we leave edit mode and the changes made are returned in a list.

packages/pyright-internal/src/analyzer/program.ts Outdated Show resolved Hide resolved
packages/pyright-internal/src/analyzer/program.ts Outdated Show resolved Hide resolved
packages/pyright-internal/src/analyzer/service.ts Outdated Show resolved Hide resolved
packages/pyright-internal/src/analyzer/sourceFile.ts Outdated Show resolved Hide resolved
@rchiodo rchiodo marked this pull request as ready for review April 28, 2023 22:39
@rchiodo rchiodo requested a review from erictraut April 28, 2023 22:39
@@ -46,6 +48,99 @@ test('test applyWorkspaceEdits changes', async () => {
assert.strictEqual(cloned.test_program.getSourceFile(range.fileName)?.getFileContent(), 'Text Changed');
});

test('test edit mode for workspace', async () => {
Copy link
Collaborator

@heejaechang heejaechang Apr 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to have tests that change the same file multiple times and has nested changes such as

First add new import statement from test import a, c and then add new symbol to the new import you added in from test import a[, b], c in different edit and then remove import statement or reorder import statement in the file (assuming it had existing import statement) in another edit

or you can think about other multiple edits, nested edit case

let's say we provide refactoring to merge import statements and remove unused imports. so user doesn't have duplicated import statements or same symbol with different name such as import os as o and import os as os and etc in the same file.

so user would do something like

python.analysis.fixAll = [ "removeDuplicateImports", "remouveUnusedImport" ]

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed the test to do a nested change. I didn't think it would matter if there was more than one, so I just added one extra nested change and verified we ended up with a single change.

Copy link
Collaborator

@erictraut erictraut left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good.

Copy link
Collaborator

@erictraut erictraut left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good. One minor consideration. See comment.

exitEditMode(): FileEditAction | undefined {
this._isEditMode = false;

// If we had an edit, return it
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: include period at end of sentence.

this._isEditMode = true;
}

exitEditMode(): FileEditAction | undefined {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was hoping to remove knowledge of VSCode's FileEditAction and TextDocument from this file. Could we simply return the new contents as a string and let the caller construct the FileEditAction from it?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure that works. You mean at the program level?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking at the service level. The service is the part that has knowledge of LSP concepts like text documents, etc.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I should have said "languageServerBase", not "service".

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something needs the file path though? You mean create another interface that just stores the content and the file path?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see what you mean. Yeah, then keeping it in the service (in the useEditMode method) makes sense.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we get can filepath from sourceFile? and I guess that means, rename is simply delete and add new file?

@rchiodo rchiodo merged commit f460787 into microsoft:main May 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants