Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: hide node and test runner internals from the stack #5714

Merged
merged 6 commits into from
Sep 11, 2024

Conversation

galargh
Copy link
Member

@galargh galargh commented Sep 4, 2024

  • Because this PR includes a bug fix, relevant tests have been included.
  • Because this PR includes a new feature, the change was previously discussed on an Issue or with someone from the team.
  • I didn't do anything of this.

This PR builds up on custom stack trace parsing introduced in #5700 and implements stack trace filtering. It adds a removal of the node stack references.

Only the node stack references grouped at the end of the stack are removed. They are not removed at all if there are no other stack lines to display to the user.

The stack traces should be much cleaner and easier to comprehend after this change.

Copy link

vercel bot commented Sep 4, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
hardhat ✅ Ready (Inspect) Visit Preview 💬 Add feedback Sep 10, 2024 10:42am

Copy link

changeset-bot bot commented Sep 4, 2024

⚠️ No Changeset found

Latest commit: bfe4015

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@github-actions github-actions bot added the status:ready This issue is ready to be worked on label Sep 4, 2024
@galargh galargh changed the title [WIP] feat: hide node and test runner internals from the stack feat: hide node and test runner internals from the stack Sep 4, 2024
@galargh galargh requested a review from alcuadrado September 4, 2024 09:28
@galargh galargh marked this pull request as ready for review September 4, 2024 09:28
Copy link
Member

@alcuadrado alcuadrado left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is starting to look so cool :) I just left a few comments about a few edge cases that I think we should treat differently, and, if possible, have tests for them.

Base automatically changed from galargh/test-reporter-stack to v-next September 9, 2024 11:18
Copy link
Member Author

@galargh galargh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like the idea of removing only the node references grouped at the end of the stack, and only if the rest of the stack is not empty. I'm going to update the PR with that change shortly.

v-next/hardhat-node-test-reporter/src/error-formatting.ts Outdated Show resolved Hide resolved
v-next/hardhat-node-test-reporter/src/error-formatting.ts Outdated Show resolved Hide resolved
v-next/hardhat-node-test-reporter/src/error-formatting.ts Outdated Show resolved Hide resolved
if (isNodeLocation(reference.location)) {
// Check if the location is strictly a node location if the node stack
// is empty. Otherwise, it is OK to accept <anonymous> and index locations
if (nodeLines.length > 0 || isNodeLocation(reference.location, true)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is there an || here? I don't understand the first part.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea was to only accept a "strictly" node location as the beginning of a node stack but it is no longer relevant after the refactor in bfe4015 (#5714)

@@ -312,3 +336,17 @@ export function formatLocation(
return locationPath;
}
}

function isNodeLocation(location: string, strict: boolean = false): boolean {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably documment what strict means, or we'll forget it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the notion of a strict check in the newest iteration. Instead, I added a new function hasNodeContext which checks whether the context of a reference is for one of the node types. Currently, for the purpose of the function, Array, Promise and SafePromise are considered node types. We can explicitly expand this definition as we find new cases we want to support. I think this approach is safer and easier to understand.

@@ -108,17 +109,43 @@ function formatSingleError(
): string {
const messageLines = [];
const stackLines = [];
const nodeLines = [];

for (const line of error.stack?.split("\n") ?? []) {
Copy link
Member

@alcuadrado alcuadrado Sep 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It took me a while to understand why i found this harder to follow than what I was expecting, and I think I just got it.

If you were iterrating in reverse order, the algorithm is much simpler/closer to how we dicussed:

let foundNodeFrame = false;
let stackLines = [];

const lines = error.stack?.split("\n") ?? [];
for (let i = lines.length - 1; i >= 0; i--) {
  const line = lines[i];
  //...
  if (!isNodeLocation(reference.location) && !foundNodeFrame) {
    // We remove all the frames below the first non-node ones
    stackLines = [];
    foundNodeFrame = true; 
  }
   
  stackLines.push(formattedLine);
}

stackLines.reverse();

Note that this is pseudocode though, i haven't run it, and it doesn't handle the messageLines

Copy link
Member Author

@galargh galargh Sep 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a fair point. It got a tad bit complicated with trying to extract lines from which we build up a message and discarding node lines, but not all of them, ...

This got me thinking and I put together another approach. I'm curious to see what you think about it.

First, I added a new interface StackLine which can hold both the original line that we attempt to parse and a StackReference if we succeed to parse the line into it. With this we can run all the lines through parseStackLine straight away and not worry wether we got undefined back.

Now, among all the parsed lines, we can find the first one which has a defined stack reference. With that in hand, we can build a message from all the lines from the beginning up to that line.

We can also find the last line which does NOT have a reference to node internals. With that, we can easily build the stack from all the lines between the first line with any stack reference and that line. With a special case being that the entire stack is node internals only.

In pseudo-code and without handling special cases the idea is as follows:

const parsedLines = error.stack.map(parseLine);

const firstLineWithReference = parsedLines.findIndex(line => hasReference(line));
const lastLineWithoutNodeReference = parseLines.findLastIndex(line => !hasNodeReference(line));

const message = parsedLines.slice(0, firstLineWithReference).map(formatLine).join("\n");
const stack = parsedLines.slice(firstLineWithReference, lastLineWithoutNodeReference + 1).map(formatLine).join("\n");

Let me know what you think :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it. Much easier to understand.

Copy link
Member

@alcuadrado alcuadrado left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice change!

@galargh galargh added this pull request to the merge queue Sep 10, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Sep 10, 2024
@galargh galargh added this pull request to the merge queue Sep 11, 2024
Merged via the queue into v-next with commit bb64e38 Sep 11, 2024
59 checks passed
@galargh galargh deleted the galargh/test-reporter-stack-cleanup branch September 11, 2024 07:51
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 11, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
status:ready This issue is ready to be worked on
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

2 participants