Running tests with nunit3-console version 3.15.0 generates empty log files

[ciconq]

Hello,

When running tests through nunit3-console version 3.15.0 for example the following command:

nunit3-console.exe Tests.dll

I am observing a generation of empty log files like internal-trace and nunit-agent after each execution. Which is adding those types of files with random numbers concatenated to them, resulting in a lot of empty log files.

I was able to disable the generation of the internal-trace log files with the --trace=off option but the nunit-agent empty log files keep getting generated after each execution.

I had a question about this on stackoverflow and was given the suggestion to add a bug report here which might help fixing this.

@CharliePoole answered that the version of the new engine version 3.15.0 has introduced a new internal feature allowing code to change the level of debugging dynamically which is not yet exposed to users.
And the side effect is that empty log files are being created.
Also that there is a fix for this in the development code for version 4.0 which is not ported back to version 3 code.

The only fix for the moment is to go back to version 3.14.0 but I am using .Net 6.0 and can't really go back to version 3.14.0.

Would be really nice if it can be fixed in version 3 code.

Thank you for your time and attention, really appreciate it.

P.S. @CharliePoole Sorry if I am not supposed to tag you but thought I wouldn't want to discredit that you were the one that answered my question on stackoverflow. :)

[CharliePoole]

I put this in the 3.15.1 milestone with one other non-critical bug. There will be a 3.15.1 release only if a critical bug comes along to trigger it.

[CharliePoole]

No longer failing in 3.15.1, so I'm closing. If it pops up again, we can reactivate the bug and take a closer look.

[CharliePoole]

This issue has been resolved in version 3.15.2

The release is available on:
GitHub.
NuGet packages are also available NuGet.org and
Chocolatey Packages may be found at Chocolatey.org

[cw397]

We are still getting this issue in 3.15.2. I created a new solution in VS2022 with a single NUnit Test Project targeting .NET 6. When calling NUnit Console from the command line on the dll produced by that project I get the two empty log files mentioned above in the current working directory.

• NUnit: 3.13.3
• NUnit Console: 3.15.2

It isn't a big issue, but after a few runs they soon stack up!

[TimothyDJewell]

Same report, but the "workaround" I've found is to run with --trace=Off as well as --inprocess. It's not a real solution, but each command takes out the internal trace and nunit agent logs respectively.

[CharliePoole]

OK... I see I missed something in the original report. Let me clarify what was "fixed" first:

The BUG...
When running with --trace=Off, empty log files were being generated. Since that's the default setting, the same thing was happening when no --trace option was specified at all. Both of these were fixed.

NOT the BUG...
Running with --trace=<anything else> you will get log files and some might (rarely) be empty.

UPDATE: The issue has been re-opened

So... questions for those still experiencing a problem:

1. Are you seeing different behavior when running with no trace option versus --trace=Off? If so, which files are generated?
2. Are there some target runtimes for which the agent log files are incorrectly generated? Which runtimes?

Depending on the answers I may either reopen this or create a new issue.

Note: Running with --inprocess will of course eliminate the agent log files, since no agents are used in that case. However it's not possible to run in process if you target .NET Core, since the runner itself runs under .NET Framework.

[CharliePoole]

No need for feedback on my two questions... I verified for myself...

1. Default setting creates all log files
2. The agent log file is created even when trace is set to Off

Reopening

[CharliePoole]

This is (again?) fixed. You can access the package from our myget feed as version 3.15.3-dev00006.

[CharliePoole]

Fix has now been applied to both the 3.15.3 build and the 4.0 development builds on MyGet

[cw397]

Just to confirm that 3.15.3-dev00006 does indeed fix the log file issue for us.

[CharliePoole]

Great! My guess is that the original fix got dropped in the release process somehow.

[alteredego]

Great! My guess is that the original fix got dropped in the release process somehow.

Probably old news, but this appears to have slipped from the 3.17 release too. Should I create a new issue? I couldn't find one open.

[OsirisTerje]

@alteredego The 3.17 is a direct descendant of 3.15, so are you sure that is not fixed there? It is confirmed fixed in 3.15.3

[alteredego]

@OsirisTerje - 3.16.* is good, 3.15.5 is not.

This MR seems to comment out a line of code related to "InternalTrace.Initialize"

It still seems to be commented out on Main branch, but is uncommented in 3.15.* releases and 3.16 seems a bit different from 3.15 and Main.

[alteredego]

I can't pull a 3.15.3 release but I can confirm 3.15.4 and 3.15.5 creates InternalTrace.[num].log and nunit-agent_[num].log files, 3.16 doesn't. 3.17 does.

[OsirisTerje]

@alteredego 3.16 is very different from 3.15. 3.17 comes directly from 3.15. Main branch is 4, which is defunct. The fix referred to was PR #1213

We had this issue with the adapter, and the fix did cure it. IIRC InternalTrace is then set off.

[alteredego]

OK, so the InternalTrace.Initialize line was removed in that PR, but it's back in 3.17. And seems to be commented out in Main.

IF that's the root cause. It might not be as simple as that of-course and I haven't pulled the source to build it tbh, I'm only going by what I'm running locally with the built nuget packages and I'm seeing these files created with 3.17 and the 3.15 versions I mention above.

Thanks for responding and looking at this - really appreciate it :-)

[OsirisTerje]

I can confirm what you see using the nunit3-console, so I'll reopen this issue.

[CharliePoole]

Still happening in my 3.18.0 development builds. Very annoying how this keeps coming back.

[CharliePoole]

Correction: I was seeing it tests I'm running using the older console runners. It's fixed in main (version3) branch. I'll check version4 as well.