Current directory C:\WINDOWS\system32 again

[janid1967]

After upgrading from nunit3testadapter 3.17.0 to 4.3.0 I start to get this in Output->Test:
"NUnit Adapter 4.3.0.0: Test execution started
An exception occurred while invoking executor 'executor://nunit3testexecutor/': Access to the path 'C:\WINDOWS\system32\InternalTrace.31528.log' is denied. Access denied while trying to create "TestResults" folder in mentioned location. You are getting this exception because you are running vstest.console.exe from a folder which requires having write access. To solve the issue: please run vstest.console.exe from a folder where you have write privileges."

(I've read #637 which should solve what I now experience)

Please assist

Installed Nuget NUnit 3.13.3 and NUnit3TestAdapter 4.3.0
Visual Studio

Target framework .NET Framework 4.72

[OsirisTerje]

Thanks @janid1967 for reporting! What settings do you use?
Preferably, could you upload or point to a very simple repro for this?

[janid1967]

I use just a simple .runsettings, setting one env var:

<!-- File name extension must be .runsettings -->
<RunSettings>
    <RunConfiguration>
        <EnvironmentVariables>
            <TESTSETTINGFILE>DebugLocalSettings.xml</TESTSETTINGFILE>
        </EnvironmentVariables>
    </RunConfiguration>
</RunSettings>```

[janid1967]

Tried WorkingDirectory with no effect:

<!-- File name extension must be .runsettings -->
<RunSettings>
    <RunConfiguration>
        <EnvironmentVariables>
            <TESTSETTINGFILE>DebugLocalSettings.xml</TESTSETTINGFILE>
        </EnvironmentVariables>
    </RunConfiguration>
    <WorkDirectory>c:\tmp</WorkDirectory>
</RunSettings>```

[OsirisTerje]

I am not able to reproduce the error. The error message indicates it is in the testhost.
The setting you enable is not related to the adapter.
If you write out the current directory from the adapter, you will get your binary folder for the test assembly. You seems to get the folder for the vstest.console, and not sure how you get that.
The setting for the workdirectory: Should that be an NUnit setting? As it is now it is a general setting, and I'm not sure that will work at all.
Can you point to where you found the information related to these settings?

If you can upload a small repro it would be easier to help out :-)

[OsirisTerje]

@CharliePoole Just had a screen-sharing session with @janid1967 , using the 4.2.1 adapter with the 3.15.2 engine, and it seems that the fault happens down in the Engine, see the following screenshot:

We set the InternalTraceLevel to Off, but it did not have any effect. This does not happen with the 4.2.1 which uses the 3.13 engine.
Also, very weird it finds the windir as current directory. We checked what the adapter sees, which is the workdirectory, also, the adapter has code for excluding windir, and if it finds that, it will set current directory to the %temp% folder.

[CharliePoole]

Do you set the engine WorkDirectory? Note that this property is not the same as the process current directory, but is the specific directory where you want test results and logs to go. It defaults to the current directory but that's only useful when using the console runner from the command-line. When I get an instance of the engine, I generally always set the TraceLevel and WorkDirectory properties.

[janid1967]

I've tried this with no luck

(PS: Setting the env var TESTSETTINGFILE is something I used internally in tests)

[CharliePoole]

@janid1967 My comment was in reference to what I believe the NUnit3 Adapter code should do. If the code (which I haven't examined) is using the WorkDirectory property from RunSettings to set current directory, that won't have any effect on where logs are saved.

[OsirisTerje]

The adapter's WorkDirectory property is sent to the engine as part of the package properties.

package.Settings[PackageSettings.WorkDirectory] = workDir;

And likewise:

   package.Settings[PackageSettings.InternalTraceLevel] = Settings.InternalTraceLevel;

[CharliePoole]

This is what the console runner does...

      if (Options.WorkDirectory != null)
          engine.WorkDirectory = Options.WorkDirectory;

      engine.InternalTraceLevel = Options.InternalTraceLevel != null
          ? (InternalTraceLevel)Enum.Parse(typeof(InternalTraceLevel), Options.InternalTraceLevel)
          : InternalTraceLevel.Off;

As you see, it allows the engine to deal with a non-specified work directory by not providing a setting at all. The engine will then use the current directory. In fact, this would give an error if the user were to run the console from a write-protected directory but this has never come up as an issue.

So the runner is not changing the current directory, it should not be necessary to provide an option at all. However, my vague memory is that the IDE at least does take control of the current directory, which is why some setting is necessary. Can you confirm that the test package setting is correct at the point where you use it to get a runner both for discovery and execution? If so, then I think this may be a bug in 3.15.2.

If there's a bug, it will go into 3.16, of course. I'll also have to figure out a way to test it without using the adapter.

[OsirisTerje]

VS fires up a testhost.exe. This is the equivalent of the runner, but that resides (at least for a net 6.0 project) in the binary output folder, like:

Also note that in THIS case the log files is also here, but they are all empty. The setting for InternalTraceLevel is Off. If I set it to Info, I get a file with content:

So it looks like the setting is being passed down correctly, but also that the file is being created regardless of its setting.

In this case I haven't set any workdirectoy, so it is just the default. What is real strange with @janid1967 's case is that regardless of whether he sets the workdirectory or not, the engine believes the current directory is that WinDir. Now, his projects uses the old VS project system (and package.configs) and targets net4.7.2, and there might be something strange going on there.
I can't reproduce the case he has there, but setting the InternalTraceLevel to Off should have closed that issue.

[CharliePoole]

The empty log files are this issue: nunit/nunit-console#1178

It will be closed in 3.16. We talked about this before. This is the issue that was fixed for 3.15.2, but somehow didn't get into the packages, so I did it over for 3.16.

One note: if you pass a relative directory to the engine, it is combined with the current directory. That may be what happens in @janid1967's case.

[OsirisTerje]

Ok, good to hear it is known. When is 3.16 ETA ? I'll need to put out an updated adapter with that one directly after that.

The path he passed in was absolute c:\tmp.

If I do the same in my project here, the tracelogs go correctly to c:\tmp.

I've tried to fire up some older projects here, but they are repro issues and got issues. Further, the old csproj system doesn't handle the switching of nuget versions very well.

@janid1967 Can you just create two very simple test programs using your old csproj system, one with 4.2.1 and one with 4.3.0, and upload? Use the same runsettings file in both. If they show the same behaviour it is some kind of bug, but if they behave correctly, there might be some older parts hanging in there in your project, remains from earlier versions or something.

[OsirisTerje]

@janid1967 Also, please check that the vstest cache is either not present or empty, that has been a really annoying issue earlier, both for NUNit and XUnit.

Also, Unload your test project and check the project XML, ensure that there is no mess like this in that file:

[CharliePoole]

@OsirisTerje Sometime in November - as I wrote when I announced my resignation.

The main thing holding it up is the .NET 7.0 final release. Since nobody is taking over from me, it seems like a good idea to wait for that. However, if it's not out in November then I'll release with support for RC-2. From another discussion, I realize you don't actually need that support, so if the dev build worked for you, we could just pin it.

[janid1967]

@janid1967 Also, please check that the vstest cache is either not present or empty, that has been a really annoying issue earlier, both for NUNit and XUnit.

Also, Unload your test project and check the project XML, ensure that there is no mess like this in that file:

There was no VisualStudioTestExplorerExtensions folder under my C:\Users\jdalsb\AppData\Local\Temp

After migrated my test project from having PackageReference to now having packages.config, my csproj look like:

(I've not added any Import Project nodes, VS did it when adding NuGet pkgs during migration from PackageReference to packages.config)

[janid1967]

@janid1967 Can you just create two very simple test programs using your old csproj system, one with 4.2.1 and one with 4.3.0, and upload? Use the same runsettings file in both. If they show the same behaviour it is some kind of bug, but if they behave correctly, there might be some older parts hanging in there in your project, remains from earlier versions or something.

I created a minimum solution, first I added one project using NuGets

A simple test ran OK in this project

Then I added project 2, using NuGets

This project also have one simple test.
After adding project 2, no tests run in either project. Output window -> Tests show

This is without any .runsettings

[OsirisTerje]

Released here https://www.nuget.org/packages/NUnit3TestAdapter/4.3.1