Fix extension

[Doprez]

PR Details

Currently Shader key files are not being generatedsince the 4.2 release. The original error was due to an incompability from net 8 to net472.

Description

This fully moves the extension over to .NET 8 and attempts to fix the errors causing many headaches while writing shaders.

Related Issue

None

Motivation and Context

If this gets fixed shaders will be easier to work with.

Types of changes

• Docs change / refactoring / dependency upgrade
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist

• My change requires a change to the documentation.
• I have added tests to cover my changes.
• All new and existing tests passed.
• I have built and run the editor to try this change out.

[Doprez]

I may need some help on this, I got the whole project building with .NET 8 but for some reason the same issue persists where it cannot restore Stride.VisualStudio.Commands 4.2.0.X. I cant seem to properly debug this to step through the code so I dont know how to se why the restore is failing.

Severity	Code	Description	Project	File	Line	Suppression State
Error		System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: Could not restore Stride.VisualStudio.Commands 4.2.0.1, this visual studio extension may fail to work properly without it. To fix this you can either build Stride.VisualStudio.Commands or pull the right version from nugget manually
   at Stride.VisualStudio.Commands.StrideCommandsProxy.<FindStrideSdkDir>d__16.MoveNext()
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at Stride.VisualStudio.Commands.StrideCommandsProxy.GetProxy()
   at Stride.VisualStudio.Shaders.ShaderKeyFileGenerator.<>c__DisplayClass3_0.<GenerateCode>b__0()
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at Stride.VisualStudio.Shaders.ShaderKeyFileGenerator.GenerateCode(String inputFileName, String inputFileContent)
---> (Inner Exception #0) System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: Could not restore Stride.VisualStudio.Commands 4.2.0.1, this visual studio extension may fail to work properly without it. To fix this you can either build Stride.VisualStudio.Commands or pull the right version from nugget manually
   at Stride.VisualStudio.Commands.StrideCommandsProxy.<FindStrideSdkDir>d__16.MoveNext()
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at Stride.VisualStudio.Commands.StrideCommandsProxy.GetProxy()
   at Stride.VisualStudio.Shaders.ShaderKeyFileGenerator.<>c__DisplayClass3_0.<GenerateCode>b__0()
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
---> (Inner Exception #0) System.InvalidOperationException: Could not restore Stride.VisualStudio.Commands 4.2.0.1, this visual studio extension may fail to work properly without it. To fix this you can either build Stride.VisualStudio.Commands or pull the right version from nugget manually
   at Stride.VisualStudio.Commands.StrideCommandsProxy.<FindStrideSdkDir>d__16.MoveNext()<---
<---
	ShaderPlayground.Game	C:\dev\stride-dev\ShaderPlayground\ShaderPlayground.Game\Effects\TestShader.sdsl	1	

[Doprez]

Ok so I found one of my issues. It was using the cached VSIX extension rather than actually rebuilding it. I added /updateconfiguration to the debug properties to try and get around that.

Now my issue is the app cant startup due to not finding some .NET packages

Severity	Code	Description	Project	File	Line	Suppression State	Details
Error		CreatePkgDef : error : FileNotFoundException: Could not load file or assembly 'System.Runtime, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
   at System.Reflection.PseudoCustomAttribute._GetSecurityAttributes(RuntimeModule module, Int32 token, Boolean assembly, Object[]& securityAttributes)
   at System.Reflection.PseudoCustomAttribute.GetCustomAttributes(RuntimeAssembly assembly, RuntimeType caType, Boolean includeSecCa, Int32& count)
   at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeAssembly assembly, RuntimeType caType)
   at Microsoft.VisualStudio.Tools.CreatePkgDef.ProcessAssembly(String fileName, Hive hive, PkgDefContext context, Boolean register, RegistrationMode mode)
   at Microsoft.VisualStudio.Tools.CreatePkgDef.DoCreatePkgDef(InputArguments inputArguments)
   at Microsoft.VisualStudio.Tools.CreatePkgDef.Main(String[] arguments)

	Stride.VisualStudio.Package	C:\Users\User\.nuget\packages\microsoft.vssdk.buildtools\17.0.5232\tools\VSSDK\Microsoft.VsSDK.targets	1193		

[manio143]

Do VSIX support net8 TFM? Quick look online said only net472 and netstandard2.0

[Doprez]

ah, well that could explain a lot..

I was wondering why Interfaces was targeting both sdks. Ok I may have to scrap this and try again then, I dont understand how it was working at all before if it never supported .NET 6.

[Nicogo1705]

I think i read somewhere that this code is shared (so .net 6 is for stride engine; .net framework is for VSIX).
So i think we just need to build using .net 8 & .net framework472

[Doprez]

gotcha, that makes sense. I will do some playing around then with either framework or standard and see what happens.

[Doprez]

You were both very right! the issue was with the nuget resolver no the actual library framework.

One more error seems to be within the thread of the proxy runner as it throws a broken pipe error when running C:\Users\User\.nuget\packages\stride.visualstudio.commands\4.2.0.1\lib\net8.0-windows7.0\Stride.VisualStudio.Commands.exe so I think its at least getting closer to usable.

[Doprez]

so I think its back to working the way it was before the 4.2 update. The final error is one I cant really get around when debugging because there are multiple applications accessing the Interfaces project so it happens all of the time.

If someone has another idea of how to get around this then please let me know but otherwise I think this PR should be good to review.

[Kryptos-FR]

Shouldn't it target net-6.0 instead of net-8.0 to make sure it works with 4.1 as well?

[xen2]

Shouldn't it target net-6.0 instead of net-8.0 to make sure it works with 4.1 as well?

That's not necessary.
Basically the VSPackage is just a redirector to the actual code in the appropriate Stride release.

VSPackage is compiled as net471 (required for VS).
It then connects to a given Stride version (by running an VS command server executable from the Stride release matching user project version -- it's using RPC calls if I remember correctly).

That way we can implement/improve/fix/version most VSPackage commands inside Stride release itself without having to release VSPackage every time.
This also allows us to implement Stride VS commands using net 6.0/8.0 without being restricted by VSPackage forcing .NET Framework.

[Basewq]

I may need some help on this, I got the whole project building with .NET 8 but for some reason the same issue persists where it cannot restore Stride.VisualStudio.Commands 4.2.0.X. I cant seem to properly debug this to step through the code so I dont know how to se why the restore is failing.

Severity	Code	Description	Project	File	Line	Suppression State
Error		System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: Could not restore Stride.VisualStudio.Commands 4.2.0.1, this visual studio extension may fail to work properly without it. To fix this you can either build Stride.VisualStudio.Commands or pull the right version from nugget manually
   at Stride.VisualStudio.Commands.StrideCommandsProxy.<FindStrideSdkDir>d__16.MoveNext()
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at Stride.VisualStudio.Commands.StrideCommandsProxy.GetProxy()
   at Stride.VisualStudio.Shaders.ShaderKeyFileGenerator.<>c__DisplayClass3_0.<GenerateCode>b__0()
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at Stride.VisualStudio.Shaders.ShaderKeyFileGenerator.GenerateCode(String inputFileName, String inputFileContent)
---> (Inner Exception #0) System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: Could not restore Stride.VisualStudio.Commands 4.2.0.1, this visual studio extension may fail to work properly without it. To fix this you can either build Stride.VisualStudio.Commands or pull the right version from nugget manually
   at Stride.VisualStudio.Commands.StrideCommandsProxy.<FindStrideSdkDir>d__16.MoveNext()
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at Stride.VisualStudio.Commands.StrideCommandsProxy.GetProxy()
   at Stride.VisualStudio.Shaders.ShaderKeyFileGenerator.<>c__DisplayClass3_0.<GenerateCode>b__0()
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
---> (Inner Exception #0) System.InvalidOperationException: Could not restore Stride.VisualStudio.Commands 4.2.0.1, this visual studio extension may fail to work properly without it. To fix this you can either build Stride.VisualStudio.Commands or pull the right version from nugget manually
   at Stride.VisualStudio.Commands.StrideCommandsProxy.<FindStrideSdkDir>d__16.MoveNext()<---
<---
	ShaderPlayground.Game	C:\dev\stride-dev\ShaderPlayground\ShaderPlayground.Game\Effects\TestShader.sdsl	1	

As per this file
It seems version 4.1 and below changes lib to tools to locate the exe, eg.
C:\Users\USER\.nuget\packages\stride.visualstudio.commands\4.1.0.1948\tools\net6.0-windows7.0\Stride.VisualStudio.Commands.exe would be found correctly, however in 4.2 the exe is now in the lib folder
C:\Users\USER\.nuget\packages\stride.visualstudio.commands\4.2.0.2043\lib\net8.0-windows7.0\Stride.VisualStudio.Commands.exe so LoaderToolLocator always fail (because there's no tools folder anymore).

Note that I think the proper fix is to revert the commit that changed all nugets packages that put the exes in the lib folder instead of the tools folder, because there's a .net issue where if you <PackageReference> any nugets that contain an exe in the lib folder with the same name as its dll (eg. stride.visualstudio.commands, stride.gamestudio etc) then your project can't run your exe as .net gets confused on loading the dll/exe.

[Doprez]

It was finding the executable last I tried Ill give it another test tomorrow to double check. The issue I had with it not finding the exe was due to the missing -windows postfix in the search values. 8bf024a

The last thing I am running into was that it was accessing an already in use DLL but it made sense since it was being used by VS2022 in debug mode. I couldnt get around it to test further but the original error is fixed.

[Eideren]

There is still one remaining issue in that the extension locks the nuget file Stride.Core.Shaders.dll through Stride.VisualStudio.Commands.exe preventing the editor's nuget restore as that process tries to delete the directory containing that dll.

This can be reproduced by having the extension installed and enabled in VS, doing a full rebuild of the engine then running the game studio, here's the exception:

[Error] Error restoring NuGet packages: System.AggregateException: One or more errors occurred. (Access to the path 'Stride.Core.Shaders.dll' is denied.)
 ---> System.UnauthorizedAccessException: Access to the path 'Stride.Core.Shaders.dll' is denied.
   at System.IO.FileSystem.RemoveDirectoryRecursive(String fullPath, WIN32_FIND_DATA& findData, Boolean topLevel)
   at System.IO.FileSystem.RemoveDirectory(String fullPath, Boolean recursive)
   at System.IO.DirectoryInfo.Delete(Boolean recursive)
   at NuGet.Packaging.PackageExtractor.<>c__DisplayClass5_0.<<InstallFromSourceAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at NuGet.Common.ConcurrencyUtilities.ExecuteWithFileLockedAsync[T](String filePath, Func`2 action, CancellationToken token)
   at NuGet.Common.ConcurrencyUtilities.ExecuteWithFileLockedAsync[T](String filePath, Func`2 action, CancellationToken token)
   at NuGet.Packaging.PackageExtractor.InstallFromSourceAsync(PackageIdentity packageIdentity, IPackageDownloader packageDownloader, VersionFolderPathResolver versionFolderPathResolver, PackageExtractionContext packageExtractionContext, CancellationToken token, Guid parentId)
   at NuGet.Commands.ProjectRestoreCommand.InstallPackageAsync(RemoteMatch installItem, NuGetv3LocalRepository userPackageFolder, PackageExtractionContext packageExtractionContext, CancellationToken token)
   at NuGet.Commands.ProjectRestoreCommand.<>c__DisplayClass15_1.<<InstallPackagesAsync>b__4>d.MoveNext()
--- End of stack trace from previous location ---
   at NuGet.Commands.ProjectRestoreCommand.InstallPackagesAsync(HashSet`1 uniquePackages, IEnumerable`1 graphs, IList`1 downloadDependencyInformations, NuGetv3LocalRepository userPackageFolder, CancellationToken token)
   at NuGet.Commands.ProjectRestoreCommand.TryRestoreAsync(LibraryRange projectRange, IEnumerable`1 frameworkRuntimePairs, NuGetv3LocalRepository userPackageFolder, IReadOnlyList`1 fallbackPackageFolders, RemoteDependencyWalker remoteWalker, RemoteWalkContext context, Boolean forceRuntimeGraphCreation, CancellationToken token, TelemetryActivity telemetryActivity, String telemetryPrefix)
   at NuGet.Commands.RestoreCommand.ExecuteRestoreAsync(NuGetv3LocalRepository userPackageFolder, IReadOnlyList`1 fallbackPackageFolders, RemoteWalkContext context, CancellationToken token, TelemetryActivity telemetryActivity)
   at NuGet.Commands.RestoreCommand.ExecuteAsync(CancellationToken token)
   at NuGet.Commands.RestoreRunner.ExecuteAsync(RestoreSummaryRequest summaryRequest, CancellationToken token)
   at NuGet.Commands.RestoreRunner.CompleteTaskAsync(List`1 restoreTasks)
   at NuGet.Commands.RestoreRunner.RunWithoutCommit(IEnumerable`1 restoreRequests, RestoreArgs restoreContext)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at Stride.Core.Assets.RestoreHelper.Restore(ILogger logger, NuGetFramework nugetFramework, String runtimeIdentifier, String packageName, VersionRange versionRange, String settingsRoot) in J:\stride\sources\shared\Stride.NuGetResolver\RestoreHelper.cs:line 209
   at Stride.Core.Assets.NuGetAssemblyResolver.<>c__DisplayClass5_0.<SetupNuGet>b__0(Object sender, ResolveEventArgs eventArgs) in J:\stride\sources\shared\Stride.NuGetResolver\NuGetAssemblyResolver.cs:line 129

The merge below kills that executable if nuget fails to restore, but another solution may be to load the extension assemblies without locking the file, maybe through Assembly.Load(Byte[]) in https://github.com/stride3d/stride/blob/master/sources/shared/Stride.NuGetResolver/NuGetAssemblyResolver.cs#L180
Though it's not that safe of a change, this would break building from the editor for example.

[Doprez]

If @Eiderens addition is the last fix needed can we merge this and make a 4.2 release? I think we finally have the last 3 reported bugs taken care of from the .NET 8 upgrade.

Code editor issue #2075
broken light probes #2077
broken extension [this]

[xen2]

LGTM, thanks!
I will merge it in master, generate package and test the final version to see if I can properly install it locally.

[VaclavElias]

@xen2, have you managed to test it?

[tebjan]

I just tried and i got this when opening a stride game project in visual studio:

Error initializing Stride command proxy: System.InvalidOperationException: Could not restore Stride.VisualStudio.Commands 4.2.0.2043, this visual studio extension may fail to work properly without it. To fix this you can either build Stride.VisualStudio.Commands or pull the right version from nugget manually
   at Stride.VisualStudio.Commands.StrideCommandsProxy.<FindStrideSdkDir>d__16.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Stride.VisualStudio.StridePackage.<InitializeCommandProxy>d__19.MoveNext()

[Doprez]

FindStrideSdkDir this was the original issue in 4.2.0.2043. If I understand how it works to get the correct Stride.VisualStudio.Commands version we need to have the new version released. if you run the source code you should not have this but it has a different error due to the debugging configuration.