Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tests fail when using NEXUSMODS_APP_USE_SYSTEM_EXTRACTOR #1836

Closed
MattSturgeon opened this issue Jul 31, 2024 · 2 comments · Fixed by #1919
Closed

Tests fail when using NEXUSMODS_APP_USE_SYSTEM_EXTRACTOR #1836

MattSturgeon opened this issue Jul 31, 2024 · 2 comments · Fixed by #1919
Labels
Bug Something isn't working os-linux This affects Linux related code.

Comments

@MattSturgeon
Copy link
Contributor

MattSturgeon commented Jul 31, 2024 •
edited
Loading

Bug Report

Summary

Tests complain they cannot find the file '/build/source/tests/NexusMods.CLI.Tests/bin/Release/net8.0/runtimes/linux-x64/native/7zz'.

Steps to reproduce

Run the tests on a build built with NEXUSMODS_APP_USE_SYSTEM_EXTRACTOR.

More specifically:

  • install nix
  • checkout nexusmods-app: 0.4.1 -> 0.6.1 NixOS/nixpkgs#331150
  • edit pkgs/by-name/ne/nexusmods-app/package.nix to remove the symlink in postPatch
  • run `nix-build -A nexusmods-app
  • run the provided nix log command and search for Failed
    • e.g. nix log /nix/store/f97pyrppffpc96wzmx9klqlh2k0z6hm0-nexusmods-app-0.5.3.drv
    • /Failed (search forward for the regex "Failed")
    • N (jump to next match)

What is the expected behaviour?

I expected 7zz to be found on the PATH.

If the PATH is being searched, I'd expect that to show up in the error message.

Other information

(Logs, screenshots, possible fixes, expected behaviour, etc.)

[xUnit.net 00:00:00.63]         System.ComponentModel.Win32Exception (2): Failed to start a process with file path '/build/source/tests/NexusMods.CLI.Tests/bin/Release/net8.0/runtimes/linux-x64/native/7zz'. Target file or working directory doesn't exist, or the provided credentials are invalid. ---> System.ComponentMo>
[xUnit.net 00:00:00.63]            at System.Diagnostics.Process.ForkAndExecProcess(ProcessStartInfo startInfo, String resolvedFilename, String[] argv, String[] envp, String cwd, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTermin>
[xUnit.net 00:00:00.63]            at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
[xUnit.net 00:00:00.63]            at CliWrap.Utils.ProcessEx.Start() in /_/CliWrap/Utils/ProcessEx.cs:line 56
[xUnit.net 00:00:00.63]            at CliWrap.Utils.ProcessEx.Start() in /_/CliWrap/Utils/ProcessEx.cs:line 68
[xUnit.net 00:00:00.63]            at CliWrap.Command.ExecuteAsync(CancellationToken forcefulCancellationToken, CancellationToken gracefulCancellationToken) in /_/CliWrap/Command.Execution.cs:line 307
[xUnit.net 00:00:00.63]            at CliWrap.Command.ExecuteAsync(CancellationToken cancellationToken) in /_/CliWrap/Command.Execution.cs:line 326
[xUnit.net 00:00:00.63]            at NexusMods.FileExtractor.Extractors.SevenZipExtractor.ExtractAllAsync_Impl(IStreamFactory sFn, AbsolutePath destination, CancellationToken token, IActivitySource`1 activity) in /build/source/src/ArchiveManagement/NexusMods.FileExtractor/Extractors/SevenZipExtractor.cs:line 147
[xUnit.net 00:00:00.63]            at NexusMods.FileExtractor.Extractors.SevenZipExtractor.ExtractAllAsync_Impl(IStreamFactory sFn, AbsolutePath destination, CancellationToken token, IActivitySource`1 activity) in /build/source/src/ArchiveManagement/NexusMods.FileExtractor/Extractors/SevenZipExtractor.cs:line 188
[xUnit.net 00:00:00.63]            at NexusMods.FileExtractor.Extractors.SevenZipExtractor.ExtractAllAsync(IStreamFactory sFn, AbsolutePath destination, CancellationToken token) in /build/source/src/ArchiveManagement/NexusMods.FileExtractor/Extractors/SevenZipExtractor.cs:line 69
[xUnit.net 00:00:00.63]            at NexusMods.FileExtractor.FileExtractor.ExtractAllAsync(IStreamFactory sFn, AbsolutePath dest, CancellationToken token) in /build/source/src/ArchiveManagement/NexusMods.FileExtractor/FileExtractor.cs:line 68
[xUnit.net 00:00:00.65]     NexusMods.CLI.Tests.VerbTests.ModManagementVerbs.CanCreateAndManageLists [FAIL]
[xUnit.net 00:00:00.65]       NexusMods.FileExtractor.FileExtractionException : No Extractors found for file /build/source/tests/NexusMods.CLI.Tests/bin/Release/net8.0/Resources/StubbedGameState.zip
[xUnit.net 00:00:00.65]       Stack Trace:
[xUnit.net 00:00:00.65]         /build/source/src/ArchiveManagement/NexusMods.FileExtractor/FileExtractor.cs(77,0): at NexusMods.FileExtractor.FileExtractor.ExtractAllAsync(IStreamFactory sFn, AbsolutePath dest, CancellationToken token)
[xUnit.net 00:00:00.65]         /build/source/src/ArchiveManagement/NexusMods.FileExtractor/FileExtractor.cs(51,0): at NexusMods.FileExtractor.FileExtractor.ExtractAllAsync(AbsolutePath path, AbsolutePath dest, CancellationToken token)
[xUnit.net 00:00:00.65]         /build/source/tests/NexusMods.StandardGameLocators.TestHelpers/StubbedGames/StubbedGame.cs(111,0): at NexusMods.StandardGameLocators.TestHelpers.StubbedGames.StubbedGame.AddTestFiles(TemporaryPath path, IServiceProvider provider)
[xUnit.net 00:00:00.65]         /build/source/tests/NexusMods.StandardGameLocators.TestHelpers/StubbedGames/StubbedGame.cs(101,0): at NexusMods.StandardGameLocators.TestHelpers.StubbedGames.StubbedGame.Create(IServiceProvider provider)
[xUnit.net 00:00:00.65]         /build/source/tests/NexusMods.CLI.Tests/VerbTests/AVerbTest.cs(93,0): at NexusMods.CLI.Tests.VerbTests.AVerbTest.CreateInstall()
[xUnit.net 00:00:00.65]         /build/source/tests/NexusMods.CLI.Tests/VerbTests/ModManagementVerbs.cs(14,0): at NexusMods.CLI.Tests.VerbTests.ModManagementVerbs.CanCreateAndManageLists()
[xUnit.net 00:00:00.65]       Output:
[xUnit.net 00:00:00.65]         [2024-07-31 16:18:20Z] dbug: NexusMods.FileExtractor.Extractors.SevenZipExtractor[0]
[xUnit.net 00:00:00.65]               Extracting StubbedGameState.zip
[xUnit.net 00:00:00.65]         [2024-07-31 16:18:20Z] dbug: NexusMods.FileExtractor.Extractors.SevenZipExtractor[0]
[xUnit.net 00:00:00.65]               Cleaning up after extraction
[xUnit.net 00:00:00.65]         [2024-07-31 16:18:20Z] fail: NexusMods.FileExtractor.FileExtractor[0]
[xUnit.net 00:00:00.65]               While extracting via NexusMods.FileExtractor.Extractors.SevenZipExtractor
[xUnit.net 00:00:00.65]         System.ComponentModel.Win32Exception (0x80004005): Failed to start a process with file path '/build/source/tests/NexusMods.CLI.Tests/bin/Release/net8.0/runtimes/linux-x64/native/7zz'. Target file or working directory doesn't exist, or the provided credentials are invalid.
[xUnit.net 00:00:00.65]          ---> System.ComponentModel.Win32Exception (2): An error occurred trying to start process '/build/source/tests/NexusMods.CLI.Tests/bin/Release/net8.0/runtimes/linux-x64/native/7zz' with working directory '/build/source/tests/NexusMods.CLI.Tests/bin/Release/net8.0'. No such file or direc>
[xUnit.net 00:00:00.65]            at System.Diagnostics.Process.ForkAndExecProcess(ProcessStartInfo startInfo, String resolvedFilename, String[] argv, String[] envp, String cwd, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTermin>
[xUnit.net 00:00:00.65]            at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
[xUnit.net 00:00:00.65]            at CliWrap.Utils.ProcessEx.Start() in /_/CliWrap/Utils/ProcessEx.cs:line 56
[xUnit.net 00:00:00.65]            at CliWrap.Utils.ProcessEx.Start() in /_/CliWrap/Utils/ProcessEx.cs:line 68
[xUnit.net 00:00:00.65]            at CliWrap.Command.ExecuteAsync(CancellationToken forcefulCancellationToken, CancellationToken gracefulCancellationToken) in /_/CliWrap/Command.Execution.cs:line 307
[xUnit.net 00:00:00.65]            at CliWrap.Command.ExecuteAsync(CancellationToken cancellationToken) in /_/CliWrap/Command.Execution.cs:line 326
[xUnit.net 00:00:00.65]            at NexusMods.FileExtractor.Extractors.SevenZipExtractor.ExtractAllAsync_Impl(IStreamFactory sFn, AbsolutePath destination, CancellationToken token, IActivitySource`1 activity) in /build/source/src/ArchiveManagement/NexusMods.FileExtractor/Extractors/SevenZipExtractor.cs:line 147
[xUnit.net 00:00:00.65]            at NexusMods.FileExtractor.Extractors.SevenZipExtractor.ExtractAllAsync_Impl(IStreamFactory sFn, AbsolutePath destination, CancellationToken token, IActivitySource`1 activity) in /build/source/src/ArchiveManagement/NexusMods.FileExtractor/Extractors/SevenZipExtractor.cs:line 188
[xUnit.net 00:00:00.65]            at NexusMods.FileExtractor.Extractors.SevenZipExtractor.ExtractAllAsync(IStreamFactory sFn, AbsolutePath destination, CancellationToken token) in /build/source/src/ArchiveManagement/NexusMods.FileExtractor/Extractors/SevenZipExtractor.cs:line 69
[xUnit.net 00:00:00.65]            at NexusMods.FileExtractor.FileExtractor.ExtractAllAsync(IStreamFactory sFn, AbsolutePath dest, CancellationToken token) in /build/source/src/ArchiveManagement/NexusMods.FileExtractor/FileExtractor.cs:line 68
  Failed NexusMods.CLI.Tests.VerbTests.ExtractArchive.CanExtractArchive [85 ms]
  Error Message:
   NexusMods.FileExtractor.FileExtractionException : No Extractors found for file /build/source/tests/NexusMods.CLI.Tests/bin/Release/net8.0/Resources/data_7zip_lzma2.7z

Building on this nixpkgs PR NixOS/nixpkgs#331150 so there may be other contributing factors. E.g. I'm not 100% sure that PATH is being set correctly when running the tests.
@erri120
Copy link
Member

erri120 commented Jul 31, 2024

Looks like the tests weren't built with NEXUSMODS_APP_USE_SYSTEM_EXTRACTOR:

NexusMods.App/src/ArchiveManagement/NexusMods.FileExtractor/Extractors/SevenZipExtractor.cs

Lines 203 to 215 in 6a9cee9

private static string GetExtractorExecutable(IFileSystem fileSystem)
{
var fileName = GetExtractorExecutableFileName();
if (UseSystemExtractor) return fileName;
var directory = OSInformation.MatchPlatform(
onWindows: static () => "runtimes/win-x64/native/",
onLinux: static () => "runtimes/linux-x64/native/",
onOSX: static () => "runtimes/osx-x64/native/"
);
return fileSystem.GetKnownPath(KnownPath.EntryDirectory).Combine(directory + fileName).ToString();
}

Otherwise, you wouldn't see runtimes/linux-x64/native/7zz in the logs.

@erri120 erri120 closed this as not planned Won't fix, can't repro, duplicate, stale Jul 31, 2024
@Sewer56
Copy link
Member

Sewer56 commented Jul 31, 2024

I imagine this will probably be handled next week at the soonest.
But I am replying to acknowledge the receipt.

https://github.com/NixOS/nixpkgs/blob/d365cd6095751b399bb868ba2057564327f79e7f/pkgs/by-name/ne/nexusmods-app/package.nix#L24-L30

https://github.com/NixOS/nixpkgs/blob/d365cd6095751b399bb868ba2057564327f79e7f/pkgs/by-name/ne/nexusmods-app/package.nix#L59-L64

https://github.com/NixOS/nixpkgs/blob/d365cd6095751b399bb868ba2057564327f79e7f/pkgs/by-name/ne/nexusmods-app/package.nix#L131-L135

I am reopening this because their package is being built with NEXUSMODS_APP_USE_SYSTEM_EXTRACTOR.

To reproduce:

  1. Build NexusMods.FileExtractor.Tests with dotnet test -p:DefineConstants="NEXUSMODS_APP_USE_SYSTEM_EXTRACTOR"
  2. Delete the 7zz binary from runtimes folder from NexusMods.FileExtractor.Tests' build folder.
  3. Start test with dotnet test -p:DefineConstants="NEXUSMODS_APP_USE_SYSTEM_EXTRACTOR" --no-build.

The test binary tried using the built-in 7zz and not the system one.
If you build with -p:DefineConstants="USE_SYSTEM_EXTRACTOR", then the tests should pass if you have 7zz binary.

  <_UseSystemExtractor Condition="'$(NEXUSMODS_APP_USE_SYSTEM_EXTRACTOR)' == '1'">true</_UseSystemExtractor>
</PropertyGroup>
  
<PropertyGroup Condition="'$(_UseSystemExtractor)' == 'true'">
    <DefineConstants>USE_SYSTEM_EXTRACTOR</DefineConstants>
</PropertyGroup>

Issue probably has something to do with this block.

@Sewer56 Sewer56 reopened this Jul 31, 2024
@LukeNexusMods LukeNexusMods added Bug Something isn't working os-linux This affects Linux related code. labels Aug 1, 2024
@MattSturgeon MattSturgeon mentioned this issue Aug 21, 2024
Merged
20 tasks
@Sewer56 Sewer56 mentioned this issue Aug 21, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug Something isn't working os-linux This affects Linux related code.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fixed: Building NMA with System Extractor Nexus-Mods/NexusMods.App
4 participants
@MattSturgeon @Sewer56 @erri120 @LukeNexusMods