Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test failure : System.Diagnostics.Tests.ProcessTests.ProcessNameMatchesScriptName #13757

Open
AriNuer opened this issue Nov 8, 2019 · 23 comments
Labels
arch-x64 area-System.Diagnostics.Process disabled-test The test is disabled in source code against the issue os-linux Linux OS (any supported distro) test-bug Problem in test source code (most likely)
Milestone

Comments

@AriNuer
Copy link

AriNuer commented Nov 8, 2019

Frequency:

  • 4/1-7/22 -- 23 hits (2x official runs, 21x PR) ... average: 1-2 hits per week
let failedTests = (testNameSubstring : string, methodName : string, includePR : bool, messageSubstr: string, includePassedOnRerun : bool) {
cluster('engsrvprod.kusto.windows.net').database('engineeringdata').AzureDevOpsTests
    | where TestName contains testNameSubstring
    | where includePassedOnRerun or (Outcome == 'Failed')
    | extend startOfTestName = indexof_regex(TestName, @"[^.]+$")
    | extend Method = substring(TestName, startOfTestName)
    | extend Type = substring(TestName, 0, startOfTestName - 1)
    | project-away startOfTestName
    | where (methodName == '') or (Method == methodName)
    | where Message contains messageSubstr
    | distinct JobId, WorkItemId, Message, StackTrace, Method, Type, Arguments, Outcome
    | join kind=inner (cluster('engsrvprod.kusto.windows.net').database('engineeringdata').Jobs
        | where ((Branch == 'refs/heads/main') or (Branch == 'refs/heads/master') or (includePR and (Source startswith "pr/")))
        | where Type startswith "test/functional/cli/"
            and not(Properties contains "runtime-staging")
        | summarize arg_max(Finished, Properties, Type, Branch, Source, Started, QueueName) by JobId
        | project-rename JobType = Type) on JobId
    | extend PropertiesJson = parse_json(Properties)
    | extend OS = replace_regex(tostring(PropertiesJson.operatingSystem), @'\((.*)\).*|([^\(].*)', @'\1\2')
    | extend Runtime = iif(PropertiesJson.runtimeFlavor == "mono", "Mono", iif(PropertiesJson.DefinitionName contains "coreclr", "CoreCLR", ""))
    | extend TargetBranch = extractjson("$.['System.PullRequest.TargetBranch']", Properties)
    | extend Architecture = PropertiesJson.architecture
    | extend Scenario = iif(isempty(PropertiesJson.scenario), "--", PropertiesJson.scenario)
    //| extend DefinitionName = PropertiesJson.DefinitionName
    | project-away PropertiesJson
};
failedTests('', 'ProcessNameMatchesScriptName', true, 'Assert', true);

Job:
coreclr-corefx-jitstress:20191106.1

Details:
https://helix.dot.net/api/2019-06-17/jobs/dc1d0990-ba17-4d58-bba6-42bfb71b843b/workitems/System.Diagnostics.Process.Tests/console

OS & Arch:
Linux x64

Mode:
export COMPlus_TieredCompilation=0
export COMPlus_DbgEnableMiniDump=1
export COMPlus_DbgMiniDumpName=$HELIX_DUMP_FOLDER/coredump.%d.dmp
export COMPlus_JitStress=2

Log:

~/work/dc1d0990-ba17-4d58-bba6-42bfb71b843b/Work/0df10b6f-33b8-45e5-93e8-6b9124ec38b0/Exec ~/work/dc1d0990-ba17-4d58-bba6-42bfb71b843b/Work/0df10b6f-33b8-45e5-93e8-6b9124ec38b0/Exec
  Discovering: System.Diagnostics.Process.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Diagnostics.Process.Tests (found 214 of 272 test cases)
  Starting:    System.Diagnostics.Process.Tests (parallel test collections = on, max threads = 2)
    System.Diagnostics.Tests.ProcessStartInfoTests.ShellExecute_Nano_Fails_Start [SKIP]
      Condition(s) not met: "IsWindowsNanoServer"
Linux
Linux
Linux
Linux
None of the following programs were installed on this machine: xdg-open,gnome-open,kfmclient.
    System.Diagnostics.Tests.ProcessTests.ProcessNameMatchesScriptName [FAIL]
      Assert.Equal() Failure
                ↓ (pos 0)
      Expected: /bin/sh\0/tmp/ProcessTests_uvir03tc.k5r/Pr···
      Actual:   
                ↑ (pos 0)
      Stack Trace:
        /_/src/System.Diagnostics.Process/tests/ProcessTests.Unix.cs(180,0): at System.Diagnostics.Tests.ProcessTests.ProcessNameMatchesScriptName()
    System.Diagnostics.Tests.ProcessTests.TestProcessRecycledPid [SKIP]
      Condition(s) not met: "IsStressModeEnabled"
  Finished:    System.Diagnostics.Process.Tests
=== TEST EXECUTION SUMMARY ===
   System.Diagnostics.Process.Tests  Total: 268, Errors: 0, Failed: 1, Skipped: 2, Time: 57.208s
~/work/dc1d0990-ba17-4d58-bba6-42bfb71b843b/Work/0df10b6f-33b8-45e5-93e8-6b9124ec38b0/Exec
----- end Thu Nov 7 07:57:14 UTC 2019 ----- exit code 1 ----------------------------------------------------------
@msftgits msftgits transferred this issue from dotnet/coreclr Jan 31, 2020
@msftgits msftgits added this to the 5.0 milestone Jan 31, 2020
@ViktorHofer
Copy link
Member

Failed again in https://dev.azure.com/dnceng/public/_build/results?buildId=508452&view=logs&jobId=79ac38c7-786b-5db2-e2cf-2984d488b711.

Configuration: netcoreapp5.0-Linux-Debug-x64-CoreCLR_checked-Ubuntu.1804.Amd64.Open

Assert.Equal() Failure\n ↓ (pos 0)\nExpected: /bin/sh\0/tmp/ProcessTests_rgk4di2i.ycq/Pr···\nActual: \n ↑ (pos 0)

   at System.Diagnostics.Tests.ProcessTests.ProcessNameMatchesScriptName() in /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs:line 177

@janvorli
Copy link
Member

@maryamariyan maryamariyan added the untriaged New issue has not been triaged by the area owner label Feb 26, 2020
@adamsitnik adamsitnik modified the milestones: 5.0.0, Future Jun 25, 2020
@adamsitnik adamsitnik removed the untriaged New issue has not been triaged by the area owner label Jul 6, 2020
@adamsitnik adamsitnik added the test-bug Problem in test source code (most likely) label Dec 16, 2020
@adamsitnik adamsitnik self-assigned this Dec 18, 2020
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Dec 21, 2020
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label May 12, 2021
@ViktorHofer
Copy link
Member

Failed again in https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-58357-merge-81d9e3ef9eac4db793/System.Diagnostics.Process.Tests/1/console.2c15c039.log?sv=2019-07-07&se=2021-09-19T11%3A46%3A46Z&sr=c&sp=rl&sig=ArtgsaoLEKX4rXmEVe2yR8GjkEMWcmL82%2ByuioXfTwU%3D

Configuration: net6.0-Linux-Debug-x64-CoreCLR_release-(Fedora.34.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-34-helix-20210728124700-4f64125

  Discovering: System.Diagnostics.Process.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Diagnostics.Process.Tests (found 255 of 312 test cases)
  Starting:    System.Diagnostics.Process.Tests (parallel test collections = on, max threads = 2)
    System.Diagnostics.Tests.ProcessStartInfoTests.ShellExecute_Nano_Fails_Start [SKIP]
      Condition(s) not met: "IsWindowsNanoServer"
Linux
Linux
Linux
Linux
None of the following programs were installed on this machine: xdg-open,gnome-open,kfmclient.
    System.Diagnostics.Tests.ProcessTests.ProcessNameMatchesScriptName [FAIL]
      Assert.Equal() Failure
                ↓ (pos 0)
      Expected: /bin/sh\0/tmp/ProcessTests_ca4tz2bk.sol/Pr···
      Actual:   
                ↑ (pos 0)
      Stack Trace:
        /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs(176,0): at System.Diagnostics.Tests.ProcessTests.ProcessNameMatchesScriptName()
    System.Diagnostics.Tests.ProcessTests.TestProcessRecycledPid [SKIP]
      Condition(s) not met: "IsStressModeEnabledAndRemoteExecutorSupported"
  Finished:    System.Diagnostics.Process.Tests
=== TEST EXECUTION SUMMARY ===
   System.Diagnostics.Process.Tests  Total: 316, Errors: 0, Failed: 1, Skipped: 2, Time: 17.844s

@adamsitnik
Copy link
Member

Failed again, from console logs:

Console log: 'System.Diagnostics.Process.Tests' from job 5ea0f2fd-833d-4508-8e58-b565b6a4f16d workitem ee26e64c-d3e5-4cef-a3de-c90c6a5816d1 (redhat.7.amd64.open.rt) executed on machine a000ZXN
+ ./RunTests.sh --runtime-path /mnt/work/B0FD097F/p
----- start Fri Mar 25 21:06:19 UTC 2022 =============== To repro directly: =====================================================
pushd .
/mnt/work/B0FD097F/p/dotnet exec --runtimeconfig System.Diagnostics.Process.Tests.runtimeconfig.json --depsfile System.Diagnostics.Process.Tests.deps.json xunit.console.dll System.Diagnostics.Process.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing 
popd
===========================================================================================================
/mnt/work/B0FD097F/w/C1C10A38/e /mnt/work/B0FD097F/w/C1C10A38/e
  Discovering: System.Diagnostics.Process.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Diagnostics.Process.Tests (found 257 of 314 test cases)
  Starting:    System.Diagnostics.Process.Tests (parallel test collections = on, max threads = 2)
    System.Diagnostics.Tests.ProcessStartInfoTests.ShellExecute_Nano_Fails_Start [SKIP]
      Condition(s) not met: "IsWindowsNanoServer"
Linux
Linux
Linux
Linux
xdg-open: file '/nosuchfile' does not exist
    System.Diagnostics.Tests.ProcessTests.ProcessNameMatchesScriptName [FAIL]
      Assert.Equal() Failure
                ↓ (pos 0)
      Expected: /bin/sh\0/tmp/ProcessTests_dcap2jdw.jl4/Pr···
      Actual:   
                ↑ (pos 0)
      Stack Trace:
        /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs(177,0): at System.Diagnostics.Tests.ProcessTests.ProcessNameMatchesScriptName()
    System.Diagnostics.Tests.ProcessTests.TestProcessRecycledPid [SKIP]

@danmoseley
Copy link
Member

@tmds any idea why we're occasionally not successfully getting a process name on Linux?

@tmds
Copy link
Member

tmds commented Apr 4, 2022

Do we have some data on how long the test runs in case it fails?

In particular: is it about the duration of the script that we're running as the child process?

File.WriteAllText(filename, $"#!/bin/sh\nsleep 600\n"); // sleep 10 min.

@carlossanlop
Copy link
Member

@dotnet/area-system-diagnostics-process this issue hit the rolling builds today, specifically the Libraries Test Run release mono Linux x64 Release pipeline : https://dev.azure.com/dnceng/public/_build/results?buildId=1705722&view=results

System.Diagnostics.Tests.ProcessTests.ProcessNameMatchesScriptName [FAIL]
      Assert.Equal() Failure
                ↓ (pos 0)
      Expected: /bin/sh\0/tmp/ProcessTests_54oj45rb.rvr/Pr···
      Actual:   
                ↑ (pos 0)
      Stack Trace:
        /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs(177,0): at System.Diagnostics.Tests.ProcessTests.ProcessNameMatchesScriptName()
        /_/src/mono/System.Private.CoreLib/src/System/Reflection/RuntimeMethodInfo.Mono.cs(386,0): at System.Reflection.RuntimeMethodInfo.InvokeWorker(Object obj, BindingFlags invokeAttr, Span`1 parameters)

@carlossanlop carlossanlop added the blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' label Apr 8, 2022
@danmoseley
Copy link
Member

To answer @tmds question, these seem to be finishing quickly:

TestResults
| join kind=inner WorkItems on WorkItemId
| join kind=inner Jobs on JobId
| where Method == "ProcessNameMatchesScriptName"
| where Finished >= now(-30d)
| where Result == "Fail"
| where Message startswith "Assert.Equal() Failure" //and Message contains "···"
| project Type, Method,
  Pipeline = tostring(parse_json(Properties).DefinitionName),//WorkItemFriendlyName,
  Pipeline_Configuration = tostring(parse_json(Properties).configuration),
  OS = QueueName,
  Arch = tostring(parse_json(Properties).architecture),
 // Test = Type1,
 //Result,
 Finished,
 Build,//, WorkItemFriendlyName, WorkItemName,
  Duration,
 // Method,
 // Build = tostring(parse_json(Properties).BuildNumber),
   Message//,//
 ,StackTrace
Type Method Pipeline Pipeline_Configuration OS Arch Finished Build Duration Message StackTrace
System.Diagnostics.Tests.ProcessTests ProcessNameMatchesScriptName runtime Debug ubuntu.1604.amd64.open.rt x64 2022-03-21 10:17:23.5910000 0.007 Assert.Equal() Failure
↓ (pos 0)
Expected: /bin/sh\0/tmp/ProcessTests_2w1u51xv.llx/Pr···
Actual:
↑ (pos 0)
at System.Diagnostics.Tests.ProcessTests.ProcessNameMatchesScriptName() in /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs:line 177
System.Diagnostics.Tests.ProcessTests ProcessNameMatchesScriptName runtime Release redhat.7.amd64.open.rt x64 2022-03-25 21:06:45.4410000 0.004 Assert.Equal() Failure
↓ (pos 0)
Expected: /bin/sh\0/tmp/ProcessTests_dcap2jdw.jl4/Pr···
Actual:
↑ (pos 0)
at System.Diagnostics.Tests.ProcessTests.ProcessNameMatchesScriptName() in /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs:line 177
System.Diagnostics.Tests.ProcessTests ProcessNameMatchesScriptName runtime Debug ubuntu.1804.amd64.open.rt x64 2022-03-25 23:42:18.8780000 0.004 Assert.Equal() Failure
↓ (pos 0)
Expected: /bin/sh\0/tmp/ProcessTests_bg1j300a.kye/Pr···
Actual:
↑ (pos 0)
at System.Diagnostics.Tests.ProcessTests.ProcessNameMatchesScriptName() in /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs:line 177
System.Diagnostics.Tests.ProcessTests ProcessNameMatchesScriptName runtime Debug ubuntu.1604.amd64.open.rt x64 2022-04-02 12:34:26.0670000 0.004 Assert.Equal() Failure
↓ (pos 0)
Expected: /bin/sh\0/tmp/ProcessTests_tf1otouy.3xk/Pr···
Actual:
↑ (pos 0)
at System.Diagnostics.Tests.ProcessTests.ProcessNameMatchesScriptName() in /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs:line 177
System.Diagnostics.Tests.ProcessTests ProcessNameMatchesScriptName runtime Debug ubuntu.1804.amd64.open.rt x64 2022-04-05 21:09:29.6060000 0.063 Assert.Equal() Failure
↓ (pos 0)
Expected: /bin/sh\0/tmp/ProcessTests_5gj4kzdy.jia/Pr···
Actual:
↑ (pos 0)
at System.Diagnostics.Tests.ProcessTests.ProcessNameMatchesScriptName() in //src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs:line 177
at System.Reflection.RuntimeMethodInfo.InvokeWorker(Object obj, BindingFlags invokeAttr, Span`1 parameters) in /
/src/mono/System.Private.CoreLib/src/System/Reflection/RuntimeMethodInfo.Mono.cs:line 386
System.Diagnostics.Tests.ProcessTests ProcessNameMatchesScriptName runtime Release ubuntu.1804.amd64.open.rt x64 2022-04-07 21:17:47.2680000 0.006 Assert.Equal() Failure
↓ (pos 0)
Expected: /bin/sh\0/tmp/ProcessTests_54oj45rb.rvr/Pr···
Actual:
↑ (pos 0)
at System.Diagnostics.Tests.ProcessTests.ProcessNameMatchesScriptName() in //src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs:line 177
at System.Reflection.RuntimeMethodInfo.InvokeWorker(Object obj, BindingFlags invokeAttr, Span`1 parameters) in /
/src/mono/System.Private.CoreLib/src/System/Reflection/RuntimeMethodInfo.Mono.cs:line 386

@danmoseley
Copy link
Member

@tmds any other idea here? Not sure how to look next.

@danmoseley
Copy link
Member

danmoseley commented Jun 6, 2022

Failing about every 2 days. @agocke @steveisok what is "windows.10.amd64.android.open" ? From the message, it's clearly not Windows.

TestResults
| join kind=inner WorkItems on WorkItemId
| join kind=inner Jobs on JobId
| where Method == "ProcessNameMatchesScriptName"
| where Finished >= now(-30d)
| where Result == "Fail"
| where Message startswith "Assert.Equal() Failure" //and Message contains "···"
| project Type, Method,
  Pipeline = tostring(parse_json(Properties).DefinitionName),//WorkItemFriendlyName,
  Pipeline_Configuration = tostring(parse_json(Properties).configuration),
  OS = QueueName,
  Arch = tostring(parse_json(Properties).architecture),
 // Test = Type1,
 //Result,
 Finished,//,Message,
 Build,//, WorkItemFriendlyName, WorkItemName,
  Duration//,
 // Method,
 // Build = tostring(parse_json(Properties).BuildNumber),
 //,StackTrace
Type Method Pipeline Pipeline_Configuration OS Arch Finished Build Duration
System.Diagnostics.Tests.ProcessTests ProcessNameMatchesScriptName runtime Debug ubuntu.1804.amd64.open.rt x64 2022-05-13 17:07:23.0780000 0.024
System.Diagnostics.Tests.ProcessTests ProcessNameMatchesScriptName runtime Debug ubuntu.1804.amd64.open.rt x64 2022-05-13 22:42:10.6780000 0.003
System.Diagnostics.Tests.ProcessTests ProcessNameMatchesScriptName runtime Debug ubuntu.1804.amd64.open.rt x64 2022-05-19 18:28:44.6890000 0.01
System.Diagnostics.Tests.ProcessTests ProcessNameMatchesScriptName runtime Debug ubuntu.1804.amd64.open.rt x64 2022-05-20 04:23:05.4560000 0.006
System.Diagnostics.Tests.ProcessTests ProcessNameMatchesScriptName runtime Debug ubuntu.1804.amd64.open.rt x64 2022-05-21 13:25:33.7590000 0.007
System.Diagnostics.Tests.ProcessTests ProcessNameMatchesScriptName runtime-staging Release windows.10.amd64.android.open arm64 2022-05-24 19:52:12.1370000 0.03
System.Diagnostics.Tests.ProcessTests ProcessNameMatchesScriptName runtime-staging Release windows.10.amd64.android.open arm64 2022-05-25 17:37:48.9960000 0.02
System.Diagnostics.Tests.ProcessTests ProcessNameMatchesScriptName runtime Debug ubuntu.1604.amd64.open.rt x64 2022-05-29 09:50:10.8050000 0.007
System.Diagnostics.Tests.ProcessTests ProcessNameMatchesScriptName runtime-coreclr libraries-jitstress Release ubuntu.1804.amd64.open.rt x64 2022-05-31 16:40:28.1310000 0.006
System.Diagnostics.Tests.ProcessTests ProcessNameMatchesScriptName runtime-extra-platforms Release ubuntu.1804.amd64.open.svc x64 2022-06-01 08:59:55.8300000 0.003
System.Diagnostics.Tests.ProcessTests ProcessNameMatchesScriptName runtime-staging Release windows.10.amd64.android.open arm64 2022-06-01 10:46:23.1430000 0.05
System.Diagnostics.Tests.ProcessTests ProcessNameMatchesScriptName runtime Debug ubuntu.1804.amd64.open.rt x64 2022-06-03 01:44:40.2080000 0.003

@agocke
Copy link
Member

agocke commented Jun 6, 2022

My guess was this is the Android emulation layer in Windows 10, but I don't understand why we would be running coreclr in that environment.

@steveisok
Copy link
Member

Failing about every 2 days. @agocke @steveisok what is "windows.10.amd64.android.open" ? From the message, it's clearly not Windows.

The queue is for Android devices. They are tethered to a windows machine.

@directhex do you have any insight on why this may be flaky on Android?

@tmds
Copy link
Member

tmds commented Jun 8, 2022

@tmds any other idea here? Not sure how to look next.

From the tables that got shared, the distros seem to be 'old'. Does it happen on newer kernels?

@danmoseley
Copy link
Member

Assuming I'm doing this right, mostly the tests run in containers on the same kernel, either Ubuntu 18.04 or 16.04. @MattGal @agocke am I reading this right? Have we considered running containers on newer kernels?

Also, do we use eg Ubuntu 18.04 with its original kernel (4.15) or do we use the refreshed versions that pick up newer kernels (https://ubuntu.com/kernel/lifecycle)

TestResults
| join kind=inner WorkItems on WorkItemId
| join kind=inner Jobs on JobId
| where Method == "ProcessNameMatchesScriptName"
| where Finished >= now(-30d)
| extend IsOneOfTheseFailures = (Result == "Fail" and Message startswith "Assert.Equal() Failure")
| project IsOneOfTheseFailures,Type, Method,
  Pipeline = tostring(parse_json(Properties).DefinitionName),//WorkItemFriendlyName,
  Pipeline_Configuration = tostring(parse_json(Properties).configuration),
  Container = tostring(parse_json(Properties).operatingSystem),
  Kernel = QueueName,
  Arch = tostring(parse_json(Properties).architecture),
 // Test = Type1,
 //Result,
 Finished,
 Build,//, WorkItemFriendlyName, WorkItemName,
  Duration,
 // Method,
 // Build = tostring(parse_json(Properties).BuildNumber),
   Message//,//
 ,StackTrace
 | summarize count() by Kernel, Container, IsOneOfTheseFailures
Kernel Container IsOneOfTheseFailures count_
ubuntu.1804.amd64.open.rt (Debian.10.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-10-helix-amd64-20210304164434-56c6673 1 4
ubuntu.1604.amd64.open.svc (Alpine.312.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.12-helix-20211214164128-aca80d8 0 9
ubuntu.1604.amd64.open.svc (Alpine.314.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.14-helix-amd64-20210910135833-1848e19 0 9
sles.12.amd64.open.svc SLES.12.Amd64.Open 0 10
centos.7.amd64.open.svc Centos.7.Amd64.Open 0 10
ubuntu.1604.amd64.open.svc (Fedora.29.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-29-helix-a12566d-20191210224553 0 8
ubuntu.1604.amd64.open.svc (Fedora.30.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-30-helix-4f8cef7-20200121150022 0 8
ubuntu.1604.amd64.open.svc Ubuntu.1604.Amd64.Open 0 10
ubuntu.1804.amd64.open.svc Ubuntu.1804.Amd64.Open 0 10
ubuntu.1604.amd64.open.svc (Ubuntu.1910.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-19.10-helix-amd64-cfcfd50-20191030180623 0 10
redhat.7.amd64.open.svc RedHat.7.Amd64.Open 0 10
debian.9.amd64.open.svc Debian.9.Amd64.Open 0 11
sles.15.amd64.open.svc SLES.15.Amd64.Open 0 10
ubuntu.1804.armarch.open (Alpine.313.Arm64.Open)ubuntu.1804.armarch.open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.13-helix-arm64v8-20210910135808-8a6f4f3 0 9
ubuntu.1604.amd64.open.svc (Debian.10.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-10-helix-amd64-bfcd90a-20200121150006 0 10
ubuntu.1804.armarch.open (Debian.9.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-9-helix-arm32v7-bfcd90a-20200121150037 0 8
ubuntu.1804.armarch.open (Ubuntu.1604.Arm64.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-16.04-helix-arm64v8-bfcd90a-20200127194925 0 8
ubuntu.1804.armarch.open (Alpine.314.Arm64.Open)ubuntu.1804.armarch.open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.14-helix-arm64v8-20210910135810-8a6f4f3 0 9
ubuntu.1604.amd64.open.svc (Fedora.30.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-30-helix-20200512010621-4f8cef7 0 2
ubuntu.1604.amd64.open.svc (Fedora.32.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-32-helix-20200512010618-efb9f14 0 2
ubuntu.1604.amd64.open.svc (Alpine.312.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.12-helix-20200601195603-e06dc59 0 1
ubuntu.1804.armarch.open (Ubuntu.1804.ArmArch.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-16.04-helix-arm64v8-bfcd90a-20200127194925 0 1
ubuntu.1804.amd64.android.29.open.rt Ubuntu.1804.Amd64.Android.29.Open 1 1
windows.10.amd64.android.open Windows.10.Amd64.Android.Open 1 3
ubuntu.1804.amd64.open.rt Ubuntu.1804.Amd64.Open 1 1
ubuntu.1804.amd64.open.svc (Alpine.313.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.13-helix-amd64-20210910135845-8a6f4f3 1 1
ubuntu.1604.amd64.open.rt (Centos.7.Amd64.Open)Ubuntu.1604.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-mlnet-helix-20210714125435-dde38af 1 1

@MattGal
Copy link
Member

MattGal commented Jun 15, 2022

@MattGal @agocke am I reading this right? Have we considered running containers on newer kernels?

That's correct; My understanding is the choice is yours, you could instead be using ubuntu.2004.amd64.* but aren't currently (no special .rt version of it, yet, since you don't use the other one), and we have not added 22.04 yet. If you want either thing (22.04 queue / a special 20.04 .rt queue) just file an issue in dotnet/arcade asking for it.

do we use eg Ubuntu 18.04 with its original kernel (4.15) or do we use the refreshed versions that pick up newer kernels

We strive to stay on the latest version published in the public Azure gallery; currently that's 18.04.202205270, or May 27th, so fairly fresh.

@danmoseley
Copy link
Member

OK, opened an issue, and we can throw another kernel version into our mix..

@buyaa-n
Copy link
Contributor

buyaa-n commented Jun 21, 2022

Failed again in https://github.com/dotnet/runtime/pull/71013/checks?check_run_id=6972717112:

System.Diagnostics.Tests.ProcessTests.ProcessNameMatchesScriptName [FAIL]
      Assert.Equal() Failure
                ↓ (pos 0)
      Expected: /bin/sh\0/tmp/#ProcessTests_gvzvm1lb.l0c/P···
      Actual:   
                ↑ (pos 0)
      Stack Trace:
        /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs(179,0): at System.Diagnostics.Tests.ProcessTests.ProcessNameMatchesScriptName()
           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
        /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodInvoker.cs(69,0): at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
    System.Diagnostics.Tests.ProcessTests.TestProcessRecycledPid [SKIP]
      Condition(s) not met: "IsStressModeEnabledAndRemoteExecutorSupported"
  Finished:    System.Diagnostics.Process.Tests
=== TEST EXECUTION SUMMARY ===
   System.Diagnostics.Process.Tests  Total: 322, Errors: 0, Failed: 1, Skipped: 4, Time: 21.929s

https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-71013-merge-cba83cce404640ce9e/System.Diagnostics.Process.Tests/1/console.4e45405d.log?helixlogtype=result

@karelz
Copy link
Member

karelz commented Jul 22, 2022

Frequency updated in top post ... lately it is 1 hit per week (in average 1-2 hits per week). Removing blocking-clean-ci label now.

@karelz karelz removed the blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' label Jul 22, 2022
@jeffhandley jeffhandley modified the milestones: 7.0.0, 8.0.0 Aug 9, 2022
@VSadov
Copy link
Member

VSadov commented Aug 14, 2022

Failed in #73912 in NativeAOT run on arm64

[FAIL] System.Diagnostics.Tests.ProcessTests.ProcessNameMatchesScriptName
Assert.Equal() Failure
          ↓ (pos 0)
Expected: /bin/sh\0/tmp/#ProcessTests_macg3lw5.q45/P···
Actual:   
          ↑ (pos 0)
   at System.Diagnostics.Tests.ProcessTests.ProcessNameMatchesScriptName() + 0x3bc
   at System.Diagnostics.Process!<BaseAddress>+0x105aad4
   at System.Reflection.DynamicInvokeInfo.Invoke(Object, IntPtr, Object[], BinderBundle, Boolean) + 0xd8

@VSadov
Copy link
Member

VSadov commented Aug 14, 2022

And here - failed again:

https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-73912-merge-6a50d560c07e4c518e/System.Diagnostics.Process.Tests/1/console.ee9855dd.log?helixlogtype=result

[FAIL] System.Diagnostics.Tests.ProcessTests.ProcessNameMatchesScriptName
Assert.Equal() Failure
          ↓ (pos 0)
Expected: /bin/sh\0/tmp/#ProcessTests_x5rcuvjz.evg/P···
Actual:   
          ↑ (pos 0)
   at System.Diagnostics.Tests.ProcessTests.ProcessNameMatchesScriptName() + 0x3bc
   at System.Diagnostics.Process!<BaseAddress>+0x105aa34
   at System.Reflection.DynamicInvokeInfo.Invoke(Object, IntPtr, Object[], BinderBundle, Boolean) + 0xd8

This is on (ubuntu.1804.armarch.open) using docker image mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm64v8-20220427172132-97d8652

@danmoseley
Copy link
Member

Is it happening at all on Ubuntu 2204 (newer kernel)?

@jkotas jkotas added the disabled-test The test is disabled in source code against the issue label Aug 15, 2022
@MichalStrehovsky
Copy link
Member

It's possible this fails a lot more often with NativeAOT. I've seen it fail in several of my PR's as well. Or we just send it to a different Helix queue for NativeAOT testing. But with NativeAOT it reproes in a significant percentage of runs. I would estimate one out of five runs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arch-x64 area-System.Diagnostics.Process disabled-test The test is disabled in source code against the issue os-linux Linux OS (any supported distro) test-bug Problem in test source code (most likely)
Projects
None yet
Development

Successfully merging a pull request may close this issue.