NUnit 3 / UITest failing with error: Exception has been thrown by the target of an invocation #421

jpc1963 · 2019-05-15T08:45:09Z

What App Center service does this affect?
testing on ios simulators / android emulators

Describe the bug
Hi I have a Xamarin UI test project with tests apps on iOS and Android simulators. I have recently updated nunit & xamarin.uitest to the latest versions. Now when I start my app on a simulator I get the error: Exception has been thrown by the target of an invocation

To Reproduce
I start the app as follows: (the app is started on the simulator - but then the test crashes)

IApp myApp =  ConfigureApp
            .iOS
            .InstalledApp(pathToApp)
            .DeviceIdentifier(123456-123456)
            .PreferIdeSettings()
            .EnableLocalScreenshots()
            .WaitTimes(new WaitTimes())
            .Debug()
            .StartApp(ClearAppData);

Expected behavior
With the app now visible and running I expect to be able to test it. With the variable myApp I should be able to run my tests. But the error occurs in the ConfigureApp and the variable myApp remains null.

Screenshots
none

Desktop (please complete the following information):

OS: Mac OS 10.14.4
Visual Studio for Mac: 8.0.5

Additional context
Nuget packages:

Nunit: 3.11
NUnit console runner: 3.10
Specflow 3.0.213
Xamarin UITest 3.0.0
xCode 10.2.1

The following is the log output after trying to start an ios app:

1 - Exception
Exception: System.Threading.ThreadAbortException: Thread was being aborted.
at (wrapper managed-to-native) System.Threading.Monitor.Monitor_wait(object,int)
at System.Threading.Monitor.ObjWait (System.Boolean exitContext, System.Int32 millisecondsTimeout, System.Object obj) [0x0002f] in /Users/builder/jenkins/workspace/build-package-osx-mono/2018-08/external/bockbuild/builds/mono-x64/mcs/class/corlib/System.Threading/Monitor.cs:83
at System.Threading.Monitor.Wait (System.Object obj, System.Int32 millisecondsTimeout, System.Boolean exitContext) [0x0000e] in /Users/builder/jenkins/workspace/build-package-osx-mono/2018-08/external/bockbuild/builds/mono-x64/mcs/class/referencesource/mscorlib/system/threading/monitor.cs:210
at System.Threading.Monitor.Wait (System.Object obj, System.Int32 millisecondsTimeout) [0x00000] in /Users/builder/jenkins/workspace/build-package-osx-mono/2018-08/external/bockbuild/builds/mono-x64/mcs/class/referencesource/mscorlib/system/threading/monitor.cs:220
at System.Threading.ManualResetEventSlim.Wait (System.Int32 millisecondsTimeout, System.Threading.CancellationToken cancellationToken) [0x00141] in /Users/builder/jenkins/workspace/build-package-osx-mono/2018-08/external/bockbuild/builds/mono-x64/mcs/class/referencesource/mscorlib/system/threading/ManualResetEventSlim.cs:669
at System.Threading.Tasks.Task.WaitAllBlockingCore (System.Collections.Generic.LowLevelListWithIList 1[T] tasks, System.Int32 millisecondsTimeout, System.Threading.CancellationToken cancellationToken) [0x00038] in /Users/builder/jenkins/workspace/build-package-osx-mono/2018-08/external/bockbuild/builds/mono-x64/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:4703
at System.Threading.Tasks.Task.WaitAll (System.Threading.Tasks.Task[] tasks, System.Int32 millisecondsTimeout, System.Threading.CancellationToken cancellationToken) [0x000e6] in /Users/builder/jenkins/workspace/build-package-osx-mono/2018-08/external/bockbuild/builds/mono-x64/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:4619
at System.Threading.Tasks.Task.WaitAll (System.Threading.Tasks.Task[] tasks, System.Int32 millisecondsTimeout) [0x00000] in /Users/builder/jenkins/workspace/build-package-osx-mono/2018-08/external/bockbuild/builds/mono-x64/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:4482
at System.Threading.Tasks.Task.WaitAll (System.Threading.Tasks.Task[] tasks) [0x00000] in /Users/builder/jenkins/workspace/build-package-osx-mono/2018-08/external/bockbuild/builds/mono-x64/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:4405
at Xamarin.UITest.XDB.Services.Processes.ProcessService.Run (System.String command, System.String arguments) [0x0004f] in <18ae7883e2424c558186d1d9edf9f14b>:0
at Xamarin.UITest.XDB.Services.OSX.XcodeService.TestWithoutBuilding (System.String deviceId, System.String xctestrunPath, System.String derivedDataPath) [0x0002b] in <18ae7883e2424c558186d1d9edf9f14b>:0
at Xamarin.UITest.XDB.Services.OSX.iOSDeviceManagerService.StartTest (System.String deviceId) [0x00022] in <18ae7883e2424c558186d1d9edf9f14b>:0
at Xamarin.UITest.XDB.Services.iOSDeviceAgentService+<>c__DisplayClass17_0.b__2 () [0x00000] in <18ae7883e2424c558186d1d9edf9f14b>:0
at Xamarin.UITest.XDB.Services.iOSDeviceAgentService.WithErrorHandling (System.Int32 eventId, System.Func 1[TResult] action, System.String errorMessage, System.Int32[] successCodes) [0x0000e] in <18ae7883e2424c558186d1d9edf9f14b>:0

The text was updated successfully, but these errors were encountered:

jeromelaban · 2019-06-21T02:45:11Z

I'm also having this issue. Is there a known workaround for it ?

NUnit 3 is increasingly becoming a blocking feature, as NUnit 2 test runner support is very unstable in VS 2019.

ahanag22 · 2019-06-21T19:02:39Z

@jeromelaban @jpc1963 Are you getting the same error if you disable the local screenshots?
I'm mainly asking this because we already have an existing bug filed here #563, SpecFlow version 3 on App Center Test doesn't display per-step screenshot.

jeromelaban · 2019-06-21T19:11:59Z

The behavior is the same with or without local screenshots.

Also, it might be of interest that it seems that this exception appers on the unit tests side:

System.Net.Http.HttpRequestException : Connection refused
  ----> System.Net.Sockets.SocketException : Connection refused
TearDown : System.NullReferenceException : Object reference not set to an instance of an object
  at System.Net.Http.ConnectHelper.ConnectAsync (System.String host, System.Int32 port, System.Threading.CancellationToken cancellationToken) [0x001ac] in <d557f813f52a4dafa05dcf2b14802e7d>:0
  at System.Net.Http.HttpConnectionPool.CreateConnectionAsync (System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken) [0x00134] in <d557f813f52a4dafa05dcf2b1                                     4802e7d>:0
  at System.Net.Http.HttpConnectionPool.WaitForCreatedConnectionAsync (System.Threading.Tasks.ValueTask`1[TResult] creationTask) [0x000a2] in <d557f813f52a4dafa05dcf2b14802e7d>:0
  at System.Net.Http.HttpConnectionPool.SendWithRetryAsync (System.Net.Http.HttpRequestMessage request, System.Boolean doRequestAuth, System.Threading.CancellationToken cancellationToken) [0x00089] in                                      <d557f813f52a4dafa05dcf2b14802e7d>:0
  at System.Net.Http.RedirectHandler.SendAsync (System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken) [0x000ba] in <d557f813f52a4dafa05dcf2b14802e7d>:0
  at Xamarin.UITest.Shared.Http.HttpClient.Request (System.String method, System.String endpoint, Xamarin.UITest.Shared.Http.ExceptionPolicy exceptionPolicy, System.Nullable`1[T] timeOut) [0x000f6] in                                      <18ae7883e2424c558186d1d9edf9f14b>:0
  at Xamarin.UITest.Shared.Http.HttpClient.Get (System.String endpoint, Xamarin.UITest.Shared.Http.ExceptionPolicy exceptionPolicy, System.Nullable`1[T] timeOut) [0x00000] in <18ae7883e2424c558186d1d9                                     edf9f14b>:0
  at Xamarin.UITest.iOS.HttpCalabashConnection.Version () [0x0001f] in <18ae7883e2424c558186d1d9edf9f14b>:0
  at Xamarin.UITest.iOS.iOSAppLauncher.GetCalabashDevice (Xamarin.UITest.iOS.ICalabashConnection connection) [0x00075] in <18ae7883e2424c558186d1d9edf9f14b>:0
  at Xamarin.UITest.iOS.iOSAppLauncher.EnsureCalabashRunning (Xamarin.UITest.iOS.ICalabashConnection connection) [0x00000] in <18ae7883e2424c558186d1d9edf9f14b>:0
  at Xamarin.UITest.iOS.iOSAppLauncher.LaunchApp (Xamarin.UITest.Configuration.IiOSAppConfiguration appConfiguration, Xamarin.UITest.Shared.Http.HttpClient httpClient, Xamarin.UITest.TestCloud.TestClo                                     udiOSAppConfiguration testCloudAppConfiguration, Xamarin.UITest.Shared.Http.HttpClient testCloudWsClient, Xamarin.UITest.Shared.Http.HttpClient xtcServicesClient, System.Boolean testCloudUseDeviceAgen                                     t) [0x0007f] in <18ae7883e2424c558186d1d9edf9f14b>:0
  at Xamarin.UITest.iOS.iOSApp..ctor (Xamarin.UITest.Configuration.IiOSAppConfiguration appConfiguration, Xamarin.UITest.Shared.Execution.IExecutor executor) [0x0028d] in <18ae7883e2424c558186d1d9edf9                                     f14b>:0
  at Xamarin.UITest.iOS.iOSApp..ctor (Xamarin.UITest.Configuration.IiOSAppConfiguration appConfiguration) [0x00000] in <18ae7883e2424c558186d1d9edf9f14b>:0
  at Xamarin.UITest.Configuration.iOSAppConfigurator.StartApp (Xamarin.UITest.Configuration.AppDataMode appDataMode) [0x00017] in <18ae7883e2424c558186d1d9edf9f14b>:0
  at SamplesApp.UITests.AppInitializer.CreateiOSApp () [0x0003a] in <a01f077c2d344485a748a1ff1dd7b04f>:0
  at SamplesApp.UITests.AppInitializer.StartApp () [0x00056] in <a01f077c2d344485a748a1ff1dd7b04f>:0
  at Sample.UITests.TestBase.StartApp () [0x00000] in <a01f077c2d344485a748a1ff1dd7b04f>:0
  at (wrapper managed-to-native) System.Reflection.RuntimeMethodInfo.InternalInvoke(System.Reflection.RuntimeMethodInfo,object,object[],System.Exception&)
  at System.Reflection.RuntimeMethodInfo.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.Culture                                     Info culture) [0x0006a] in <63a185bccd644d7ea7199ca475c500f2>:0
--SocketException
  at System.Net.Http.ConnectHelper.ConnectAsync (System.String host, System.Int32 port, System.Threading.CancellationToken cancellationToken) [0x00110] in <d557f813f52a4dafa05dcf2b14802e7d>:0
--TearDown
  at Sample.UITests.TestBase.CloseApp () [0x00006] in <a01f077c2d344485a748a1ff1dd7b04f>:0
  at (wrapper managed-to-native) System.Reflection.RuntimeMethodInfo.InternalInvoke(System.Reflection.RuntimeMethodInfo,object,object[],System.Exception&)
  at System.Reflection.RuntimeMethodInfo.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.Culture                                     Info culture) [0x0006a] in <63a185bccd644d7ea7199ca475c500f2>:0

ahanag22 · 2019-06-21T19:27:12Z

@jeromelaban Are you using Xamarin.TestCloud.Agent latest version 0.21.8 in your iOS app?

jeromelaban · 2019-06-21T19:31:14Z

Yes, exactly this version, and Xamarin.UITest 3.0.0.

ahanag22 · 2019-06-21T22:24:32Z

Thanks for confirming. I'll try to reproduce this behavior on my end. Meanwhile, can you try this sample app & UITest project?https://github.com/King-of-Spades/AppCenter-Samples/tree/master/Xamarin.UITest/UITestDemo/iOS -
please let me know if that project works for you? If the sample project doesn't work for you, does it hit the same error, or something else?

jeromelaban · 2019-06-21T22:51:10Z

The code I'm using is open source, so I can also provide you with a repro. I'll prepare that and test with that. Thanks!

jeromelaban · 2019-06-21T23:00:53Z

I've having the exact same issue with your sources. The simulator starts multiple times, then the uitests fail while the app starts.

How can I help you troubleshoot that further ? I'm using VS4mac 8.1 Preview (build 2460)

stoneman · 2019-06-24T11:05:44Z

I get the System.Net.Http.HttpRequestException : Connection refused error running the tests in https://github.com/King-of-Spades/AppCenter-Samples/tree/master/Xamarin.UITest/UITestDemo/iOS when using VS for Mac 8.2 Preview (build 732). They work fine if I use the stable version of VS for Mac.

This will be investigated and someone will let you know when it's been solved but for now I would suggest using VS for Mac stable if possible.

jeromelaban · 2019-06-25T02:25:52Z

@JonStoneman Thanks for the followup! Unfortunately, I cannot do this, as there are specifics in the way VS4Mac has improved (particularly with regards to nuget support for cross-targeted projects) that enable key scenarios.

It was previously a very complex task for us to test applications that are built inside of a cross-targeted framework (those needed to be built with VS4Win) and those VS4Mac updates enable that. This is the last key piece that will make this UI Testing scenario possible for us.

Thanks!

adamzucchi · 2019-07-22T21:28:52Z

I am currently running VS for Mac 8.1.5 (Stable Channel) and am experiencing this as well. Is there a fix and/or workaround? Thanks!

jeromelaban · 2019-07-23T15:30:44Z

I've been poking at this issue again, and the latest 3.0.1-dev2 along with VS4Mac 8.2 (build 1090) is getting farther along, but is now giving errors such as this one:

Unable to scan assembly: nunit.engine, Version=3.9.0.0, Culture=neutral, PublicKeyToken=2638cd05610744eb
Exception: System.IO.FileNotFoundException: Could not load file or assembly 'Mono.Cecil, Version=0.9.6.0, Culture=neutral, PublicKeyToken=0738eb9f132ed756' or one of its dependencies.
File name: 'Mono.Cecil, Version=0.9.6.0, Culture=neutral, PublicKeyToken=0738eb9f132ed756'
  at (wrapper managed-to-native) System.Reflection.MonoMethodInfo.get_method_info(intptr,System.Reflection.MonoMethodInfo&)
  at System.Reflection.MonoMethodInfo.GetMethodInfo (System.IntPtr handle) [0x00000] in <db6d3f68ee4f4e0ab4ffdf0911bd3e33>:0
  at System.Reflection.RuntimeMethodInfo.GetPseudoCustomAttributes () [0x00002] in <db6d3f68ee4f4e0ab4ffdf0911bd3e33>:0
  at System.MonoCustomAttrs.GetPseudoCustomAttributes (System.Reflection.ICustomAttributeProvider obj, System.Type attributeType) [0x0000c] in <db6d3f68ee4f4e0ab4ffdf0911bd3e33>:0
  at System.MonoCustomAttrs.GetCustomAttributesBase (System.Reflection.ICustomAttributeProvider obj, System.Type attributeType, System.Boolean inheritedOnly) [0x0001f] in <db6d3f68ee4f4e0ab4ffdf0911bd3e33>:0
  at System.MonoCustomAttrs.GetCustomAttributes (System.Reflection.ICustomAttributeProvider obj, System.Type attributeType, System.Boolean inherit) [0x00037] in <db6d3f68ee4f4e0ab4ffdf0911bd3e33>:0
  at System.Reflection.RuntimeMethodInfo.GetCustomAttributes (System.Type attributeType, System.Boolean inherit) [0x00000] in <db6d3f68ee4f4e0ab4ffdf0911bd3e33>:0
  at System.Attribute.GetCustomAttributes (System.Reflection.MemberInfo element, System.Type type, System.Boolean inherit) [0x00088] in <db6d3f68ee4f4e0ab4ffdf0911bd3e33>:0
  at System.Attribute.GetCustomAttributes (System.Reflection.MemberInfo element, System.Type type) [0x00000] in <db6d3f68ee4f4e0ab4ffdf0911bd3e33>:0
  at System.Reflection.CustomAttributeExtensions.GetCustomAttributes (System.Reflection.MemberInfo element, System.Type attributeType) [0x00000] in <db6d3f68ee4f4e0ab4ffdf0911bd3e33>:0
  at Xamarin.UITest.Events.EventManager+<>c__DisplayClass2_0.<GetStaticMethodsWithAttribute>b__2 (<>f__AnonymousType3`2[<type>j__TPar,<method>j__TPar] <>h__TransparentIdentifier0) [0x00006] in <8139e2cb182043debeabb4fdea6d573a>:0
  at System.Linq.Enumerable+<SelectManyIterator>d__167`3[TSource,TCollection,TResult].MoveNext () [0x0004e] in <5b3a07cbef83409ea3183d1ea9f1b76a>:0
  at System.Collections.Generic.List`1[T].AddEnumerable (System.Collections.Generic.IEnumerable`1[T] enumerable) [0x00059] in <db6d3f68ee4f4e0ab4ffdf0911bd3e33>:0
  at System.Collections.Generic.List`1[T].InsertRange (System.Int32 index, System.Collections.Generic.IEnumerable`1[T] collection) [0x000f4] in <db6d3f68ee4f4e0ab4ffdf0911bd3e33>:0
  at System.Collections.Generic.List`1[T].AddRange (System.Collections.Generic.IEnumerable`1[T] collection) [0x00000] in <db6d3f68ee4f4e0ab4ffdf0911bd3e33>:0
  at Xamarin.UITest.Events.EventManager.GetStaticMethodsWithAttribute (System.Type attributeType) [0x000bb] in <8139e2cb182043debeabb4fdea6d573a>:0

jeromelaban · 2019-07-24T18:55:40Z

Also, now that VS 8.2 is out, it'll become critical for this to work.

owenniblock · 2019-07-25T12:10:20Z

This seems to be related to the version of mono - if I change my mono version:

Then UITest works correctly for me locally.

I'm not sure at what mono version. this started failing (5.18.1 was just a version I happened to have on my machine) but as a workaround for now I would try an earlier mono while we. investigate.

owenniblock · 2019-07-25T12:11:14Z

@jeromelaban - we're aware of the Cecil issue. Hoping to have a new dev release out with a fix soon.

stoneman · 2019-07-25T12:52:52Z

@jeromelaban: UITest 3.0.2-dev1 has been released which should fix the Cecil issue: https://www.nuget.org/packages/Xamarin.UITest/3.0.2-dev1

jeromelaban · 2019-07-25T12:58:27Z

@JonStoneman thanks! Actually, the latest I mentioned uses 3.0.1-dev2, so the issue seems to still be present. Preview bits (dev1 included) do not work at all for me (connection issues), dev2 is the only one that goes farther along but fails for other reasons.

brminnick · 2019-07-25T16:31:52Z

I am experiencing a similar issue, including on Xamarin.UITest 3.0.2-dev1.

This exception only appears on Xamarin.iOS; Xamarin.Android is working fine.

Reproduction Sample

https://github.com/brminnick/InvestmentDataSampleApp

Stack Trace

at System.Net.Http.ConnectHelper.ConnectAsync (System.String host, System.Int32 port, System.Threading.CancellationToken cancellationToken) [0x00180] in /Users/builder/jenkins/workspace/build-package-osx-mono/2019-02/external/bockbuild/builds/mono-x64/external/corefx/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/ConnectHelper.cs:84
at System.Net.Http.HttpConnectionPool.CreateConnectionAsync (System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken) [0x000ea] in /Users/builder/jenkins/workspace/build-package-osx-mono/2019-02/external/bockbuild/builds/mono-x64/external/corefx/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:371
at System.Net.Http.HttpConnectionPool.WaitForCreatedConnectionAsync (System.Threading.Tasks.ValueTask1[TResult] creationTask) [0x000a2] in /Users/builder/jenkins/workspace/build-package-osx-mono/2019-02/external/bockbuild/builds/mono-x64/external/corefx/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:529 at System.Net.Http.HttpConnectionPool.SendWithRetryAsync (System.Net.Http.HttpRequestMessage request, System.Boolean doRequestAuth, System.Threading.CancellationToken cancellationToken) [0x0003f] in /Users/builder/jenkins/workspace/build-package-osx-mono/2019-02/external/bockbuild/builds/mono-x64/external/corefx/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:284 at System.Net.Http.RedirectHandler.SendAsync (System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken) [0x00070] in /Users/builder/jenkins/workspace/build-package-osx-mono/2019-02/external/bockbuild/builds/mono-x64/external/corefx/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/RedirectHandler.cs:32 at Xamarin.UITest.Shared.Http.HttpClient.Request (System.String method, System.String endpoint, Xamarin.UITest.Shared.Http.ExceptionPolicy exceptionPolicy, System.Nullable1[T] timeOut) [0x000f6] in :0
at Xamarin.UITest.Shared.Http.HttpClient.Get (System.String endpoint, Xamarin.UITest.Shared.Http.ExceptionPolicy exceptionPolicy, System.Nullable`1[T] timeOut) [0x00000] in :0
at Xamarin.UITest.iOS.HttpCalabashConnection.Version () [0x0001f] in :0
at Xamarin.UITest.iOS.iOSAppLauncher.GetCalabashDevice (Xamarin.UITest.iOS.ICalabashConnection connection) [0x00075] in :0
at Xamarin.UITest.iOS.iOSAppLauncher.EnsureCalabashRunning (Xamarin.UITest.iOS.ICalabashConnection connection) [0x00000] in :0
at Xamarin.UITest.iOS.iOSAppLauncher.LaunchApp (Xamarin.UITest.Configuration.IiOSAppConfiguration appConfiguration, Xamarin.UITest.Shared.Http.HttpClient httpClient, Xamarin.UITest.TestCloud.TestCloudiOSAppConfiguration testCloudAppConfiguration, Xamarin.UITest.Shared.Http.HttpClient testCloudWsClient, Xamarin.UITest.Shared.Http.HttpClient xtcServicesClient, System.Boolean testCloudUseDeviceAgent) [0x0007f] in :0
at Xamarin.UITest.iOS.iOSApp..ctor (Xamarin.UITest.Configuration.IiOSAppConfiguration appConfiguration, Xamarin.UITest.Shared.Execution.IExecutor executor) [0x0026f] in :0
at Xamarin.UITest.iOS.iOSApp..ctor (Xamarin.UITest.Configuration.IiOSAppConfiguration appConfiguration) [0x00000] in :0
at Xamarin.UITest.Configuration.iOSAppConfigurator.StartApp (Xamarin.UITest.Configuration.AppDataMode appDataMode) [0x00017] in :0
at InvestmentDataSampleApp.UITests.AppInitializer.StartApp (Xamarin.UITest.Platform platform) [0x0001e] in /Users/bramin/GitHub/InvestmentDataSampleApp/Src/InvestmentDataSampleApp.UITests/AppInitializer.cs:16
at InvestmentDataSampleApp.UITests.BaseTest.TestSetup () [0x00001] in /Users/bramin/GitHub/InvestmentDataSampleApp/Src/InvestmentDataSampleApp.UITests/Tests/BaseTest.cs:28
at (wrapper managed-to-native) System.Reflection.RuntimeMethodInfo.InternalInvoke(System.Reflection.RuntimeMethodInfo,object,object[],System.Exception&)
at System.Reflection.RuntimeMethodInfo.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in /Users/builder/jenkins/workspace/build-package-osx-mono/2019-02/external/bockbuild/builds/mono-x64/mcs/class/corlib/System.Reflection/RuntimeMethodInfo.cs:391
--SocketException
at System.Net.Http.ConnectHelper.ConnectAsync (System.String host, System.Int32 port, System.Threading.CancellationToken cancellationToken) [0x000c8] in /Users/builder/jenkins/workspace/build-package-osx-mono/2019-02/external/bockbuild/builds/mono-x64/external/corefx/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/ConnectHelper.cs:65

Environment

=== Visual Studio Enterprise 2019 for Mac ===

Version 8.2 (build 1105)
Installation UUID: e8d39274-f49e-49c5-ac5a-5e89b4e131e6
GTK+ 2.24.23 (Raleigh theme)
Xamarin.Mac 5.14.0.85 (d16-2-fix-mac-updates / 86973979)

Package version: 600000311

=== Mono Framework MDK ===

Runtime:
Mono 6.0.0.311 (2019-02/494641b300c) (64-bit)
Package version: 600000311

=== NuGet ===

Version: 5.1.0.6013

=== .NET Core ===

Runtime: /usr/local/share/dotnet/dotnet
Runtime Versions:
3.0.0-preview7-27912-14
3.0.0-preview6-27804-01
2.1.11
SDK: /usr/local/share/dotnet/sdk/3.0.100-preview7-012821/Sdks
SDK Versions:
3.0.100-preview7-012821
3.0.100-preview6-012264
2.1.700
MSBuild SDKs: /Library/Frameworks/Mono.framework/Versions/6.0.0/lib/mono/msbuild/Current/bin/Sdks

=== Xamarin.Profiler ===

Version: 1.6.11
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

=== Updater ===

Version: 11

=== Apple Developer Tools ===

Xcode 10.3 (14492.2)
Build 10G8

=== Xamarin.Mac ===

Xamarin.Mac not installed. Can't find /Library/Frameworks/Xamarin.Mac.framework/Versions/Current/Version.

=== Xamarin.iOS ===

Version: 12.14.0.110 (Visual Studio Enterprise)
Hash: a8bcecce
Branch: d16-2
Build date: 2019-06-28 07:31:42-0400

=== Xamarin Designer ===

Version: 16.2.0.354
Hash: 5f80a14f8
Branch: remotes/origin/d16-2
Build date: 2019-07-11 17:28:26 UTC

=== Xamarin.Android ===

Version: 9.4.0.51 (Visual Studio Enterprise)
Commit: xamarin-android/d16-2/9fa7775
Android SDK: /Users/bramin/Library/Developer/Xamarin/android-sdk-macosx
Supported Android versions:
None installed

SDK Tools Version: 26.1.1
SDK Platform Tools Version: 29.0.1
SDK Build Tools Version: 28.0.3

Build Information:
Mono: mono/mono@e6f5369c2d2
Java.Interop: xamarin/java.interop/d16-2@d64ada5
LibZipSharp: grendello/LibZipSharp/d16-2@caa0c74
LibZip: nih-at/libzip@b95cf3f
ProGuard: xamarin/proguard@905836d
SQLite: xamarin/sqlite@8212a2d
Xamarin.Android Tools: xamarin/xamarin-android-tools/d16-2@6f6c969

=== Microsoft Mobile OpenJDK ===

Java SDK: /Users/bramin/Library/Developer/Xamarin/jdk/microsoft_dist_openjdk_1.8.0.25
1.8.0-25
Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

=== Android Device Manager ===

Version: 1.2.0.58
Hash: d03e3ea
Branch: remotes/origin/d16-2~1
Build date: 2019-07-16 17:37:14 UTC

=== Xamarin Inspector ===

Version: 1.4.3
Hash: db27525
Branch: 1.4-release
Build date: Mon, 09 Jul 2018 21:20:18 GMT
Client compatibility: 1

=== Build Information ===

Release ID: 802001105
Git revision: a187f6852ae62ba52b944b47e53702957edfcf7b
Build date: 2019-07-22 18:20:10+00
Build branch: release-8.2
Xamarin extensions: 34bd2378d8d707df25989727c23ce68d9a232a67

=== Operating System ===

Mac OS X 10.14.6
Darwin 18.7.0 Darwin Kernel Version 18.7.0
Thu Jun 20 18:42:21 PDT 2019
root:xnu-4903.270.47~4/RELEASE_X86_64 x86_64

=== Enabled user installed extensions ===

NuGet Package Management Extensions 0.18

owenniblock · 2019-07-25T16:42:15Z

@briminnick - our current working theory is that this is an issue with mono. Please could you try this workaround and let me know if it resolves the issue?

brminnick · 2019-07-25T16:58:28Z

Thanks @owenniblock! Reverting to Mono v5.18.1.28 fixed it 👍

owenniblock · 2019-07-25T16:58:57Z

Thanks for letting us know. I've filed an issue on the mono project here: mono/mono#15830

cheles · 2019-08-01T18:45:20Z

actually they've just closed it on Mono side with this message
UITest code needs to be updated to check for HttpRequestException.

owenniblock · 2019-08-01T21:45:43Z

@cheles - I'm on it. My local dev set-up seems unhappy after I installed Xcode 11 so might be Monday before we get a dev release out.

owenniblock · 2019-08-05T17:10:35Z

Please try the latest dev release of UITest and let us know if you hit further issues: https://www.nuget.org/packages/Xamarin.UITest/3.0.3-dev1

adamzucchi · 2019-08-05T18:53:33Z

@owenniblock - I am seeing some improvement using 3.0.3-dev1 (I can now run the REPL on iOS) however I am still seeing the ThreadAbortException.

brminnick · 2019-08-05T19:00:19Z

Thanks @owenniblock!

The System.Net.Http.ConnectHelper.ConnectAsync exception I was experiencing is resolved with Xamarin.UITest v3.0.3-dev1; iOS tests now run as expected using Mono v6.0.0.311.

brminnick · 2019-08-05T19:11:25Z

@adamzucchi I'm not sure if this will resolve your error, but I saw this work-around in the Release Notes for Xamarin.UITest v3.0.3-dev1:

Known Limitations

[iOS] Suspected timing issue running on iOS locally may result in
connection failures to the test server running on a
simulator. Running Console.app and reading simulator logs while testing may offer a workaround for this issue.

adamzucchi · 2019-08-05T19:31:24Z

@brminnick - Thanks for the suggestion (which I had not previously seen) : )

Unfortunately nothing seems glaringly obvious when looking at the logs. I will report back if I come across anything that seems to have been the culprit.

Thanks again!

owenniblock · 2019-08-05T20:10:40Z

@adamzucchi - I'm just looking at this with fresh eyes (I think it's been muddied slightly by additional issues we've seen lately).

The bit of code which is failing here is running a command like this:

xcrun xcodebuild test-without-building -xctestrun {xctestrunPath} -destination 'id={deviceId}' -derivedDataPath {derivedDataPath}

Do you have access to another development system you could test this on? If we can get a small project which replicates this consistently it would really help us work out what's going on here. If it works on another system, let's try to discount some Xcode weirdness (maybe removing and re-installing Xcode will make a difference).

cheles · 2019-08-07T08:45:17Z

I no longer see any "Connection refused" in 3.0.3-dev1 . Thanks @owenniblock & @brminnick

jpc1963 · 2019-08-13T13:38:12Z

I am currently running my tests using NUnit 3.12, Xamarin.UITest 3.0.3
I no longer get the original error or the "connection refused" error

jpc1963 · 2019-08-30T07:47:16Z

Hang on. I don't think I tested this properly the last time.
I am (again) trying to update my UI test to nunit 3 using the following packages:

Nunit: 3.12
Specflow 3.0.225
Xamarin UITest 3.0.3

I am using VS2019 for mac (version 8.2.5 build 5). I have mono 6.0.0.327 but I have set 5.18.1.28 as the default (as this was proposed as a workaround). When I run my test (on ios simulator) I get the error: Exception has been thrown by the target of an invocation.

This is the reason I created the post originally. Have I missed some fix?

owenniblock · 2019-08-30T21:36:12Z

Hi @jpc1963 - UITest 3.0.3 should be compatible with the latest mono.

Can I ask what version of Xcode you're using? And also whether you have Xcode 11 installed (regardless of whether you are using it for this simulator).

jpc1963 · 2019-09-02T07:29:26Z

I am using xcode 10.3. xcode 11 is not installed.

jpc1963 · 2019-09-02T13:08:38Z

One more update...
I removed all references to SpecFlow from my test project. And converted the specflow feature to a simple nunit test. This works OK using the latest version of nunit & xamarin.uitest.
I am running the tests from vs2019 on a mac. Using the latest mono version.
So the problem seems to be specflow - or the interaction between visual studio & specflow.

owenniblock · 2019-09-04T06:50:15Z

Thanks @jpc1963 - let me record this over here and get someone to try and replicate. Is it possible to provide a sample project using SpecFlow which replicates this for our engineers?

jpc1963 · 2019-09-04T09:41:52Z

UploadTestApp.zip

@owenniblock here is a simple app & ui test.
Install the app onto an iphone simulator. Open the ui test solution, restore nugets and run the login test. You should get the error.

owenniblock · 2019-09-04T12:53:04Z

Thanks - I'll add this to the ticket.

Oddj0b · 2019-09-05T12:00:49Z

Hi, @jpc1963

Thank you for taking the time to help diagnose this issue. I agree that it appears to be a problem with the interaction between Visual Studio and SpecFlow. However, we don't own Specflow and can't fix the issue. I would recommend taking a look at the SpecFlow bug tracker and see if there are similar issues there regarding Visual Studio as well as any potential workarounds.

jpc1963 · 2019-09-18T12:47:48Z

Another update..
I created an issue at specflow (here: https://github.com/techtalk/SpecFlow/issues/1726).
Their response is basically that since the logs show nothing relating to specflow itself, that the problem may lie with Xamarin.uitest.

King-of-Spades · 2019-09-18T15:45:06Z

@jpc1963 Your link is set as this:
[https://github.com/techtalk/SpecFlow/issues/1726](url)
which points to https://github.com/microsoft/appcenter/issues/url

Corrected target: SpecFlowOSS/SpecFlow#1726

From that discussion:

No ideas on my side.
We don't maintain the SpecFlow VS Extension for VS for Mac/MonoDevelop/XamarinStudio and in the callstacks I don't see anything related to SpecFlow.
What I know is, that Xamarin UITest has some ugly hacks for SpecFlow support that is version dependent. Perhaps they didn't update it yet for SpecFlow 3.

I'm not sure the best way to resolve this "we say ____, they say _____" back and forth with SpecFlow, @Oddj0b or @owenniblock can we confirm if we still feel this is out of our scope and highlight data relevant to that?

Stephen-Mc · 2019-09-19T10:07:13Z

Just to clarify the quote above: The Mac integration for SpecFlow is not maintained by the SpecFlow team, but is a third-party solution. We only maintain the Windows version. So if the issue is in the SpecFlow Visual Studio extension for Mac, there's nothing the SpecFlow team can do about it. That would need to be addressed to the maintainer of the Mac Visual Studio extension project.

If the issue is in SpecFlow itself, then we can address it. I hope that makes it clear that Andi was just trying to clarify ownership and why he is unable to comment on the Mac extension.

jpc1963 · 2019-11-06T13:48:56Z

@Oddj0b is this problem still being looked at?
I know its closed but I dont think it has been resolved. I just downloaded the latest version of VS2019 for mac and I still see the problem

Jeanjean · 2021-03-05T19:55:19Z

Sorry, to comment on such an old issue but I just seem unable to get SpecFlow working with Xamarin.UITest to work. I have some working UITests that I've set up without SpecFlow but as soon as I add the SpecFlow and SpecFlow.NUnit packages all my tests fail with
System.Reflection.TargetInvocationException : Exception has been thrown by the target of an invocation. ----> System.TypeLoadException : VTable setup of type Xamarin.UITest.SpecFlow.Shared.TestTracerDecorator failed
Is it not possible to combine 'vanilla flavour' uitesting with SpecFlow features, or am I missing something?

Jeanjean · 2021-03-05T21:07:27Z

So it turned out to be a compatibility issue between SpecFlow and Xamarin.UITest. Downgrading the SpecFlow nugets solved my issue. A working combination for me right now is:

NUnit 3.13.1
SpecFlow 3.5.14
SpecFlow.NUnit 3.5.14
Xamarin.UITest 3.0.15

Slm0nB · 2021-04-06T14:16:25Z

@Jeanjean Thank you so much for the info. I encountered the same issue, but with your help, I now have a working test pipeline, yeah!

barrymculhane · 2021-04-06T17:11:33Z

@Jeanjean What version of the Xamarin.UITest nuget are you using with NUnit 3.13.1 and SpecFlow 3.5.14 ?

Jeanjean · 2021-04-07T06:26:26Z

@Jeanjean What version of the Xamarin.UITest nuget are you using with NUnit 3.13.1 and SpecFlow 3.5.14 ?

@barrymculhane I'm currently on 3.0.15. I've edited my post to also include the Xamarin.UITest version.

kristofberge · 2021-04-22T16:42:45Z

@Jeanjean This solved it for me. Thank you so much.

FYI, I also had to downgrade BoDi to version 1.4.1.

EllaVader · 2021-10-06T19:13:22Z

One other point - @Jeanjean solution worked but I also had to change that target framework for the test project to 4.7.2
As of me writing this, I played around with the following nugets and this is combination that works:
Nunit3TestAdapter: 3.13.0
Nunit: 3.13.0
SpecFlow: 3.5.14
SpecFlow.Nunit: 3.5.14
Xamarin.UITest: 3.2.2

Note that as of me writing this, I am using the latest versions of Nunit3TestAdapter, Nunit and Xamarin.UITest. The issue is with the version of SpecFlow and SpecFlow.Nunit. It has to be 3.5.14 for both SpecFlow and SpecFlow.Nunit. Bumping it up to anything higher than 3.5.14 causes issue that @Jeanjean reported. Note that there issue 2263 specific to this issue @Jeanjean reported

Jeanjean · 2022-02-08T08:20:17Z

The release notes for Xamarin.UITest 3.2.6 say - [SpecFlow] Fixed "VTable setup of type Xamarin.UITest.SpecFlow.Shared.TestTracerDecorator" exception for SpecFlow 3.*

And I can confirm my tests are generated and running correctly after updating my nugets to:

NUnit 3.13.2
NUnit3TestAdapter 4.2.1
SpecFlow.NUnit 3.9.50
Xamarin.UITest 3.2.6

jpc1963 added the bug A bug or issue with App Center label May 15, 2019

patniko assigned Oddj0b May 15, 2019

patniko added the test Related to the App Center Test service label May 15, 2019

jeromelaban mentioned this issue Jun 25, 2019

[Epic] Opened Microsoft issues tracking unoplatform/uno#982

Open

jeromelaban mentioned this issue Jul 16, 2019

[iOS] Enable support for iOS UI Testing unoplatform/Uno.UITest#8

Closed

King-of-Spades mentioned this issue Aug 1, 2019

Xamarin.UITest on iOS stopped restarting the app even with StartApp #340

Closed

Oddj0b closed this as completed Sep 5, 2019

NUnit 3 / UITest failing with error: Exception has been thrown by the target of an invocation #421

NUnit 3 / UITest failing with error: Exception has been thrown by the target of an invocation #421

Comments

jpc1963 commented May 15, 2019

jeromelaban commented Jun 21, 2019

ahanag22 commented Jun 21, 2019

jeromelaban commented Jun 21, 2019

ahanag22 commented Jun 21, 2019

jeromelaban commented Jun 21, 2019

ahanag22 commented Jun 21, 2019

jeromelaban commented Jun 21, 2019

jeromelaban commented Jun 21, 2019 • edited Loading

stoneman commented Jun 24, 2019

jeromelaban commented Jun 25, 2019

adamzucchi commented Jul 22, 2019

jeromelaban commented Jul 23, 2019

jeromelaban commented Jul 24, 2019

owenniblock commented Jul 25, 2019

owenniblock commented Jul 25, 2019

stoneman commented Jul 25, 2019

jeromelaban commented Jul 25, 2019 • edited Loading

brminnick commented Jul 25, 2019

Reproduction Sample

Stack Trace

Environment

owenniblock commented Jul 25, 2019

brminnick commented Jul 25, 2019 • edited Loading

owenniblock commented Jul 25, 2019 • edited Loading

cheles commented Aug 1, 2019

owenniblock commented Aug 1, 2019

owenniblock commented Aug 5, 2019

adamzucchi commented Aug 5, 2019

brminnick commented Aug 5, 2019 • edited Loading

brminnick commented Aug 5, 2019 • edited Loading

Known Limitations

adamzucchi commented Aug 5, 2019

owenniblock commented Aug 5, 2019

cheles commented Aug 7, 2019 • edited Loading

jpc1963 commented Aug 13, 2019

jpc1963 commented Aug 30, 2019 • edited Loading

owenniblock commented Aug 30, 2019

jpc1963 commented Sep 2, 2019

jpc1963 commented Sep 2, 2019

owenniblock commented Sep 4, 2019 • edited Loading

jpc1963 commented Sep 4, 2019

owenniblock commented Sep 4, 2019

Oddj0b commented Sep 5, 2019

jpc1963 commented Sep 18, 2019

King-of-Spades commented Sep 18, 2019

Stephen-Mc commented Sep 19, 2019

jpc1963 commented Nov 6, 2019

Jeanjean commented Mar 5, 2021

Jeanjean commented Mar 5, 2021 • edited Loading

Slm0nB commented Apr 6, 2021

barrymculhane commented Apr 6, 2021

Jeanjean commented Apr 7, 2021 • edited Loading

kristofberge commented Apr 22, 2021 • edited Loading

EllaVader commented Oct 6, 2021 • edited Loading

Jeanjean commented Feb 8, 2022

jeromelaban commented Jun 21, 2019 •

edited

Loading

jeromelaban commented Jul 25, 2019 •

edited

Loading

brminnick commented Jul 25, 2019 •

edited

Loading

owenniblock commented Jul 25, 2019 •

edited

Loading

brminnick commented Aug 5, 2019 •

edited

Loading

brminnick commented Aug 5, 2019 •

edited

Loading

cheles commented Aug 7, 2019 •

edited

Loading

jpc1963 commented Aug 30, 2019 •

edited

Loading

owenniblock commented Sep 4, 2019 •

edited

Loading

Jeanjean commented Mar 5, 2021 •

edited

Loading

Jeanjean commented Apr 7, 2021 •

edited

Loading

kristofberge commented Apr 22, 2021 •

edited

Loading

EllaVader commented Oct 6, 2021 •

edited

Loading