Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically fetch mazerunner commands rather than waiting for a button press #215

Merged
merged 6 commits into from
Nov 21, 2023
Merged

Automatically fetch mazerunner commands rather than waiting for a button press #215

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
19e6ad9
Automatically fetch mazerunner commands rather than waiting for a but…
kstenerud Nov 17, 2023
3a5d78b
Make sure administration network requests aren't captured by e2e tests
kstenerud Nov 17, 2023
17498ce
Fetch mazerunner address in the background to avoid OS shutdown for h…
kstenerud Nov 20, 2023
5401d66
Refactor e2e test URL handling code
kstenerud Nov 20, 2023
6ab90ef
Disable BackgroundForegroundScenario for now
kstenerud Nov 20, 2023
308b144
reflect URL doesn't require a different port anymore
kstenerud Nov 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 4 additions & 8 deletions features/default/automatic_spans.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -251,17 +251,15 @@ Feature: Automatic instrumentation spans
Scenario: Automatically start a network span that has a parent
Given I run "AutoInstrumentNetworkWithParentScenario"
And I wait for 2 seconds
And I wait for 3 spans
# Discard the request to http://bs-local.com:9339/command
And I discard the oldest trace
And I wait for 2 spans
Then the trace "Content-Type" header equals "application/json"
* the trace "Bugsnag-Sent-At" header matches the regex "^\d\d\d\d-\d\d-\d\dT\d\d:\d\d:\d\d\.\d\d\dZ$"
* a span field "parentSpanId" exists
* a span field "parentSpanId" is greater than 0
* a span field "parentSpanId" does not exist
* a span field "name" equals "[HTTP/GET]"
* a span string attribute "http.flavor" exists
* a span string attribute "http.url" matches the regex "http://.*:9340/reflect/"
* a span string attribute "http.url" matches the regex "http://.*:9339/reflect\?status=200"
* a span string attribute "http.method" equals "GET"
* a span integer attribute "http.status_code" is greater than 0
* a span integer attribute "http.response_content_length" is greater than 0
Expand All @@ -280,15 +278,13 @@ Feature: Automatic instrumentation spans
Scenario: Automatically start a network span that has no parent
Given I run "AutoInstrumentNetworkNoParentScenario"
And I wait for 2 seconds
And I wait for 3 spans
# Discard the request to http://bs-local.com:9339/command
And I discard the oldest trace
And I wait for 2 spans
Then the trace "Content-Type" header equals "application/json"
* the trace "Bugsnag-Sent-At" header matches the regex "^\d\d\d\d-\d\d-\d\dT\d\d:\d\d:\d\d\.\d\d\dZ$"
* every span field "parentSpanId" does not exist
* a span field "name" equals "[HTTP/GET]"
* a span string attribute "http.flavor" exists
* a span string attribute "http.url" matches the regex "http://.*:9340/reflect/"
* a span string attribute "http.url" matches the regex "http://.*:9339/reflect\?status=200"
* a span string attribute "http.method" equals "GET"
* a span integer attribute "http.status_code" is greater than 0
* a span integer attribute "http.response_content_length" is greater than 0
Expand Down
2 changes: 1 addition & 1 deletion features/default/manual_spans.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ Feature: Manual creation of spans
* the trace payload field "resourceSpans.0.resource" string attribute "telemetry.sdk.version" matches the regex "[0-9]\.[0-9]\.[0-9]"
* every span field "name" equals "[HTTP/GET]"
* every span string attribute "http.flavor" exists
* every span string attribute "http.url" matches the regex "http://.*:9340/reflect/"
* every span string attribute "http.url" matches the regex "http://.*:9339/reflect\?status=200"
* every span string attribute "http.method" equals "GET"
* every span integer attribute "http.status_code" is greater than 0
* every span integer attribute "http.response_content_length" is greater than 0
Expand Down
29 changes: 15 additions & 14 deletions features/default/triggers.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,16 +1,17 @@
Feature: Automatic send triggers

Scenario: BackgroundForegroundScenario
Given I run "BackgroundForegroundScenario"
And I send the app to the background for 2 seconds
And I wait for 1 span
Then the trace "Content-Type" header equals "application/json"
* the trace "Bugsnag-Span-Sampling" header equals "1:1"
* the trace "Bugsnag-Sent-At" header matches the regex "^\d\d\d\d-\d\d-\d\dT\d\d:\d\d:\d\d\.\d\d\dZ$"
* every span field "name" equals "BackgroundForegroundScenario"
* every span field "spanId" matches the regex "^[A-Fa-f0-9]{16}$"
* every span field "traceId" matches the regex "^[A-Fa-f0-9]{32}$"
* every span field "kind" equals 1
* every span field "startTimeUnixNano" matches the regex "^[0-9]+$"
* every span field "endTimeUnixNano" matches the regex "^[0-9]+$"
* every span bool attribute "bugsnag.app.in_foreground" is false
# Disabled per https://smartbear.atlassian.net/browse/PLAT-11232
#Scenario: BackgroundForegroundScenario
# Given I run "BackgroundForegroundScenario"
# And I send the app to the background for 2 seconds
# And I wait for 1 span
# Then the trace "Content-Type" header equals "application/json"
# * the trace "Bugsnag-Span-Sampling" header equals "1:1"
# * the trace "Bugsnag-Sent-At" header matches the regex "^\d\d\d\d-\d\d-\d\dT\d\d:\d\d:\d\d\.\d\d\dZ$"
# * every span field "name" equals "BackgroundForegroundScenario"
# * every span field "spanId" matches the regex "^[A-Fa-f0-9]{16}$"
# * every span field "traceId" matches the regex "^[A-Fa-f0-9]{32}$"
# * every span field "kind" equals 1
# * every span field "startTimeUnixNano" matches the regex "^[0-9]+$"
# * every span field "endTimeUnixNano" matches the regex "^[0-9]+$"
# * every span bool attribute "bugsnag.app.in_foreground" is false
42 changes: 36 additions & 6 deletions features/fixtures/ios/Fixture.xcodeproj/project.pbxproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,11 @@
01FE4DC528E1AF9600D1F239 /* Scenario.swift in Sources */ = {isa = PBXBuildFile; fileRef = 01FE4DC428E1AF9600D1F239 /* Scenario.swift */; };
01FE4DC728E1D5A400D1F239 /* Fixture.swift in Sources */ = {isa = PBXBuildFile; fileRef = 01FE4DC628E1D5A400D1F239 /* Fixture.swift */; };
0921F02E2A69262300C764EB /* AutoInstrumentNetworkCallbackScenario.swift in Sources */ = {isa = PBXBuildFile; fileRef = 0921F02D2A69262300C764EB /* AutoInstrumentNetworkCallbackScenario.swift */; };
09637A3C2B0607F300F4F776 /* Logging.swift in Sources */ = {isa = PBXBuildFile; fileRef = 09637A3B2B0607F300F4F776 /* Logging.swift */; };
09637A3F2B06082200F4F776 /* Logging.m in Sources */ = {isa = PBXBuildFile; fileRef = 09637A3E2B06082200F4F776 /* Logging.m */; };
09637A412B060E7C00F4F776 /* CommandReaderThread.swift in Sources */ = {isa = PBXBuildFile; fileRef = 09637A402B060E7C00F4F776 /* CommandReaderThread.swift */; };
09637A432B0617FE00F4F776 /* MazeRunnerCommand.swift in Sources */ = {isa = PBXBuildFile; fileRef = 09637A422B0617FE00F4F776 /* MazeRunnerCommand.swift */; };
09637A452B0B883B00F4F776 /* FixtureConfig.swift in Sources */ = {isa = PBXBuildFile; fileRef = 09637A442B0B883B00F4F776 /* FixtureConfig.swift */; };
09DA59A52A6E866B00A06EEE /* ManualNetworkCallbackScenario.swift in Sources */ = {isa = PBXBuildFile; fileRef = 09DA59A42A6E866B00A06EEE /* ManualNetworkCallbackScenario.swift */; };
9657A8992A3CF75B001CEF5D /* AutoInstrumentTabViewLoadScenario.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9657A8982A3CF75B001CEF5D /* AutoInstrumentTabViewLoadScenario.swift */; };
9657A89B2A3D06EB001CEF5D /* AutoInstrumentNavigationViewLoadScenario.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9657A89A2A3D06EB001CEF5D /* AutoInstrumentNavigationViewLoadScenario.swift */; };
Expand Down Expand Up @@ -68,6 +73,12 @@
01FE4DC428E1AF9600D1F239 /* Scenario.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Scenario.swift; sourceTree = "<group>"; };
01FE4DC628E1D5A400D1F239 /* Fixture.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Fixture.swift; sourceTree = "<group>"; };
0921F02D2A69262300C764EB /* AutoInstrumentNetworkCallbackScenario.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AutoInstrumentNetworkCallbackScenario.swift; sourceTree = "<group>"; };
09637A3B2B0607F300F4F776 /* Logging.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Logging.swift; sourceTree = "<group>"; };
09637A3D2B06082200F4F776 /* Logging.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = Logging.h; sourceTree = "<group>"; };
09637A3E2B06082200F4F776 /* Logging.m */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.objc; path = Logging.m; sourceTree = "<group>"; };
09637A402B060E7C00F4F776 /* CommandReaderThread.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CommandReaderThread.swift; sourceTree = "<group>"; };
09637A422B0617FE00F4F776 /* MazeRunnerCommand.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MazeRunnerCommand.swift; sourceTree = "<group>"; };
09637A442B0B883B00F4F776 /* FixtureConfig.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = FixtureConfig.swift; sourceTree = "<group>"; };
09DA59A42A6E866B00A06EEE /* ManualNetworkCallbackScenario.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ManualNetworkCallbackScenario.swift; sourceTree = "<group>"; };
9657A8982A3CF75B001CEF5D /* AutoInstrumentTabViewLoadScenario.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AutoInstrumentTabViewLoadScenario.swift; sourceTree = "<group>"; };
9657A89A2A3D06EB001CEF5D /* AutoInstrumentNavigationViewLoadScenario.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AutoInstrumentNavigationViewLoadScenario.swift; sourceTree = "<group>"; };
Expand Down Expand Up @@ -112,6 +123,7 @@
01FE4D9C28E1AEBD00D1F239 = {
isa = PBXGroup;
children = (
09637A3A2B0607DA00F4F776 /* utils */,
01FE4DA728E1AEBD00D1F239 /* Fixture */,
01FE4DC128E1AF0700D1F239 /* Scenarios */,
01FE4DBC28E1AED000D1F239 /* Packages */,
Expand All @@ -132,15 +144,18 @@
isa = PBXGroup;
children = (
01FE4DA828E1AEBD00D1F239 /* AppDelegate.swift */,
01FE4DC628E1D5A400D1F239 /* Fixture.swift */,
01FE4DAA28E1AEBD00D1F239 /* SceneDelegate.swift */,
01FE4DAC28E1AEBD00D1F239 /* ViewController.swift */,
01FE4DAE28E1AEBD00D1F239 /* Main.storyboard */,
01FE4DB128E1AEBF00D1F239 /* Assets.xcassets */,
01FE4DB328E1AEBF00D1F239 /* LaunchScreen.storyboard */,
01FE4DB628E1AEBF00D1F239 /* Info.plist */,
CB211D0629EEB615008F748D /* BugsnagPerformanceConfiguration+Private.h */,
09637A402B060E7C00F4F776 /* CommandReaderThread.swift */,
CBAAE25429125F5F006D4AA0 /* Fixture-Bridging-Header.h */,
01FE4DC628E1D5A400D1F239 /* Fixture.swift */,
01FE4DB628E1AEBF00D1F239 /* Info.plist */,
01FE4DB328E1AEBF00D1F239 /* LaunchScreen.storyboard */,
01FE4DAE28E1AEBD00D1F239 /* Main.storyboard */,
09637A422B0617FE00F4F776 /* MazeRunnerCommand.swift */,
01FE4DAA28E1AEBD00D1F239 /* SceneDelegate.swift */,
01FE4DAC28E1AEBD00D1F239 /* ViewController.swift */,
09637A442B0B883B00F4F776 /* FixtureConfig.swift */,
);
path = Fixture;
sourceTree = "<group>";
Expand Down Expand Up @@ -199,6 +214,16 @@
path = Scenarios;
sourceTree = "<group>";
};
09637A3A2B0607DA00F4F776 /* utils */ = {
isa = PBXGroup;
children = (
09637A3B2B0607F300F4F776 /* Logging.swift */,
09637A3D2B06082200F4F776 /* Logging.h */,
09637A3E2B06082200F4F776 /* Logging.m */,
);
path = utils;
sourceTree = "<group>";
};
/* End PBXGroup section */

/* Begin PBXNativeTarget section */
Expand Down Expand Up @@ -280,11 +305,13 @@
files = (
CBE43A9929A8EFA3000B4205 /* ProbabilityExpiryScenario.swift in Sources */,
CBE0872B29F81BBB007455F2 /* AutoInstrumentNetworkNoParentScenario.swift in Sources */,
09637A432B0617FE00F4F776 /* MazeRunnerCommand.swift in Sources */,
0185C47228F6C983006F9BDC /* AutoInstrumentViewLoadScenario.swift in Sources */,
01FE4DAD28E1AEBD00D1F239 /* ViewController.swift in Sources */,
CBEC89232A458BA70088A3CE /* AutoInstrumentNetworkMultiple.swift in Sources */,
01FE4DA928E1AEBD00D1F239 /* AppDelegate.swift in Sources */,
CBAAE2592912601D006D4AA0 /* BatchingScenario.swift in Sources */,
09637A3C2B0607F300F4F776 /* Logging.swift in Sources */,
9657A89B2A3D06EB001CEF5D /* AutoInstrumentNavigationViewLoadScenario.swift in Sources */,
01D3A7E028F0290D0063D79E /* AutoInstrumentAppStartsScenario.swift in Sources */,
01E3F99128F46B6700003F44 /* ManualViewLoadScenario.swift in Sources */,
Expand All @@ -293,6 +320,7 @@
01FE4DAB28E1AEBD00D1F239 /* SceneDelegate.swift in Sources */,
CBF62109291A4F47004BEE0B /* RetryScenario.swift in Sources */,
CB7FD92B299BB4E300499E13 /* ManualUIViewLoadScenario.swift in Sources */,
09637A3F2B06082200F4F776 /* Logging.m in Sources */,
CB3477182901481F0033759C /* AutoInstrumentNetworkWithParentScenario.swift in Sources */,
CBE6B66B28FD66B400D1CF78 /* ManualNetworkSpanScenario.swift in Sources */,
01FE4DC528E1AF9600D1F239 /* Scenario.swift in Sources */,
Expand All @@ -306,9 +334,11 @@
CB2B8A9D2A0CCEF90054FBBE /* AutoInstrumentFileURLRequestScenario.swift in Sources */,
01FE4DC328E1AF3700D1F239 /* ManualSpanScenario.swift in Sources */,
01FE4DC728E1D5A400D1F239 /* Fixture.swift in Sources */,
09637A412B060E7C00F4F776 /* CommandReaderThread.swift in Sources */,
CBC90CE029CDD02800280884 /* FirstClassYesScenario.swift in Sources */,
CBC90CDE29CDCFF700280884 /* FirstClassNoScenario.swift in Sources */,
CBC90C4329C466BD00280884 /* ForceUBSan.m in Sources */,
09637A452B0B883B00F4F776 /* FixtureConfig.swift in Sources */,
CBC90CE429D1BDE400280884 /* AutoInstrumentSubViewLoadScenario.swift in Sources */,
0921F02E2A69262300C764EB /* AutoInstrumentNetworkCallbackScenario.swift in Sources */,
01E7918A28EC7B5E00855993 /* ManualSpanBeforeStartScenario.swift in Sources */,
Expand Down
2 changes: 1 addition & 1 deletion features/fixtures/ios/Fixture/AppDelegate.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ class AppDelegate: UIResponder, UIApplicationDelegate {
var window: UIWindow?

func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
// Override point for customization after application launch.
logInfo(">>>>>>>>>> Fixture app has launched <<<<<<<<<<")
return true
}
}
Expand Down
19 changes: 2 additions & 17 deletions features/fixtures/ios/Fixture/Base.lproj/Main.storyboard
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<document type="com.apple.InterfaceBuilder3.CocoaTouch.Storyboard.XIB" version="3.0" toolsVersion="21225" targetRuntime="iOS.CocoaTouch" propertyAccessControl="none" useAutolayout="YES" useTraitCollections="YES" useSafeAreas="YES" colorMatched="YES" initialViewController="BYZ-38-t0r">
<document type="com.apple.InterfaceBuilder3.CocoaTouch.Storyboard.XIB" version="3.0" toolsVersion="22155" targetRuntime="iOS.CocoaTouch" propertyAccessControl="none" useAutolayout="YES" useTraitCollections="YES" useSafeAreas="YES" colorMatched="YES" initialViewController="BYZ-38-t0r">
<device id="retina6_0" orientation="portrait" appearance="light"/>
<dependencies>
<deployment identifier="iOS"/>
<plugIn identifier="com.apple.InterfaceBuilder.IBCocoaTouchPlugin" version="21207"/>
<plugIn identifier="com.apple.InterfaceBuilder.IBCocoaTouchPlugin" version="22131"/>
<capability name="Safe area layout guides" minToolsVersion="9.0"/>
<capability name="System colors in document resources" minToolsVersion="11.0"/>
<capability name="documents saved in the Xcode 8 format" minToolsVersion="8.0"/>
Expand All @@ -16,23 +16,8 @@
<view key="view" contentMode="scaleToFill" id="8bC-Xf-vdC">
<rect key="frame" x="0.0" y="0.0" width="390" height="844"/>
<autoresizingMask key="autoresizingMask" widthSizable="YES" heightSizable="YES"/>
<subviews>
<button opaque="NO" contentMode="scaleToFill" contentHorizontalAlignment="center" contentVerticalAlignment="center" buttonType="system" lineBreakMode="middleTruncation" translatesAutoresizingMaskIntoConstraints="NO" id="3p5-hL-l83">
<rect key="frame" x="112.66666666666669" y="404.66666666666669" width="165" height="35"/>
<accessibility key="accessibilityConfiguration" identifier="execute_command"/>
<state key="normal" title="Button"/>
<buttonConfiguration key="configuration" style="filled" title="Execute command"/>
<connections>
<action selector="fetchCommand:" destination="BYZ-38-t0r" eventType="touchUpInside" id="3jD-SG-HJc"/>
</connections>
</button>
</subviews>
<viewLayoutGuide key="safeArea" id="6Tk-OE-BBY"/>
<color key="backgroundColor" systemColor="systemBackgroundColor"/>
<constraints>
<constraint firstItem="3p5-hL-l83" firstAttribute="centerY" secondItem="8bC-Xf-vdC" secondAttribute="centerY" id="juX-k9-C29"/>
<constraint firstItem="3p5-hL-l83" firstAttribute="centerX" secondItem="8bC-Xf-vdC" secondAttribute="centerX" id="mea-zc-oBD"/>
</constraints>
</view>
</viewController>
<placeholder placeholderIdentifier="IBFirstResponder" id="dkx-z0-nzr" sceneMemberID="firstResponder"/>
Expand Down
93 changes: 93 additions & 0 deletions features/fixtures/ios/Fixture/CommandReaderThread.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
//
// CommandReaderThread.swift
// iOSTestApp
//
// Created by Karl Stenerud on 16.11.23.
// Copyright © 2023 Bugsnag. All rights reserved.
//

import UIKit
import os

class CommandReaderThread: Thread {
var fixtureConfig: FixtureConfig
var commandReceiver: CommandReceiver

init(fixtureConfig: FixtureConfig, commandReceiver: CommandReceiver) {
self.fixtureConfig = fixtureConfig
self.commandReceiver = commandReceiver
}

override func main() {
while true {
if self.commandReceiver.canReceiveCommand() {
receiveNextCommand()
} else {
logDebug("A command is already in progress, waiting 1 second more...")
}
Thread.sleep(forTimeInterval: 1)
}
}

func receiveNextCommand() {
let fetchTask = CommandFetchTask(url: fixtureConfig.commandURL)
fetchTask.start()

while true {
switch fetchTask.state {
case CommandFetchState.success:
commandReceiver.receiveCommand(command: fetchTask.command!)
return
case CommandFetchState.fetching:
logDebug("Command fetch server hasn't responded yet, waiting 1 second more...")
Thread.sleep(forTimeInterval: 1)
break
case CommandFetchState.failed:
logInfo("Command fetch request failed. Trying again...")
Thread.sleep(forTimeInterval: 1)
fetchTask.start()
break
}
}
}
}

enum CommandFetchState {
case failed, fetching, success
}

class CommandFetchTask {
var url: URL
var state = CommandFetchState.failed
var command: MazeRunnerCommand?

init(url: URL) {
self.url = url
}

func start() {
logInfo("Fetching next command from \(url)")
state = CommandFetchState.fetching
let request = URLRequest(url: url)
let task = URLSession.shared.dataTask(with: request) { data, response, error in
if let data = data {
let decoder = JSONDecoder()
decoder.keyDecodingStrategy = .convertFromSnakeCase
do {
let command = try decoder.decode(MazeRunnerCommand.self, from: data)
logInfo("Command fetched and decoded")
self.command = command;
self.state = CommandFetchState.success
} catch {
self.state = CommandFetchState.failed
let dataAsString = String(data: data, encoding: .utf8)
logError("Failed to fetch command: Invalid Response from \(String(describing: self.url)): [\(String(describing: dataAsString))]: Error is: \(error)")
}
} else if let error = error {
self.state = CommandFetchState.failed
logError("Failed to fetch command: HTTP Request to \(String(describing: self.url)) failed: \(error)")
}
}
task.resume()
}
}
1 change: 1 addition & 0 deletions features/fixtures/ios/Fixture/Fixture-Bridging-Header.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@
#import <stdint.h>
#import <Foundation/Foundation.h>
#import "BugsnagPerformanceConfiguration+Private.h"
#import "Logging.h"
Loading
Loading