Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple fixes to the CLT tasks (QuickPerfTestV1|RunLoadTestV1|RunJMeterLoadTestV1) #7226

Merged
merged 8 commits into from
May 18, 2018
Merged

Multiple fixes to the CLT tasks (QuickPerfTestV1|RunLoadTestV1|RunJMeterLoadTestV1) #7226

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
e4d7ea0
Fixes for: new domain issue, self provisioned agent default values
ganesp May 16, 2018
1712ade
Updated default value for numOfSelfProvisionedAgents
ganesp May 16, 2018
22d6c64
Set the default values for self provisioned option
ganesp May 16, 2018
8fcea48
Fixed the summary markdown
ganesp May 16, 2018
dc32c67
Added /updated geo location list for CLT tasks
ganesp May 17, 2018
84afe04
Support for load location input in jmeter task
ganesp May 17, 2018
ab51ce7
Removed location input from runloadtest task
ganesp May 17, 2018
026052a
Generated files
ganesp May 17, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions Tasks/QuickPerfTestV1/CltTasksUtility.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -157,11 +157,6 @@ function ValidateInputs($websiteUrl, $tfsCollectionUrl, $connectedServiceName, $
throw "Website Url is not well formed."
}

if([string]::IsNullOrWhiteSpace($connectedServiceName) -and $tfsCollectionUrl -notlike "*VISUALSTUDIO.COM*" -and $tfsCollectionUrl -notlike "*TFSALLIN.NET*")
{
throw "VS Team Services Connection is mandatory for using performance test tasks on non hosted TFS builds.Please specify a VS Team Services connection and try again "
}

# validate load test name
# code taken from definitionNameInvalid
$invalidPattern1 = "(^\\.$|^\\.\\.$|^-$|^_$)"
Expand Down
4 changes: 4 additions & 0 deletions Tasks/QuickPerfTestV1/Strings/resources.resjson/en-US/resources.resjson
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@
"loc.input.label.geoLocation": "Load Location",
"loc.input.help.geoLocation": "Geographical region to generate the load from.",
"loc.input.label.machineType": "Run load test using",
"loc.input.label.resourceGroupName": "Resource group rig",
"loc.input.help.resourceGroupName": "Name of Resource group hosting the self-provisioned rig of load test agents.",
"loc.input.label.numOfSelfProvisionedAgents": "No. of agents to use",
"loc.input.help.numOfSelfProvisionedAgents": "Number of self provisioned agents to use for this test.",
"loc.input.label.avgResponseTimeThreshold": "Fail test if Avg.Response Time(ms) exceeds",
"loc.input.help.avgResponseTimeThreshold": "Average response time above which the load test outcome is considered unsuccessful."
}
30 changes: 16 additions & 14 deletions Tasks/QuickPerfTestV1/task.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
"version": {
"Major": 1,
"Minor": 0,
"Patch": 28
"Patch": 29
},
"demands": [
"msbuild",
Expand Down Expand Up @@ -88,21 +88,23 @@
"defaultValue": "Default",
"options": {
"Default": "Default",
"East US": "East US (Virginia)",
"East US 2": "East US 2 (Virginia)",
"Australia East": "Australia East (New South Wales)",
"Australia Southeast": "Australia Southeast (Victoria)",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did u do upgrade testing. not sure if existing build definition will break

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, validated the upgrade path as well.

"Brazil South": "Brazil South (Sao Paulo State)",
"Central India": "Central India (Pune)",
"Central US": "Central US (Iowa)",
"West US": "West US (California)",
"North Central US": "North Central US (Illinois)",
"South Central US": "South Central US (Texas)",
"North Europe": "North Europe (Ireland)",
"West Europe": "West Europe (Netherlands)",
"Southeast Asia": "Southeast Asia (Singapore)",
"East Asia": "East Asia (Hong Kong)",
"East US 2": "East US 2 (Virginia)",
"East US": "East US (Virginia)",
"Japan East": "Japan East (Saitama Prefecture)",
"Japan West": "Japan West (Osaka Prefecture)",
"Brazil South": "Brazil South (Sao Paulo State)",
"Australia East": "Australia East (New South Wales)",
"Australia Southeast": "Australia Southeast (Victoria)"
"North Central US": "North Central US (Illinois)",
"North Europe": "North Europe (Ireland)",
"South Central US": "South Central US (Texas)",
"South India": "South India (Chennai)",
"Southeast Asia": "Southeast Asia (Singapore)",
"West Europe": "West Europe (Netherlands)",
"West US": "West US (California)"
},
"properties": {
"EditableOptions": "True"
Expand All @@ -124,7 +126,7 @@
"type": "string",
"label": "Resource group rig",
"required": false,
"defaultValue": "",
"defaultValue": "default",
"visibleRule": "machineType == 2",
"helpMarkDown": "Name of Resource group hosting the self-provisioned rig of load test agents."
},
Expand All @@ -133,7 +135,7 @@
"type": "int",
"label": "No. of agents to use",
"required": false,
"defaultValue": "",
"defaultValue": 1,
"visibleRule": "machineType == 2",
"helpMarkDown": "Number of self provisioned agents to use for this test."
},
Expand Down
53 changes: 41 additions & 12 deletions Tasks/QuickPerfTestV1/task.loc.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
"version": {
"Major": 1,
"Minor": 0,
"Patch": 27
"Patch": 29
},
"demands": [
"msbuild",
Expand Down Expand Up @@ -56,6 +56,9 @@
"50": "50",
"100": "100",
"250": "250"
},
"properties": {
"EditableOptions": "True"
}
},
{
Expand All @@ -71,6 +74,9 @@
"180": "180",
"240": "240",
"300": "300"
},
"properties": {
"EditableOptions": "True"
}
},
{
Expand All @@ -82,21 +88,26 @@
"defaultValue": "Default",
"options": {
"Default": "Default",
"East US": "East US (Virginia)",
"East US 2": "East US 2 (Virginia)",
"Australia East": "Australia East (New South Wales)",
"Australia Southeast": "Australia Southeast (Victoria)",
"Brazil South": "Brazil South (Sao Paulo State)",
"Central India": "Central India (Pune)",
"Central US": "Central US (Iowa)",
"West US": "West US (California)",
"North Central US": "North Central US (Illinois)",
"South Central US": "South Central US (Texas)",
"North Europe": "North Europe (Ireland)",
"West Europe": "West Europe (Netherlands)",
"Southeast Asia": "Southeast Asia (Singapore)",
"East Asia": "East Asia (Hong Kong)",
"East US 2": "East US 2 (Virginia)",
"East US": "East US (Virginia)",
"Japan East": "Japan East (Saitama Prefecture)",
"Japan West": "Japan West (Osaka Prefecture)",
"Brazil South": "Brazil South (Sao Paulo State)",
"Australia East": "Australia East (New South Wales)",
"Australia Southeast": "Australia Southeast (Victoria)"
"North Central US": "North Central US (Illinois)",
"North Europe": "North Europe (Ireland)",
"South Central US": "South Central US (Texas)",
"South India": "South India (Chennai)",
"Southeast Asia": "Southeast Asia (Singapore)",
"West Europe": "West Europe (Netherlands)",
"West US": "West US (California)"
},
"properties": {
"EditableOptions": "True"
}
},
{
Expand All @@ -110,6 +121,24 @@
"2": "Self-provisioned agents"
}
},
{
"name": "resourceGroupName",
"type": "string",
"label": "ms-resource:loc.input.label.resourceGroupName",
"required": false,
"defaultValue": "default",
"visibleRule": "machineType == 2",
"helpMarkDown": "ms-resource:loc.input.help.resourceGroupName"
},
{
"name": "numOfSelfProvisionedAgents",
"type": "int",
"label": "ms-resource:loc.input.label.numOfSelfProvisionedAgents",
"required": false,
"defaultValue": 1,
"visibleRule": "machineType == 2",
"helpMarkDown": "ms-resource:loc.input.help.numOfSelfProvisionedAgents"
},
{
"name": "avgResponseTimeThreshold",
"type": "string",
Expand Down
29 changes: 17 additions & 12 deletions Tasks/RunJMeterLoadTestV1/CltTasksUtility.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,35 +7,40 @@ function InvokeRestMethod($headers, $contentType, $uri , $method= "Get", $body)
return $result
}

function ComposeTestDropJson($name, $duration, $homepage, $vu, $geoLocation)
function ComposeTestDropJson($name, $agentCount, $duration, $geoLocation)
{
$coresPerAgent = 2
$coreCount = $agentCount*$coresPerAgent
$tdjson = @"
{
"dropType": "InplaceDrop",
"dropType": "TestServiceBlobDrop",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TestServiceBlobDrop [](start = 21, length = 19)

why is there a change in the drop type ?

Copy link
Contributor Author

@ganesp ganesp May 17, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Jmeter it was always TestServiceBlobDrop.
This particular function was unused.
I am using it now, hence the change in the dropType here.

"loadTestDefinition":{
"loadTestName":"$name",
"agentCount":$agentCount,
"runDuration":$duration,
"urls":["$homepage"],
"browserMixs":[
{"browserName":"Internet Explorer 11.0","browserPercentage":60.0},
{"browserName":"Chrome 2","browserPercentage":40.0}
],
"loadPatternName":"Constant",
"maxVusers":$vu,
"loadGenerationGeoLocations":[
{"Location":"$geoLocation","Percentage":100}
]
],

"coresPerAgent": $coresPerAgent,
"coreCount": $coreCount,
"samplingRate": 15,
"thinkTime": 0,
"urls": [],
"browserMixs": [],
"loadPatternName": "Constant",
"maxVusers": -1
}
}
"@

return $tdjson
}

function CreateTestDrop($headers, $CltAccountUrl)
function CreateTestDrop($headers, $dropjson, $CltAccountUrl)
{
$uri = [String]::Format("{0}/_apis/clt/testdrops?{1}", $CltAccountUrl, $global:apiVersion)
$drop = InvokeRestMethod -contentType "application/json" -uri $uri -headers $headers -method Post -body "{ ""dropType"": ""TestServiceBlobDrop"" }"
$drop = InvokeRestMethod -contentType "application/json" -uri $uri -headers $headers -method Post -body $dropJson
return $drop
}

Expand Down
10 changes: 8 additions & 2 deletions Tasks/RunJMeterLoadTestV1/Start-ApacheJMeterTest.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ $LoadTest,
$agentCount,
[String] [Parameter(Mandatory = $true)]
$runDuration,
[String] [Parameter(Mandatory = $true)]
$geoLocation,
[String] [Parameter(Mandatory = $true)] [ValidateNotNullOrEmpty()]
$machineType
)
Expand Down Expand Up @@ -86,6 +88,7 @@ import-module "Microsoft.TeamFoundation.DistributedTask.Task.DevTestLabs"

Write-Output "Test drop = $TestDrop"
Write-Output "Load test = $LoadTest"
Write-Output "Load location = $geoLocation"
Write-Output "Load generator machine type = $machineType"
Write-Output "Run source identifier = build/$env:SYSTEM_DEFINITIONID/$env:BUILD_BUILDID"

Expand Down Expand Up @@ -116,7 +119,10 @@ Write-Output "CLT account Url = $CltAccountUrl" -Verbose

#Upload the test drop
$elapsed = [System.Diagnostics.Stopwatch]::StartNew()
$drop = CreateTestDrop $headers $CltAccountUrl

$dropjson = ComposeTestDropJson $LoadTest $agentCount $runDuration $geoLocation

$drop = CreateTestDrop $headers $dropjson $CltAccountUrl

if ($drop.dropType -eq "TestServiceBlobDrop")
{
Expand Down Expand Up @@ -159,7 +165,7 @@ if ($drop.dropType -eq "TestServiceBlobDrop")
Remove-Item $resultsMDFolder\$resultFilePattern -Exclude $excludeFilePattern -Force
$summaryFile = ("{0}\ApacheJMeterTestResults_{1}_{2}_{3}_{4}.md" -f $resultsMDFolder, $env:AGENT_ID, $env:SYSTEM_DEFINITIONID, $env:BUILD_BUILDID, $run.id)

$summary = ('[Test Run: {0}]({1}) using {2}.<br/>' -f $run.runNumber, $webResultsUrl , $run.name)
$summary = ('<a href="{1}" target="_blank">Test Run: {0}</a> using {2}.' -f $run.runNumber, $webResultsUrl , $run.name)

('<p>{0}</p>' -f $summary) | Out-File $summaryFile -Encoding ascii -Append
UploadSummaryMdReport $summaryFile
Expand Down
2 changes: 2 additions & 0 deletions Tasks/RunJMeterLoadTestV1/Strings/resources.resjson/en-US/resources.resjson
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,7 @@
"loc.input.help.agentCount": "Number of test agents (dual-core) used in the run.",
"loc.input.label.runDuration": "Run Duration (sec)",
"loc.input.help.runDuration": "Load test run duration in seconds.",
"loc.input.label.geoLocation": "Load Location",
"loc.input.help.geoLocation": "Geographical region to generate the load from.",
"loc.input.label.machineType": "Run load test using"
}
33 changes: 32 additions & 1 deletion Tasks/RunJMeterLoadTestV1/task.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
"version": {
"Major": 1,
"Minor": 0,
"Patch": 19
"Patch": 20
},
"demands": [
"azureps"
Expand Down Expand Up @@ -73,6 +73,37 @@
"300": "300"
}
},
{
"name": "geoLocation",
"type": "pickList",
"label": "Load Location",
"required": false,
"helpMarkDown": "Geographical region to generate the load from.",
"defaultValue": "Default",
"options": {
"Default": "Default",
"Australia East": "Australia East (New South Wales)",
"Australia Southeast": "Australia Southeast (Victoria)",
"Brazil South": "Brazil South (Sao Paulo State)",
"Central India": "Central India (Pune)",
"Central US": "Central US (Iowa)",
"East Asia": "East Asia (Hong Kong)",
"East US 2": "East US 2 (Virginia)",
"East US": "East US (Virginia)",
"Japan East": "Japan East (Saitama Prefecture)",
"Japan West": "Japan West (Osaka Prefecture)",
"North Central US": "North Central US (Illinois)",
"North Europe": "North Europe (Ireland)",
"South Central US": "South Central US (Texas)",
"South India": "South India (Chennai)",
"Southeast Asia": "Southeast Asia (Singapore)",
"West Europe": "West Europe (Netherlands)",
"West US": "West US (California)"
},
"properties": {
"EditableOptions": "True"
}
},
{
"name": "machineType",
"type": "radio",
Expand Down
33 changes: 32 additions & 1 deletion Tasks/RunJMeterLoadTestV1/task.loc.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
"version": {
"Major": 1,
"Minor": 0,
"Patch": 19
"Patch": 20
},
"demands": [
"azureps"
Expand Down Expand Up @@ -73,6 +73,37 @@
"300": "300"
}
},
{
"name": "geoLocation",
"type": "pickList",
"label": "ms-resource:loc.input.label.geoLocation",
"required": false,
"helpMarkDown": "ms-resource:loc.input.help.geoLocation",
"defaultValue": "Default",
"options": {
"Default": "Default",
"Australia East": "Australia East (New South Wales)",
"Australia Southeast": "Australia Southeast (Victoria)",
"Brazil South": "Brazil South (Sao Paulo State)",
"Central India": "Central India (Pune)",
"Central US": "Central US (Iowa)",
"East Asia": "East Asia (Hong Kong)",
"East US 2": "East US 2 (Virginia)",
"East US": "East US (Virginia)",
"Japan East": "Japan East (Saitama Prefecture)",
"Japan West": "Japan West (Osaka Prefecture)",
"North Central US": "North Central US (Illinois)",
"North Europe": "North Europe (Ireland)",
"South Central US": "South Central US (Texas)",
"South India": "South India (Chennai)",
"Southeast Asia": "Southeast Asia (Singapore)",
"West Europe": "West Europe (Netherlands)",
"West US": "West US (California)"
},
"properties": {
"EditableOptions": "True"
}
},
{
"name": "machineType",
"type": "radio",
Expand Down
6 changes: 5 additions & 1 deletion Tasks/RunLoadTestV1/Strings/resources.resjson/en-US/resources.resjson
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,9 @@
"loc.input.help.TestSettings": "The testsettings file name to be used from the load test folder specified above or a full path. <ul><li><b>Build Example:</b><br /> $(System.DefaultWorkingDirectory)\\LoadTestproject\\bin\\$(BuildConfiguration)\\load.testsettings </li><li><b>Release Example:</b><br /> $(System.DefaultWorkingDirectory)\\SourceCI\\drop\\LoadTestproject\\bin\\Release\\load.testsettings <br />where SourceCI is the source alias and drop is artifact name</li></ul>",
"loc.input.label.ThresholdLimit": "Number of permissible threshold violations",
"loc.input.help.ThresholdLimit": "Number of threshold violations above which the load test outcome is considered unsuccessful.",
"loc.input.label.MachineType": "Run load test using"
"loc.input.label.MachineType": "Run load test using",
"loc.input.label.resourceGroupName": "Resource group rig",
"loc.input.help.resourceGroupName": "Name of Resource group hosting the self-provisioned rig of load test agents.",
"loc.input.label.numOfSelfProvisionedAgents": "No. of agents to use",
"loc.input.help.numOfSelfProvisionedAgents": "Number of self provisioned agents to use for this test."
}
Loading