Releases: pester/Pester
5.2.0-rc1
Pester 5.2.0 is finally close to release! 🥳🥳🥳
First of, thanks to all the contributors, especially @fflaten who responded to a ton of issues, made multiple PRs, helped me diagnose problems, and checked my fixes. Thank you!
Code coverage
Coverage report is back, on screen and on the result object
The theme of this release was Code Coverage. I finally fixed the coverage report which is now output to the screen when Detailed (or Diagnostic) output level is specified. And it is always attached to the result object when CodeCoverage is enabled. The CodeCoverage data are attached to the result object as well, if you want to do further processing on it.
Performance is better
I focused on performance as well and all breakpoints are now set in one place, making it 50% faster in my tests, but I would love to see your numbers. There is also new option, CodeCoverage.SingleHitBreakpoints, which will remove the breakpoint as soon as it is hit, lowering the overhead in PowerShell. This option is enabled by default and makes the execution a bit faster as well.
But not as great as it can be
I also implemented CodeCoverage using the still unreleased PowerShell profiler, which I started about half a year ago that @iSazonov has been working on. PowerShell/PowerShell#13673 Once this is merged and released Pester is ready to start using it #1884. This will make CodeCoverage almost as fast running without it.
You can specify your desired code coverage percent
Using option CodeCoverage.CoveragePercentTarget you can now specify the desired code coverage. The default is 75%. This has only visual effect for now, it shows as green message in the output when the target is met, or as red when it is not met.
(See it in the gif below)
VSCode + Pester CodeCoverage are now friends
Using CodeCoverage in VSCode is very painful and that's a shame. I added new format based on JaCoCo which is especially catered to Coverage Gutters extension in VSCode. This, plus some boilerplate code enables you to easily see code coverage when developing in VSCode. The format is also compatible with Azure DevOps coverage view.
Full DEMO of the feature is here: https://youtu.be/qeiy8fRMHf8?t=5697
And code here https://gist.github.com/nohwnd/efc339339dc328d93e0fe000249aea25
❗ -CI no longer enables CodeCoverage
Lastly, the -CI switch no longer enables CodeCoverage. There is no way to make CodeCoverage fast on all versions of PowerShell so it is not a good default for beginners, or build pipelines you want to quickly setup. If you don't mind the overhead, use this configuration to get the functionality of the -CI switch:
$configuration = New-PesterConfiguration
$configuration.CodeCoverage.Enabled = $true
$configuration.TestResult.Enabled = $true
$configuration.Run.Exit = $true
Invoke-Pester -Configuration $configurationRelated changes:
- Fix blank code coverage 1621 #1807
- Coverage gutters #1887
- Fix wildcards in dirs #1858
- Add Coverage to result object #1860
- Little improvements to coverage output #1866
- Bunch bps and one hit bps #1865
- CI switch does not enable CodeCoverage by default #1911
Configuration
New-PesterConfiguration
New-PesterConfiguration cmdlet is added which returns [PesterConfiguration]::Default. It is recommended to be used instead of the .NET call because it will auto-load Pester if it was not loaded already.
Throw on failed run
New option Run.Throw is added which is similar to Run.Exit (-EnableExit). When enabled Pester will throw when there are any failed tests. When both Run.Exit and Run.Throw are enabled throwing exception is preferred, because it is more informative and because it works better with VSCode where exit is ignored.
Pester.dll version is checked on import
The Dll holding the configuration and other types is now versioned based on the version of Pester that it is released with. There is also a minimal version check that will ensure that you will get an error on module load, when you already have an older version of Pester loaded in the current session. This unfortunately cannot be avoided and throwing a sensible error is better than getting failure during runtime because some property on an object was renamed.
Related changes:
- Extend and export New-PesterConfiguration #1728
- Add assembly version check during module import #1790
- Add option to throw on failure #1908
Should
Should -Be for string
The useful arrow on Should -Be when comparing strings is back. I updated the implementation to show as much as it can based on how wide your window is, without wrapping the lines for big outputs. Notice the difference in the last example is on index 985 in string which is over 4000 characters long.
- Useful string assert message #1880
Mocking
The mocking was not focus of the current sprint, but I made a lot of fixes there as well.
Cleanup
When you cancel test run using Ctrl+C Pester mock functions and aliases may stay in the session. In subsequent Invoke-Pester call we need to clean them up to ensure stale mocks are not making your tests fail. Pester now looks for those stale mocks not just in Pester scope, but also in all currently loaded modules and user scope.
$PesterBoundParameters variable
In -MockWith and -ParameterFilter you can now use $PesterBoundParameters variable which holds all the bound parameters for that function call. This variable is like the $PSBoundParameters which is not correctly populated, and cannot be without breaking Mock debugging.
Logging
The diagnostic output when searching for mock behaviors is much improved. Calling a mock will show all the behaviors that are present for that command, and reason why they are used or not used. There is also a list of all the behaviors that will execute parameter filter, and default behaviors. The log also shows the target module in which a mock was defined more clearly (in the log below it is module m). When mock is in script scope $none is used to denote it in the log. The mock hook function has a clearer name showing for which command the mock is.
In the code example below, mocks are defined in two different scopes (module m and script scope):
Invoke-Pester -Container (
New-PesterContainer -ScriptBlock {
Describe 'Mock logging' {
It 'is more detailed' {
Get-Module m | Remove-Module
New-Module m -ScriptBlock {
function Invoke-Job ($ScriptBlock){
Start-Job -ScriptBlock $ScriptBlock
}
} | Import-Module
Mock Start-Job -ModuleName m -MockWith { "default mock in module m" }
Mock Start-Job -ModuleName m -MockWith {
"parametrized mock in module m"
} -ParameterFilter { $Name -eq "hello-job" }
Mock Start-Job -MockWith { "default mock in script" }
# call Start-Job in script
# this will call the Mock defined in script
Start-Job -ScriptBlock { Write-Host "hello" }
# call mock of Start-Job via Invoke-Job inside of module m
# this will call the Mock defined in module m
Invoke-Job -ScriptBlock { Write-Host "hello" }
}
}
}
) -Output DiagnosticThis is the improved log:
Mock: Setting up default mock for m - Start-Job.
Mock: Resolving command Start-Job.
Mock: ModuleName was specified searching for the command in module m.
Mock: Found module m version 0.0.
Mock: Found the command Start-Job in a different module.
Mock: Mock does not have a hook yet, creating a new one.
Mock: Defined new hook with bootstrap function PesterMock_m_Start-Job_a611abe3-203b-42c7-b81f-668945eb29eb and aliases Start-Job, Microsoft.PowerShell.Core\Start-Job.
Mock: Adding a new default behavior to m - Start-Job.
Mock: Setting up parametrized mock for m - Start-Job.
[...]
Mock: Found the command Start-Job in a different module and it resolved to PesterMock_m_Start-Job_a611abe3-203b-42c7-b81f-668945eb29eb.
Mock: Mock resolves to an existing hook, will only define mock behavior.
Mock: Adding a new parametrized behavior to m - Start-Job.
[...]
Mock: Setting up default mock for Start-Job.
Mock: We are in a test. Returning mock table from test scope.
Mock: Resolving command Start-Job.
Mock: Searching for command Start-Job in the script scope.
Mock: Found the command Start-Job in the script scope.
Mock: Mock does not have a hook yet, creating a new one.
Mock: Defined new hook with bootstrap function PesterMock_script_Start-Job_7745c21c-4173-4c11-be80-7274dd7b93ec and aliases Start-Job, Microsoft.PowerShell.Core\Start-Job.
[...]
Mock: Found 0 behaviors in all parent blocks, and 3 behaviors in test.
Mock: Filtering behaviors for command Start-Job, for target module $null (Showing all behaviors for this command, actual filtered list is further in the log, look for 'Filtered parametriz...
5.2.0-beta1
5.2.0-beta1
Issues Fixed
- Add $PesterBoundParameters to -MockWith and -ParameterFilter scriptblocks #1916
- Fix mock behavior for Mock and Should -Invoke #1915
- Resolve class metadata in the correct scope #1913
- Add Pester assembly version check during module import #1790
- Fix output of fullname filter in log #1893
- Add New-PesterConfiguration cmdlet to create [PesterConfiguration] #1728
- Disable CodeCoverage in CI switch #1911
- Add option to throw on failure #1908
- Rename Script.ps1 to Pester.ps1 #1907
- Ensure plugins are null #1900
- Isolate test files and scriptblocks to run in their it's own script scope #1889
- Report failures in discovery into result object #1898
- Fix setting passed on parent block when some blocks are not run #1897
- Fix testdrive example #1894
See full log here
5.2.0-alpha3
Issues Fixed
- Coverage gutters #1887
- In should Invoke, log output of parameter filter and variables we are redefining #1881
- Exclude noisy Measure-SafeCommands rule by default #1882
- Enable only discovery #1870
- Useful string assert message #1880
- Build locally without inlining #1872
- Tiny readme fixes
- Update readme.md #1874
- Fix broken Version-property in Pester4Result object #1789
- Initialize state in Invoke-Pester and inherit it to children #1869
See full log here
5.2.0-alpha2
5.2.0-alpha2
-
See this to set code coverage target:
#1866 (comment) -
In Detailed output, full coverage report is printed.
-
CodeCoverage property added to -PassThru result.
Issues Fixed
- Little improvements to coverage output #1866
- Bunch bps and one hit bps #1865
- Add Coverage to result object #1860
See full log here
5.2.0-alpha1
5.2.0-alpha1
Issues Fixed
- Bump to 5.2.0-alpha1
- Fix wildcards in dirs #1858
- Fix searching for mock table when deep in tests #1856
- Fix conflicting name "arguments" in parameter filter #1855
- Fix interactive execution of parameterized tests #1787
- Fix default InModuleScope args-value #1812
- Fix missing initial value for Data in Invoke-File #1852
- Fix missing LICENSE file in Nuget package (#1839) #1840
- Fix blank code coverage 1621 #1807
- Replace non-breaking spaces (ascii code 160) with regular spaces (ascii code 32). Fixes #1820. #1823
- Fixed type in README.md #1834
- Set 20 minute timeout for tests
- fixed typo in the examples of Invoke-Pester #1841
- Fix typo #1844
- Merge branch 'rel/5.1.1xx' into v5.0
- Fix safecomands spelling #1800
- Bump branding to 5.2.0-beta1
See full log here
5.1.1
5.1.0
5.1.0
First of all, huge thanks to @fflaten who contributed a huge amount of code, bug fixing, issue responses, and improvements in this and previous releases. 🍾🥂
Also thanks to all other contributors who contributed by reporting and fixing bugs, posting PRs, discussing features, or just by sharing their opinion.
Catching up with v4
Parametrized scripts
Passing parameters to scripts works again. Read all about it, and how to generate tests in https://pester.dev/docs/usage/data-driven-tests.
New-Fixture is back
New-Fixture was added back into Pester v5, with a better behavior. It will now fail by default to show you that you did not complete the template.
Updated help
Help was updated both on function and in the usage docs on Pester docs site, including sections specific for migrating from Pester v4. See:
- https://pester.dev/docs/quick-start
- https://pester.dev/docs/usage/file-placement-and-naming
- https://pester.dev/docs/usage/importing-tested-functions
- https://pester.dev/docs/usage/test-file-structure
- https://pester.dev/docs/usage/discovery-and-run
- https://pester.dev/docs/usage/data-driven-tests
- https://pester.dev/docs/usage/setup-and-teardown
JUnit output is working again
JUnit output for Pester results is available again. And NUnit output is fixed.
New features
BeforeDiscovery block
BeforeDiscovery is a semantic block that allows you to group your code together if you need it to run during discovery. It just takes your code and runs it in place. But it gives a nice way of showing that this code is here intentionally, and not just a forgotten piece of code that you did not migrate to Pester v5.
BeforeDiscovery {
$testConfigs = Get-ChildItem -Path $PSScriptRoot/TestConfigs
}
Describe "Platform dependent tests" {
foreach ($config in $testConfigs) {
It "Runs for config $($config.Name)" {
# ...
}
}
}"TestCases" for Describe and Context
-ForEach parameter was added to Describe and Context, and also to It as an alias for -TestCases. Like -TestCases on It you can use this to create one block for each example.
This has been long awaited and this thread has a lot of examples of the usage. #1679
This can also be used with the parameters that you provide directly to the script which was introduced in the previous preview. See example of the usage in the thread. #1679 (comment)
"TestCases" can take any array, not just arrays of hashtables
You can now provide any array to -TestCases / -ForEach, and it will be defined as $_ in your tests. Additionally if your array is an array of hashtables, each key in that hashtable will be expanded to a variable, the same way it already works for -TestCases.
(Just don't use _ as the name of any Key in your hashtable, it won't work well.
Templates expand to all variables and allow dot-notation
Using this syntax <user>, that you know from -TestCases you can now expand value from any variable that is defined in the scope, not just from the provided hashtable. You can use <_> to expand the $_. You can also use dot-notation <user.name>, to navigate into the object. Here an example of the dot-notation in action:
Describe "<animal.name>" -ForEach @(
@{
Animal = @{
Name = "dog"
Sounds = @("woof", "barf")
}
}
@{
Animal = @{
Name = "cat"
Sounds = @("meow", "purr")
}
}
) {
It "<animal.name> goes <_>" -ForEach $Animal.Sounds {
}
}Starting discovery in 1 files.
Discovering in C:\Users\jajares\Desktop\ex.tests.ps1.
Found 4 tests. 48ms
Discovery finished in 61ms.
Running tests from 'C:\Users\jajares\Desktop\ex.tests.ps1'
Describing dog
[+] dog goes woof 10ms (4ms|5ms)
[+] dog goes barf 11ms (2ms|8ms)
Describing cat
[+] cat goes meow 8ms (2ms|7ms)
[+] cat goes purr 8ms (5ms|3ms)
Tests completed in 370ms
See https://pester.dev/docs/usage/data-driven-tests for even more info on this.
Smaller improvements
- Add missing and update help for exported functions #1785
- Add JUnit XML output #1748
- Always set exit code exit #1734
- Print active filters in Detailed/Diagnostic output #1727
- Re-implement New-Fixture in Pester v5 #1726
- Add support for VSCode problem-tracking for failed assertions #1700
- Allow AfterAll in top-level #1707
- Throw when legacy syntax is used #1706
- Remove pipeline support from Should -Invoke #1698
- Filter excluded tests from NUnit results #1686
- Add -ForEach parameter to Describe, Context, and as an alias to -Test… #1681
- Add BeforeDiscovery semantic block #1680
- Add devcontainer to project #1661
- Parametric scripts #1671
See full log here
5.1.0-rc2
5.1.0-rc1
5.1.0-beta4
5.1.0-beta4
Notable changes
- Add
JUnitXMLoutput - Rename
New-TestContainertoNew-PesterContainer
Issues Fixed
- Fix JUnit counts for skipped and notrun #1758
- Fix path in NUnit #1755
- Fix joining fallback ExpandedPath #1754
- Add JUnit XML output #1748
- Disable CC in build to make it much faster #1751
- Create CODE_OF_CONDUCT.md and SECURITY.md #1676
- Cleanup alias and missing SafeCommands #1745
- Removed test-step in PR template #1749
- Rename New-TestContainer to New-PesterContainer and fix Run.Container usage #1743
- Add labels to templates #1742
- Update github templates #1739
See full log here