Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Code cov perf and bug fix in if statements #264

Merged
merged 46 commits into from
Feb 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
0e07019
Added function to bast test suite to store global mocked functions
chrisdp Jan 11, 2024
77d06a4
Moved some logic to the utils
chrisdp Jan 11, 2024
9f75924
Added logic to inject code that adds global mock checks at the start …
chrisdp Jan 11, 2024
90edc04
Fixed a reversed check
chrisdp Jan 15, 2024
2c4feef
Removed new api in favor of expanding stubcall and updated injection …
chrisdp Jan 15, 2024
9f6111e
Updated some of the global function detection logic
chrisdp Jan 15, 2024
fadbf2c
Unit tests, fixes, sample test project update
chrisdp Jan 15, 2024
fc6ce1d
removed console logs
chrisdp Jan 15, 2024
8b3fee5
Transpile the file if we had to touch modify it
chrisdp Jan 15, 2024
3307955
Fixed global mocks and stubs not clearing
chrisdp Jan 15, 2024
a155a29
updated some of the checks around transforming stubcall functions
chrisdp Jan 16, 2024
a342165
Made some code reusable and fixed some imports
chrisdp Jan 16, 2024
02c6f4e
Added back the disablemocking logic and removed unused code
chrisdp Jan 16, 2024
02c64df
Fixed bad ast related to noEarlyExit
chrisdp Jan 16, 2024
86656dd
Updated bsc in tests app
chrisdp Jan 16, 2024
db71048
added tests for global stubcall on device
chrisdp Jan 16, 2024
4c3d753
Fixed some device tests
chrisdp Jan 16, 2024
b4024ee
Fixed more on device tests
chrisdp Jan 16, 2024
67ec05a
More test fixes as the result of moving to ast editor
chrisdp Jan 16, 2024
bcfd24d
Fixed some indenting
chrisdp Jan 16, 2024
9f4eabd
Fixed node test xml files being added after modifying assertions lead…
chrisdp Jan 17, 2024
3acbf16
Updated the modify stub detection logic for globals
chrisdp Jan 17, 2024
1be78f6
more tests for global stub call modifications
chrisdp Jan 17, 2024
3164492
Fixed some race conditons and more global function detection refinments
chrisdp Jan 17, 2024
ec6a105
Removed some raw code statements for the session and fixed test forma…
chrisdp Jan 17, 2024
fb0e852
removed trimLeading from mock util tests
chrisdp Jan 17, 2024
18c1c98
Removed some raw code statements in code cov and updated test file fo…
chrisdp Jan 17, 2024
f2e0959
Added a test showing the issue
chrisdp Jan 17, 2024
f8ef1cf
Fixed the code cov handling of if statements
chrisdp Jan 18, 2024
f22a8ef
Updated test app for better covrage
chrisdp Jan 18, 2024
96919c7
Fixed a bug with the reporting of missed files
chrisdp Jan 18, 2024
e4193eb
Improved code cov reporting perfomance
chrisdp Jan 18, 2024
ff6624c
perf optimization to cc reporting
chrisdp Jan 18, 2024
3961348
Some perf changed for code cov
chrisdp Jan 18, 2024
127d9e8
Fixed device by zero crash in reports for cov
chrisdp Jan 19, 2024
4132da2
Fixed some issues picking the wrong scope and make sure stubcall work…
chrisdp Jan 19, 2024
f435bba
Moved global stub clearing to clearStubs()
chrisdp Jan 20, 2024
91d0d87
Merge branch 'feature/runtime-global-function-mocking' into reduce-ra…
chrisdp Jan 20, 2024
6154b49
Merge branch 'reduce-raw-code-statements' into code-cov-in-if-statements
chrisdp Jan 20, 2024
a9e1450
Merge branch 'master' into feature/runtime-global-function-mocking
chrisdp Jan 21, 2024
06a4bbe
Merge branch 'feature/runtime-global-function-mocking' into reduce-ra…
chrisdp Jan 21, 2024
7d31717
Merge branch 'reduce-raw-code-statements' into code-cov-in-if-statements
chrisdp Jan 21, 2024
5d6bbca
Merge pull request #250 from chrisdp/feature/runtime-global-function-…
chrisdp Jan 22, 2024
ed3d6b6
Merge pull request #253 from chrisdp/reduce-raw-code-statements
chrisdp Jan 22, 2024
3ab269f
Merge pull request #254 from chrisdp/code-cov-in-if-statements
chrisdp Jan 22, 2024
5dfb2a3
Merge branch 'master' into code-cov-in-if-statements
chrisdp Jan 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
330 changes: 219 additions & 111 deletions bsc-plugin/src/lib/rooibos/CodeCoverageProcessor.spec.ts

Large diffs are not rendered by default.

68 changes: 35 additions & 33 deletions bsc-plugin/src/lib/rooibos/CodeCoverageProcessor.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
/* eslint-disable @typescript-eslint/no-var-requires */
/* eslint-disable @typescript-eslint/no-unsafe-assignment */
import type { BrsFile, Editor, ExpressionStatement, Program, ProgramBuilder, Statement } from 'brighterscript';
import { Parser, isIfStatement, WalkMode, createVisitor } from 'brighterscript';
import * as brighterscript from 'brighterscript';
import { Parser, WalkMode, createVisitor, BinaryExpression, createToken, TokenKind, GroupingExpression, isForStatement, isBlock } from 'brighterscript';
import type { RooibosConfig } from './RooibosConfig';
import { RawCodeStatement } from './RawCodeStatement';
import { BrsTranspileState } from 'brighterscript/dist/parser/BrsTranspileState';
Expand All @@ -20,23 +19,18 @@ export class CodeCoverageProcessor {

private coverageBrsTemplate = `
function RBS_CC_#ID#_reportLine(lineNumber, reportType = 1)
if m.global = invalid
'? "global is not available in this scope!! it is not possible to record coverage: #FILE_PATH#(lineNumber)"
_rbs_ccn = m._rbs_ccn
if _rbs_ccn <> invalid
_rbs_ccn.entry = { "f": "#ID#", "l": lineNumber, "r": reportType }
return true
else
if m._rbs_ccn = invalid
'? "Coverage maps are not created - creating now"
if m.global._rbs_ccn = invalid
'? "Coverage maps are not created - creating now"
m.global.addFields({
"_rbs_ccn": createObject("roSGNode", "CodeCoverage")
})
end if
m._rbs_ccn = m.global._rbs_ccn
end if
end if

m._rbs_ccn.entry = {"f":"#ID#", "l":stri(lineNumber), "r":reportType}
_rbs_ccn = m?.global?._rbs_ccn
if _rbs_ccn <> invalid
_rbs_ccn.entry = { "f": "#ID#", "l": lineNumber, "r": reportType }
m._rbs_ccn = _rbs_ccn
return true
end if
return true
end function
`;
Expand Down Expand Up @@ -65,9 +59,7 @@ export class CodeCoverageProcessor {
private astEditor: Editor;

public generateMetadata(isUsingCoverage: boolean, program: Program) {
if (isUsingCoverage) {
this.fileFactory.createCoverageComponent(program, this.expectedCoverageMap, this.filePathMap);
}
this.fileFactory.createCoverageComponent(program, this.expectedCoverageMap, this.filePathMap);
}

public addCodeCoverage(file: BrsFile, astEditor: Editor) {
Expand All @@ -89,20 +81,30 @@ export class CodeCoverageProcessor {
this.addStatement(ds);
ds.forToken.text = `${this.getFuncCallText(ds.range.start.line, CodeCoverageLineType.code)}: for`;
},
IfStatement: (ds, parent, owner, key) => {
let ifStatement = ds;
while (isIfStatement(ifStatement)) {
this.addStatement(ds);
// eslint-disable-next-line @typescript-eslint/no-unsafe-argument
(ifStatement as any).condition = new brighterscript.BinaryExpression(new RawCodeExpression(this.getFuncCallText(ds.condition.range.start.line, CodeCoverageLineType.branch)), brighterscript.createToken(brighterscript.TokenKind.And), (ifStatement as any).condition);
ifStatement = ifStatement.elseBranch as any;
}
let blockStatements = (ifStatement as any)?.statements as any[] ?? [];
if (blockStatements?.length > 0) {
// eslint-disable-next-line @typescript-eslint/no-unsafe-argument
let coverageStatement = new RawCodeStatement(this.getFuncCallText((ifStatement as any).range.start.line - 1, CodeCoverageLineType.branch));
IfStatement: (ifStatement, parent, owner, key) => {
this.addStatement(ifStatement);
(ifStatement as any).condition = new BinaryExpression(
new RawCodeExpression(this.getFuncCallText(ifStatement.condition.range.start.line, CodeCoverageLineType.condition)),
createToken(TokenKind.And),
new GroupingExpression({
left: createToken(TokenKind.LeftParen),
right: createToken(TokenKind.RightParen)
}, ifStatement.condition)
);

let blockStatements = ifStatement?.thenBranch?.statements;
if (blockStatements) {
let coverageStatement = new RawCodeStatement(this.getFuncCallText(ifStatement.range.start.line, CodeCoverageLineType.branch));
blockStatements.splice(0, 0, coverageStatement);
}

// Handle the else blocks
let elseBlock = ifStatement.elseBranch;
if (isBlock(elseBlock) && elseBlock.statements) {
let coverageStatement = new RawCodeStatement(this.getFuncCallText(elseBlock.range.start.line - 1, CodeCoverageLineType.branch));
elseBlock.statements.splice(0, 0, coverageStatement);
}

},
GotoStatement: (ds, parent, owner, key) => {
this.addStatement(ds);
Expand Down Expand Up @@ -143,7 +145,7 @@ export class CodeCoverageProcessor {

},
AssignmentStatement: (ds, parent, owner, key) => {
if (!brighterscript.isForStatement(parent)) {
if (!isForStatement(parent)) {
this.addStatement(ds);
this.convertStatementToCoverageStatement(ds, CodeCoverageLineType.code, owner, key);
}
Expand Down Expand Up @@ -186,6 +188,6 @@ export class CodeCoverageProcessor {
}

private getFuncCallText(lineNumber: number, lineType: CodeCoverageLineType) {
return `RBS_CC_${this.fileId}_reportLine(${lineNumber.toString().trim()}, ${lineType.toString().trim()})`;
return `RBS_CC_${this.fileId}_reportLine("${lineNumber.toString().trim()}", ${lineType.toString().trim()})`;
}
}
26 changes: 15 additions & 11 deletions framework/src/source/CodeCoverage.brs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ function init()
m.resolvedMap = {}
m.top.observeField("entry", "onEntryChange")
m.top.observeField("save", "onSave")

m.results = []
end function

function setExpectedMap()
Expand All @@ -15,20 +15,24 @@ end function

function onEntryChange()
entry = m.top.entry
if entry <> invalid
lineMap = m.resolvedMap[entry.f]

if lineMap = invalid
lineMap = {}
end if
lineMap[entry.l] = entry.r

m.resolvedMap[entry.f] = lineMap
end if
' defer till later
m.results.push(entry)
end function

function onSave()
? "saving data"
for each entry in m.results
if entry <> invalid
fileId = entry.f
lineMap = m.resolvedMap[fileId]

if lineMap = invalid
lineMap = {}
m.resolvedMap[fileId] = lineMap
end if
lineMap[entry.l] = entry.r
end if
end for
m.top.resolvedMap = m.resolvedMap
setExpectedMap()
setFilePathMap()
Expand Down
36 changes: 27 additions & 9 deletions framework/src/source/Coverage.bs
Original file line number Diff line number Diff line change
Expand Up @@ -5,39 +5,54 @@ namespace rooibos.Coverage
? "There was no rooibos code coverage component - not generating coverage report"
return
end if
t = createObject("roTimespan")
? ""
? "...Generating code coverage report"
? ""
m.global._rbs_ccn.save = true
cc = m.global._rbs_ccn
expectedMap = cc.expectedMap
filePathMap = cc.filePathMap
resolvedMap = cc.resolvedMap
chrisdp marked this conversation as resolved.
Show resolved Hide resolved

hitFiles = []
missFiles = []
allLinesCount = 0
allLinesHit = 0
for each key in cc.expectedMap
filename = cc.filePathMap[key]
expectedCount = cc.expectedMap[key].count()
for each key in expectedMap
filename = filePathMap[key]
expectedCount = expectedMap[key].count()
allLinesCount += expectedCount
if expectedCount > 0
if cc.resolvedMap[key] <> invalid
resolvedCount = cc.resolvedMap[key].count()
if resolvedMap[key] <> invalid
resolvedCount = resolvedMap[key].count()
allLinesHit += resolvedCount
resolvedPercent = (resolvedCount / expectedCount) * 100
if resolvedCount = 0
resolvedPercent = 0
else
resolvedPercent = (resolvedCount / expectedCount) * 100
end if
hitFiles.push({ percent: resolvedPercent, text: filename + ": " + str(resolvedPercent).trim() + "% (" + stri(resolvedCount).trim() + "/" + stri(expectedCount).trim() + ")" })
else
missFiles.push(filename + ": MISS!")
end if
else
missFiles.push(filename + ": MISS!")
end if
end for
allLinesPercent = (allLinesHit / allLinesCount) * 100
if allLinesHit = 0
allLinesPercent = 0
else
allLinesPercent = (allLinesHit / allLinesCount) * 100
end if
? ""
? ""
? "+++++++++++++++++++++++++++++++++++++++++++"
? "Code Coverage Report"
? "+++++++++++++++++++++++++++++++++++++++++++"
? ""
? "Total Coverage: " ; str(allLinesPercent).trim() ; "% (" ; stri(allLinesHit).trim() ; "/" + stri(allLinesCount).trim() ; ")"
? "Files: " ; cc.resolvedMap.count(); "/" ; cc.expectedMap.count()
? "Files: " ; resolvedMap.count(); "/" ; expectedMap.count()
? ""
? "HIT FILES"
? "---------"
Expand All @@ -51,7 +66,10 @@ namespace rooibos.Coverage
for i = 0 to missFiles.count() - 1
? missFiles[i]
end for

? ""
? "+++++++++++++++++++++++++++++++++++++++++++"
? "Code Coverage Report Complete"; t.totalMilliseconds(); "ms"
? "+++++++++++++++++++++++++++++++++++++++++++"
end function

function createLCovOutput()
Expand Down
5 changes: 4 additions & 1 deletion framework/src/source/Rooibos.bs
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,10 @@ namespace rooibos
screen.show()

m.global = screen.getGlobalNode()
m.global.addFields({ "testsScene": scene })
m.global.addFields({
"testsScene": scene
"_rbs_ccn": createObject("roSGNode", "CodeCoverage")' bs:disable-line
})

if scene.hasField("isReadyToStartTests") and scene.isReadyToStartTests = false
? "The scene is not ready yet - waiting for it to set isReadyToStartTests to true"
Expand Down
21 changes: 0 additions & 21 deletions tests/src/components/NodeExample.bs
Original file line number Diff line number Diff line change
Expand Up @@ -16,27 +16,6 @@ function SetLabelText(newText = "") as void
m.nameText.text = newText
end function


function r_real_Init() as void
if m.top.stubs["init"] <> invalid
'mock call here
else
Init()
end if
end function

function r_real_HelloFromNode(name, age) as string
return "HELLO " + name + " age:" + stri(age)
end function

function r_real_UpdateState(newState) as void
m.top.state = newState
end function

function r_real_SetLabelText(newText) as void
m.nameText.text = newText
end function

function delayCall(delay, callback) as void
timer = createObject("roSgNode", "Timer")
timer.update({
Expand Down
6 changes: 6 additions & 0 deletions tests/src/components/NodeExample.xml
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,10 @@
<component
name="NodeExample"
extends="Group">
<interface>
<field id="state" type="string" />
</interface>
<children>
<Label id="nameText" />
</children>
</component>
3 changes: 0 additions & 3 deletions tests/src/source/Main.bs
Original file line number Diff line number Diff line change
@@ -1,8 +1,5 @@

function Main(args)

? "here is my code"
? "hello"
end function

function globalFunctionWithReturn() as dynamic
Expand Down
26 changes: 26 additions & 0 deletions tests/src/source/NewExpectSyntax.spec.bs
Original file line number Diff line number Diff line change
Expand Up @@ -319,6 +319,15 @@ namespace tests

@it("stubs namespace and then cleans it")
function _()
getGlobalAA().wasCalled = false
m.stubCall(testNamespace.functionWithReturn, function()
m.wasCalled = true
return true
end function)

m.assertTrue(testNamespace.functionWithReturn())
m.assertTrue(getGlobalAA().wasCalled)

getGlobalAA().wasCalled = false
m.stubCall(testNamespace.functionWithoutReturn, sub()
m.wasCalled = true
Expand All @@ -329,6 +338,10 @@ namespace tests

m.cleanStubs()

m.assertFalse(testNamespace.functionWithReturn())
m.assertFalse(getGlobalAA().wasCalled)

getGlobalAA().wasCalled = true
m.assertInvalid(testNamespace.functionWithoutReturn())
m.assertFalse(getGlobalAA().wasCalled)

Expand All @@ -337,6 +350,15 @@ namespace tests

@it("stubs global and then cleans it")
function _()
getGlobalAA().wasCalled = false
m.stubCall(globalFunctionWithReturn, function()
m.wasCalled = true
return true
end function)

m.assertTrue(globalFunctionWithReturn())
m.assertTrue(getGlobalAA().wasCalled)

getGlobalAA().wasCalled = false
m.stubCall(globalFunctionWithoutReturn, sub()
m.wasCalled = true
Expand All @@ -347,6 +369,10 @@ namespace tests

m.cleanStubs()

m.assertFalse(globalFunctionWithReturn())
m.assertFalse(getGlobalAA().wasCalled)

getGlobalAA().wasCalled = true
m.assertInvalid(globalFunctionWithoutReturn())
m.assertFalse(getGlobalAA().wasCalled)

Expand Down
23 changes: 23 additions & 0 deletions tests/src/source/NodeExample.spec.bs
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,29 @@ namespace tests
m.timer.observeFieldScoped("fire", callback)
m.timer.control = "start"
end function

@it("updates state")
@params("start")
@params("stop")
@params("error")
function _(state)
m.node.top.state = ""
UpdateState(state)
m.assertEqual(m.node.top.state, state)
end function

@it("updates name text")
@params("jon")
@params("ringo")
@params("ringo")
@params("ringo")
@params("george")
@params("paul")
function _(text)
m.node.nameText.text = ""
SetLabelText(text)
m.assertEqual(m.node.nameText.text, text)
end function
end class
end namespace

Expand Down