Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Submission/rhauck #43

Closed
wants to merge 2 commits into from
Closed

Submission/rhauck #43

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6660819
Made some minor changes to Robin's awesome scripts to get them workin…
rhauck Feb 26, 2013
4dfeee8
Merge branches 'rhauck' and 'submission/rhauck' into submission/rhauck
rhauck Feb 26, 2013
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions tools/coverage/analyse-specs.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,11 @@ var fs = require("fs")
, exec = require("child_process").execFile
, phanthomScript = pth.join(__dirname, "get-analysis-for.phjs")
, specs = {
html: "http://www.w3.org/html/wg/drafts/html/master/single-page.html"
, canvas2d: "http://www.w3.org/html/wg/drafts/2dcontext/html5_canvas/Overview.html"
// , microdata: "http://www.w3.org/html/wg/drafts/microdata/master/Overview.html"
html: "http://www.w3.org/html/wg/drafts/html/master/single-page.html"
, canvas2d: "http://www.w3.org/html/wg/drafts/2dcontext/html5_canvas/Overview.html"
// , microdata: "http://www.w3.org/html/wg/drafts/microdata/master/Overview.html"
// , css3transforms: "http://dev.w3.org/csswg/css3-transforms/Overview.html"
// , css3backgrounds: "http://dev.w3.org/csswg/css3-background/Overview.html"
}
, perSec = JSON.parse(fs.readFileSync(pth.join(__dirname, "tests-per-section.json"), "utf8"))
, data = {}
Expand Down Expand Up @@ -40,7 +42,9 @@ for (var spec in specs) {
if (err) return console.log("ERROR: " + err);
for (var i = 0, n = data[spec].length; i < n; i++) {
var sec = data[spec][i];
sec.tests = perSec[spec][sec.id] || 0;
// Only read in the test data if it isn't already there
if( !sec.hasOwnProperty("tests") )
sec.tests = perSec[spec][sec.id] || 0;
}
fs.writeFileSync(pth.join(__dirname, "spec-data-" + spec + ".json"), JSON.stringify(data[spec], null, 4), "utf8");
});
Expand Down
55 changes: 55 additions & 0 deletions tools/coverage/css-index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
<!DOCTYPE html>
<html lang='en'>
<head>
<meta charset='utf-8'>
<title>Test Suite Coverage Analysis</title>
<link rel='stylesheet' href='coverage.css'>
</head>
<body>
<h1>Test Suite Coverage Analysis</h1>
<p>
The following tables summarise test coverage for the specifications contained in this
test suite.
</p>
<div>
<h4>Show levels:</h4>
<p>
The report analyses sections at a maximal depth of 3. If you want to see an overview
at section levels 1 or 2 instead, you can choose those options below.
</p>
<ul>
<li>Level 1 <input type='radio' name='level' value='1'></li>
<li>Level 2 <input type='radio' name='level' value='2'></li>
<li>Level 3 <input type='radio' name='level' value='3'></li>
</ul>
</div>
<div>
<h4>Thresholds:</h4>
<p>
Picking a good ratio of tests to something (or something to test) is difficult.
You can set values for specific aspects below. I assume that at some point we
will come up with a more or less definitive set of values to roughly aim for.
</p>
<ul>
<li>Words per test: <input type='number' name='words' size='4' value='10'></li>
<li>Tests per RFC2119: <input type='number' name='rfc2119' size='4' value='5'></li>
<li>Tests per algorithm step: <input type='number' name='algos' size='4' value='1'></li>
<li>Tests per IDL item: <input type='number' name='idl' size='4' value='5'></li>
</ul>
<button id='update'>Update</button>
</div>
<div id='output'></div>

<!--
- add a filter that can colour the cells based on user-defined thresholds
- hide levels
-->
<script src='http://ajax.googleapis.com/ajax/libs/jquery/1.9.0/jquery.min.js'></script>
<script src='coverage.js'></script>
<script>
cover(["css3transforms", "css3backgrounds"]
, ["CSS Transforms", "CSS Backgrounds & Borders"]
, $("#output"));
</script>
</body>
</html>
23 changes: 20 additions & 3 deletions tools/coverage/get-analysis-for.phjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,9 @@ page.open(url, function (status) {
, original_id: href
, level: level
}
, $ol = $li.find("> ol").first()
;
var $ol = $li.find("> ol").first().length ? $li.find("> ol").first() : $li.find("> ul").first();

list.push(def);
if ($ol.length && level < 3) {
walkToC($, $ol, list, level);
Expand Down Expand Up @@ -99,9 +100,21 @@ page.open(url, function (status) {
return div;
}

function getTestAnnotations(tocList) {
for(var i = 0; i < tocList.length; i++)
{
// If the spec uses annotate.js, each section will have a widget with the id prepended with 'annotation_'
var $annotation = $("#annotation_"+ tocList[i].id);
if( $annotation.length )
tocList[i].tests = parseInt($annotation.attr("testcount"));
else if( $("#annotation_root_").length )
// If there's no annotation on this section but this spec has annotate.js, that means there are no tests
tocList[i].tests = 0;
}
}

// process ToC
var $toc = $("body > ol.toc").first()
;
var $toc = $("body > ol.toc").first().length ? $("body > ol.toc").first() : $("body > ul.toc").first();
walkToC($, $toc, tocList, 0);
processTocForRanges(tocList);

Expand Down Expand Up @@ -156,6 +169,10 @@ page.open(url, function (status) {
var norms = $df.text().match(/\b(should|must)\b/g);
section.normativeStatements = norms ? norms.length : 0;
}

// Pull testcounts from spec if it has annotate.js
getTestAnnotations(tocList);

return tocList;
});
console.log(JSON.stringify(toc));
Expand Down
Loading