Skip to content

Commit

Permalink
Changes for the report organisation, formatting
Browse files Browse the repository at this point in the history
  • Loading branch information
Shyju PV committed Nov 3, 2016
1 parent 0b37a57 commit 2031e6a
Show file tree
Hide file tree
Showing 9 changed files with 150 additions and 68 deletions.
18 changes: 15 additions & 3 deletions frontend/polls/templates/polls/algorithm.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,18 @@
<!-- link the caliper report css here -->
<link rel="stylesheet" type="text/css" href="../static/css/caliper_report.css"/>

<!-- To resize the windows while opening int he new window -->
<script type="text/javascript">
function changeSize( w, h){
window.resizeTo(w,h);
}
function movePosition( x, y){
window.moveTo(x,y);
}
//$( document ).ready(function() {
changeSize(1000,700);
//});
</script>
</head>

<body>
Expand All @@ -73,7 +85,7 @@ <h1><small>Test description </small></h1>
<p> <li><b>Openssl Benchmark Suite</b> : This tool tests all the openssl ciphers and measures the performance. The test suite is executed in single thread hence it measures the single core performance. </p></li>
<mark><p> <b>Notes: </b> In D02/D03 platform, the hardware accelerators for algorithms are not utilized. Hence the D02 results may be lower and is expected. We can expect better results once the drivers enable the hardware accelerators </p></mark>

<p> To have a very quick reference of the hardware configurations of platforms evaluated, please refer to <a href="{%static '../static/TargetInfo/Platform_Configuration.xlsx' %}" target ="_blank"> <b>Platform_Configuration.xlsx </b> </a></p>

<p> We have preserved block-wise performance details of each cipher across platforms, please refer to <a href="{%static '../static/TestInfo/Report-Data/openssl_TestCases.xlsx' %}" target ="_blank"> <b>Openssl_TestCases.xlsx </b> </a></p>
</div>

Expand Down Expand Up @@ -105,15 +117,15 @@ <h2><small>Algorithm Score Details: Digital Verify</small></h2>
<p><b>Note: </b> The test case names are same for Digital Verify and Digital Sign. However encryption is tested for Digital Sign and decryption is tested for Digital Verify.</p>

{% if hash_algorithm %}
<h2><small>Algorithm Score Details: Hash Algorithm</small></h2>
<h2 id="hash"> <small>Algorithm Score Details: Hash Algorithm</small></h2>
<div style="text-align: center;">
<p> <img src="{% static 'polls/pictures/algorithm_hash_algorithm.png' %}" height="400" width="850" /> </p>
</div>
<div id="alg_hash" class="sensei-grid"></div>
{% endif %}

{% if symmetric_cyphers %}
<h2><small>Algorithm Score Details: Symmetric Cyphers</small></h2>
<h2 id="Symmetric Ciphers"><small>Algorithm Score Details: Symmetric Cyphers</small></h2>
<div style="text-align: center;">
<p> <img src="{% static 'polls/pictures/algorithm_symmetric_cyphers.png' %}" height="400" width="900" /> </p>
</div>
Expand Down
27 changes: 16 additions & 11 deletions frontend/polls/templates/polls/application.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,18 @@
<link rel="stylesheet" type="text/css" href="../static/css/caliper_report.css"/>


<!-- To resize the windows while opening int he new window -->
<script type="text/javascript">
function changeSize( w, h){
window.resizeTo(w,h);
}
function movePosition( x, y){
window.moveTo(x,y);
}
//$( document ).ready(function() {
changeSize(1200,900);
//});
</script>
</head>
<body>
<div class="example">
Expand All @@ -70,14 +82,7 @@
<h1> Caliper Performance Tests: Application </h1>
</div>

<div style="text-align: left;">
<p> Tests under this section captures the performace of certain application scenarios such as <br>
1. <b>Kernel compilation </b>: Compiling a kernel in multiple threads can avaluate the multi core/multi threding capability of the system.<br>
2. <b>Database</b> : Sysbench oltp benchmark evaluate the database performance.<br>
3. <b>Hadoop</b> : Hibench measures the common hadoop performance parameters like HadoopTerasort,HadoopWordcount,HadoopBayes,HadoopKmeans,HadoopSort,HadoopDfsioe etc. <br>
4. <b>Ebizzy</b> : Web application sever load. <br>
5. <b>JDK</b> : Uses SciMark2 to evaluates the Java performance for scientific and numerical computing.<br>
</p>
<div style="text-align: left;">
<h1><small>Report Layout </small></h1>
<p> The score of each test item is represented as the percentage of performance compared to the highest value obtained across all platform. So a value of <b>x%</b> means that the test case/scenario performance is <b> x% </b> compared to the highest scored platform, which is always given <b> 100% </b>.</br> To know the ranking calculation and actual values scored by each platform on specific test cases, please refer to the raw values excel provided along with the report. Please click <a href="{% static 'polls/pictures/score_Calculation.html' %}" target="_blank"> <b>here</b></a> for more details. </p> </br>
The <b>score summary</b> is the percentage of performance of each platform for individual test scenarios. The detailed sections shows the individual test case score(rank) in the same fashion. <br>
Expand All @@ -95,7 +100,7 @@ <h1><small>Test description </small></h1>
<mark><p> <b>Notes: </b> <br>
Hadoop performance is based on single node. Soon Caliper to expand the scope of testing multi node performance as well</p></mark>

<p> To have a very quick reference of the hardware configurations of platforms evaluated, please refer to <a href="{%static '../static/TargetInfo/Platform_Configuration.xlsx' %}" target ="_blank"> <b>Platform_Configuration.xlsx </b> </a></p>

</div>

{% if sum %}
Expand All @@ -110,7 +115,7 @@ <h1><small>Application Score Summary</small></h1>
{% endif %}

{% if compile %}
<h2><small>Application Score Details: Compile</small></h2>
<h2 id= "Compile"> <small>Application Score Details: Compile</small></h2>
<div style="text-align: center;">
<p> <img src="{% static 'polls/pictures/application_compile.png' %}" height="500"
width="700" /> </p>
Expand Down Expand Up @@ -181,7 +186,7 @@ <h2><small>Application Score Details: JDK</small></h2>
{% endif %}

{% if sysbench %}
<h2><small>Application Score Details: Sysbench</small></h2>
<h2 id="sysbench"><small>Application Score Details: Sysbench</small></h2>
<div style="text-align: center;">
<p> <img src="{% static 'polls/pictures/application_sysbench.png' %}" height="500"
width="700" /> </p>
Expand Down
16 changes: 14 additions & 2 deletions frontend/polls/templates/polls/cpu_multicore.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,18 @@
<!-- link the caliper report css here -->
<link rel="stylesheet" type="text/css" href="../static/css/caliper_report.css"/>

<!-- To resize the windows while opening int he new window -->
<script type="text/javascript">
function changeSize( w, h){
window.resizeTo(w,h);
}
function movePosition( x, y){
window.moveTo(x,y);
}
//$( document ).ready(function() {
changeSize(1000,700);
//});
</script>
</head>
<body>
<div class="example">
Expand Down Expand Up @@ -71,10 +83,10 @@ <h1><small>Test description </small></h1>

</p>

<p> To have a very quick reference of the hardware configurations of platforms evaluated, please refer to <a href="{%static '../static/TargetInfo/Platform_Configuration.xlsx' %}" target ="_blank"> <b>Platform_Configuration.xlsx </b> </a></p>

</div>
{% if sum %}
<h1><small>CPU Score Summary</small></h1> <!--SKD++-->
<h1><small>CPU Multi Core Score Summary</small></h1>
<div style="text-align: center;">
<p> <img src="{% static 'polls/pictures/cpu_multicore_summary.png' %}" height="400" width="700" /> </p>
</div>
Expand Down
18 changes: 15 additions & 3 deletions frontend/polls/templates/polls/cpu_sincore.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,18 @@
<link rel="stylesheet" type="text/css" href="../static/css/caliper_report.css"/>


<!-- To resize the windows while opening int he new window -->
<script type="text/javascript">
function changeSize( w, h){
window.resizeTo(w,h);
}
function movePosition( x, y){
window.moveTo(x,y);
}
//$( document ).ready(function() {
changeSize(1000,700);
//});
</script>
</head>
<body>
<div class="example">
Expand Down Expand Up @@ -88,7 +100,7 @@ <h1><small>Test description </small></h1>
<p> To have a very quick reference of the hardware configurations of platforms evaluated, please refer to <a href="{%static '../static/TargetInfo/Platform_Configuration.xlsx' %}" target ="_blank"> <b>Platform_Configuration.xlsx </b> </a></p>
</div>
{% if sum %}
<h1><small>CPU Score Summary</small></h1> <!--SKD++-->
<h1><small>CPU Single Score Summary</small></h1>
<div style="text-align: center;">
<p> <img src="{% static 'polls/pictures/cpu_sincore_summary.png' %}" height="400" width="700" /> </p>
</div>
Expand All @@ -114,15 +126,15 @@ <h2><small>CPU Score Details: Single Core Float </small></h2>
{% endif %}

{% if sincore_int %}
<h2><small>CPU Score Details: Single Core Integer </small></h2>
<h2 id= "Single Core Integer"><small>CPU Score Details: Single Core Integer </small></h2>
<div style="text-align: center;">
<p> <img src="{% static 'polls/pictures/cpu_sincore_sincore_int.png' %}" height="400" width="800" /> </p>
</div>
<div id="sin-int" class="sensei-grid"></div>
{% endif %}

{% if sincore_misc %}
<h2><small>CPU Score Details: Single Core Misc </small></h2>
<h2 id="Single Core Misc" ><small>CPU Score Details: Single Core Misc </small></h2>
<div style="text-align: center;">
<p> <img src="{% static 'polls/pictures/cpu_sincore_sincore_misc.png' %}" height="400" width="850" /> </p>
</div>
Expand Down
43 changes: 24 additions & 19 deletions frontend/polls/templates/polls/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@

<!-- link the caliper report css here -->
<link rel="stylesheet" type="text/css" href="../static/css/caliper_report.css"/>
<!-- End : Caliper Head section -->

</head>
<body>
Expand All @@ -54,25 +55,29 @@
<h1> Caliper Benchmarking Report </h1> <!--SKD++-->
<p><b>Report Version & Date : Version-yyyymmdd</b> </p>
</div>
<!-- START Updating data-->
<h2> Introduction </h2>
<p>
Caliper is a test framework focuses on benchmarking and performance
evaluation of server platforms. It can access the performance of ARM as
well as X86 platforms . Caliper allows you to build, execute and
create a benchmarking report of the platforms under evaluation. It can
help to discover issues related to hardware or software.
</p>

<h2> Test classification </h2>
<p>
Caliper has already integrated with many industry standard open source
benchmarking tools. Even though Caliper mainly focuses on the
performance aspects of the system, it gives some insight on the
functional aspects as well. The integrated tools are divided under two
top dimensions, Performance and Functional. Under each of these
categories sub subcategories and then the test cases are organised.
</p>
<!-- START Updating data-->
<p>
<b>Caliper</b> is a benchmarking framework for <b>server platforms</b>, integrated with industry standard tools
and test cases. It provides detailed performance assessments for <b>server platforms</b> based on ARM and x86.
The <b>uniqueness</b> of Caliper is a consolidated score based evaluation. The best performance in each test
will get a score of 100. All other test scores are calculated relative to this. This provides a clear relative
positioning of each target platform. It enables easy analysis of the performance levels and bottlenecks.
</p>

<p>
As the performance testing can depend on system hardware resources at the time of execution, it is bound to vary
over repeated testing. In general, this makes it difficult to analyse the performance results. Caliper tries to
solve it for the performance analysts by executing at least <b>5 iterations of each test on every platform and
ensure low variance</b>. This provides a more believable and stable scores throughout the report.<br>

This version of report is prepared with:
<mark>
5 executions each on 8 target platforms. Each execution consists of 27 test suits collecting 2052 test
points
across all platforms.
</mark>

</p>

<h2> Report presentation </h2>
<p>
Expand Down
40 changes: 28 additions & 12 deletions frontend/polls/templates/polls/latency.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,21 @@
<!-- link the caliper report css here -->
<link rel="stylesheet" type="text/css" href="../static/css/caliper_report.css"/>


<!-- To resize the windows while opening int he new window -->
<script type="text/javascript">
function changeSize( w, h){
window.resizeTo(w,h);
}
function movePosition( x, y){
window.moveTo(x,y);
}
//$( document ).ready(function() {
changeSize(1000,700);
//});
</script>
</head>


<body>
<div class="example">
<div><input type="hidden" id="latency_tst" value="{{ dic_lat }}"> </div>
Expand All @@ -64,13 +77,20 @@ <h1> Caliper Performance Tests: Latency </h1>

<div style="text-align: left;">
<p> The test under this section is mainly targetted towards the performance evaluation of Latency in the sytems through various benchmarking tests which exercises it.</p>
<h1><small>Report Layout </small></h1>
<p> The scores of each scenario shown below is followed by the participating tests cases from each of these scenarios. Remember the scores are based on the percentile ranking across each participating platforms which is already discussed <a href="{% static 'polls/pictures/score_Calculation.html' %}" target="_blank"> <b>here.</b></a> </p>
<h1><small>Test description </small></h1>
<p>
Currently Caliper derives these tests from various well known open source tools to test performance. </p>

<h1><small>Report Layout </small></h1>
<p> The score of each test item is represented as the percentage of performance compared to the highest value obtained across all platform. So a value of <b>x%</b> means that the test case/ scenario performance is <b> x% </b> compared to the highest scored platform, which is always given <b> 100% </b>.</br>
To know the ranking calculation and actual values scored by each platform on specific test cases, please refer to the raw values excel provided along with the report.
Please click <a href="{% static 'polls/pictures/score_Calculation.html' %}" target="_blank"> <b>here</b></a> for more details. </p>
The <b>score summary</b> is the percentage of performance of each platform for individual test scenarios. The detailed sections shows the individual test case score(rank) in the same fashion. <br>

<p> To have a very quick reference of the hardware configurations of platforms evaluated, please refer to <a href="{%static '../static/TargetInfo/Platform_Configuration.xlsx' %}" target ="_blank"> <b>Platform_Configuration.xlsx </b> </a>
<h1><small>Test description </small></h1>
<p> Tests under this section captures the performace of the system measuring latencies for various system operations using following tools: </li>
<li><b>LMBENCH </b>:This is a series of micro benchmarks intended to measure basic operating system and hardware system metrics. </li>
<li><b>RTTEST</b> : This is a set of programs that test and measure various components of real-time kernel behavior such as timer, signal, and hardware latency.</li>

</p>

</div>

{% if sum %}
Expand All @@ -83,13 +103,9 @@ <h1><small>Latency Score Summary</small></h1>

<div id="lat-summary" class="sensei-grid"></div>
{% endif %}
<p><b>FixMe: </b>file/vm overall score above shows PC-I5650_U15.04 as 100. This is has a minor issue.
Based on the analysis of data in detail, <b>MOONSHOT-M710P_U14.04 score should be 100 for file/vm.</b>
<br>Please check the 'Latency Score Details: File/VM Latency' section below. Its very clear.
However, due to a page-fault during the test on PC-I5650_U15.04, there was one unwanted data observed in the test results. Hence, the overall score calculation had a minor issue. This will be fixed in the next report.</p>

{% if ctx %}
<h2><small>Latency Score Details: Context Switch</small></h2>
<h2 id="Context Switch"><small>Latency Score Details: Context Switch</small></h2>
<div style="text-align: center;">
<p> <img src="{% static 'polls/pictures/latency_ctx.png' %}" height="400" width="750" /> </p>
</div>
Expand Down
23 changes: 14 additions & 9 deletions frontend/polls/templates/polls/memory.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,18 @@
<link rel="stylesheet" type="text/css" href="../static/css/caliper_report.css"/>


<!-- To resize the windows while opening int he new window -->
<script type="text/javascript">
function changeSize( w, h){
window.resizeTo(w,h);
}
function movePosition( x, y){
window.moveTo(x,y);
}
//$( document ).ready(function() {
changeSize(1000,700);
//});
</script>
</head>
<body>
<div class="example">
Expand All @@ -72,17 +84,10 @@ <h1><small>Test description </small></h1>
2. <b>Lmbench</b> : lmbench is a series of micro benchmarks intended to measure basic operating system and hardware system metrics.<br>
3. <b>Cachebench</b> : CacheBench is a benchmark designed to evaluate the performance of the memory hierarchy.
</p>
<p>
Currently Caliper executes various benchmark and developed scripts to evaluate the performance focusing the memory subsystem. </p>

<p> To have a very quick reference of the hardware configurations of platforms evaluated, please refer to <a href="{%static '../static/TargetInfo/Platform_Configuration.xlsx' %}" target ="_blank"> <b>Platform_Configuration.xlsx </b> </a>
<h1><small>Report Layout </small></h1>
<p> The scores of each scenario shown below is followed by the participating tests cases from each of these scenarios. Remember the scores are based on the percentile ranking across each participating platforms which is already discussed <a href="score_Calculation.html" target="_blank"> <b>here.</b></a> </p>

</div>
</div>

{% if sum %}
<h1><small>Memory Score Summary</small></h1>
<h1 id="tiny_bandwidth" ><small>Memory Score Summary</small></h1>
<div style="text-align: center;">
<p> <img src="{% static 'polls/pictures/memory_summary.png' %}" height="300"
width="600" /> </p>
Expand Down
18 changes: 10 additions & 8 deletions frontend/polls/templates/polls/network.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -62,21 +62,23 @@ <h1><small>Report Layout </small></h1>
The <b>score summary</b> is the percentage of performance of each platform for individual test scenarios. The detailed sections shows the individual test case score(rank) in the same fashion. <br>
</div>

<div>
<mark><p> <b>Notes: </b> <br>
The caliper testbed network infrastucture is not very good(only 100mbps) hence it does not put any stress on the XGE modules.
We have tested network subsystem using normal lan cable connected network. On the other side you can see that moonshots at HP Lab are having higher bandwith (10G) hence the performance figures will be higher for Moonshot targets.
</p></mark>
</div>



<div style="text-align: left;">
<p> The test under this section is mainly targetted towards the performance evaluation of Network subsystem through various benchmarking tests which exercises the same such as <br>
<h1><small>Test description </small></h1>
<p>
The test under this section is mainly targetted towards the performance evaluation of Network subsystem through various benchmarking tests which exercises the same such as <br>
1. <b>Netperf</b> : Netperf is a benchmark that can be used to measure various aspects of networking performance. <br>
2. <b>Iperf </b>: Iperf is a commonly-used network testing tool that can create Transmission Control Protocol (TCP) and User Datagram Protocol (UDP). <br>
3. <b>Lmbench</b> : lmbench is a series of micro benchmarks intended to measure basic operating system and hardware system metrics. <br>
</p>
<p> To have a very quick reference of the hardware configurations of platforms evaluated, please refer to <a href="{%static '../static/TargetInfo/Platform_Configuration.xlsx' %}" target ="_blank"> <b>Platform_Configuration.xlsx </b> </a></p>
</div>
<div>
<mark><p> <b>Notes: </b> <br>
The caliper testbed network infrastucture is not very good(only 100mbps) hence it does not put any stress on the XGE modules.
We have tested network subsystem using normal lan cable connected network. On the other side you can see that moonshots at HP Lab are having higher bandwith (10G) hence the performance figures will be higher for Moonshot targets.
</p></mark>
</div>
{% if sum %}
<h1><small>Network Score Summary</small></h1>
Expand Down
Loading

0 comments on commit 2031e6a

Please sign in to comment.