Notes: In D02/D03 platform, the hardware accelerators for algorithms are not utilized. Hence the D02 results may be lower and is expected. We can expect better results once the drivers enable the hardware accelerators
-To have a very quick reference of the hardware configurations of platforms evaluated, please refer to Platform_Configuration.xlsx
+We have preserved block-wise performance details of each cipher across platforms, please refer to Openssl_TestCases.xlsx
@@ -105,7 +117,7 @@Note: The test case names are same for Digital Verify and Digital Sign. However encryption is tested for Digital Sign and decryption is tested for Digital Verify.
{% if hash_algorithm %} -
Tests under this section captures the performace of certain application scenarios such as
- 1. Kernel compilation : Compiling a kernel in multiple threads can avaluate the multi core/multi threding capability of the system.
- 2. Database : Sysbench oltp benchmark evaluate the database performance.
- 3. Hadoop : Hibench measures the common hadoop performance parameters like HadoopTerasort,HadoopWordcount,HadoopBayes,HadoopKmeans,HadoopSort,HadoopDfsioe etc.
- 4. Ebizzy : Web application sever load.
- 5. JDK : Uses SciMark2 to evaluates the Java performance for scientific and numerical computing.
-
The score of each test item is represented as the percentage of performance compared to the highest value obtained across all platform. So a value of x% means that the test case/scenario performance is x% compared to the highest scored platform, which is always given 100% . To know the ranking calculation and actual values scored by each platform on specific test cases, please refer to the raw values excel provided along with the report. Please click here for more details.
The score summary is the percentage of performance of each platform for individual test scenarios. The detailed sections shows the individual test case score(rank) in the same fashion. Notes:
Hadoop performance is based on single node. Soon Caliper to expand the scope of testing multi node performance as well
To have a very quick reference of the hardware configurations of platforms evaluated, please refer to Platform_Configuration.xlsx
+ 
To have a very quick reference of the hardware configurations of platforms evaluated, please refer to Platform_Configuration.xlsx
+ 
To have a very quick reference of the hardware configurations of platforms evaluated, please refer to Platform_Configuration.xlsx
Report Version & Date : Version-yyyymmdd
- Caliper is a test framework focuses on benchmarking and performance -evaluation of server platforms. It can access the performance of ARM as -well as X86 platforms . Caliper allows you to build, execute and -create a benchmarking report of the platforms under evaluation. It can -help to discover issues related to hardware or software. -
- -- Caliper has already integrated with many industry standard open source -benchmarking tools. Even though Caliper mainly focuses on the -performance aspects of the system, it gives some insight on the -functional aspects as well. The integrated tools are divided under two -top dimensions, Performance and Functional. Under each of these -categories sub subcategories and then the test cases are organised. -
+ ++ Caliper is a benchmarking framework for server platforms, integrated with industry standard tools + and test cases. It provides detailed performance assessments for server platforms based on ARM and x86. + The uniqueness of Caliper is a consolidated score based evaluation. The best performance in each test + will get a score of 100. All other test scores are calculated relative to this. This provides a clear relative + positioning of each target platform. It enables easy analysis of the performance levels and bottlenecks. +
+ +
+ As the performance testing can depend on system hardware resources at the time of execution, it is bound to vary
+ over repeated testing. In general, this makes it difficult to analyse the performance results. Caliper tries to
+ solve it for the performance analysts by executing at least 5 iterations of each test on every platform and
+ ensure low variance. This provides a more believable and stable scores throughout the report.
+
+ This version of report is prepared with:
+
+ 5 executions each on 8 target platforms. Each execution consists of 27 test suits collecting 2052 test
+ points
+ across all platforms.
+
+
+
diff --git a/frontend/polls/templates/polls/latency.html b/frontend/polls/templates/polls/latency.html index 153312a4..b3cc98f8 100644 --- a/frontend/polls/templates/polls/latency.html +++ b/frontend/polls/templates/polls/latency.html @@ -51,8 +51,21 @@ - + + + +
The test under this section is mainly targetted towards the performance evaluation of Latency in the sytems through various benchmarking tests which exercises it.
-The scores of each scenario shown below is followed by the participating tests cases from each of these scenarios. Remember the scores are based on the percentile ranking across each participating platforms which is already discussed here.
--Currently Caliper derives these tests from various well known open source tools to test performance.
+ +The score of each test item is represented as the percentage of performance compared to the highest value obtained across all platform. So a value of x% means that the test case/ scenario performance is x% compared to the highest scored platform, which is always given 100% . + To know the ranking calculation and actual values scored by each platform on specific test cases, please refer to the raw values excel provided along with the report. + Please click here for more details.
+ The score summary is the percentage of performance of each platform for individual test scenarios. The detailed sections shows the individual test case score(rank) in the same fashion.To have a very quick reference of the hardware configurations of platforms evaluated, please refer to Platform_Configuration.xlsx +
Tests under this section captures the performace of the system measuring latencies for various system operations using following tools: +
FixMe: file/vm overall score above shows PC-I5650_U15.04 as 100. This is has a minor issue.
- Based on the analysis of data in detail, MOONSHOT-M710P_U14.04 score should be 100 for file/vm.
-
Please check the 'Latency Score Details: File/VM Latency' section below. Its very clear.
- However, due to a page-fault during the test on PC-I5650_U15.04, there was one unwanted data observed in the test results. Hence, the overall score calculation had a minor issue. This will be fixed in the next report.
- Currently Caliper executes various benchmark and developed scripts to evaluate the performance focusing the memory subsystem.
- -To have a very quick reference of the hardware configurations of platforms evaluated, please refer to Platform_Configuration.xlsx -
The scores of each scenario shown below is followed by the participating tests cases from each of these scenarios. Remember the scores are based on the percentile ranking across each participating platforms which is already discussed here.
- - 
Notes:
- The caliper testbed network infrastucture is not very good(only 100mbps) hence it does not put any stress on the XGE modules.
- We have tested network subsystem using normal lan cable connected network. On the other side you can see that moonshots at HP Lab are having higher bandwith (10G) hence the performance figures will be higher for Moonshot targets.
-
The test under this section is mainly targetted towards the performance evaluation of Network subsystem through various benchmarking tests which exercises the same such as
+
+ The test under this section is mainly targetted towards the performance evaluation of Network subsystem through various benchmarking tests which exercises the same such as
1. Netperf : Netperf is a benchmark that can be used to measure various aspects of networking performance.
2. Iperf : Iperf is a commonly-used network testing tool that can create Transmission Control Protocol (TCP) and User Datagram Protocol (UDP).
3. Lmbench : lmbench is a series of micro benchmarks intended to measure basic operating system and hardware system metrics.
To have a very quick reference of the hardware configurations of platforms evaluated, please refer to Platform_Configuration.xlsx
+ Notes:
+ The caliper testbed network infrastucture is not very good(only 100mbps) hence it does not put any stress on the XGE modules.
+ We have tested network subsystem using normal lan cable connected network. On the other side you can see that moonshots at HP Lab are having higher bandwith (10G) hence the performance figures will be higher for Moonshot targets.
+
To have a very quick reference of the hardware configurations of platforms evaluated, please refer to Platform_Configuration.xlsx
+ {% if sum %}