Rewrite of HTML tables with React (#477)

This provides a complete rewrite of the HTML tables that `table-generator` produces. Instead of statically creating a HTML `<table>`, we now insert the data as JSON and use [React](https://reactjs.org/) to render a table dynamically. Furthermore, all the spaghetti code for interactive features like plots is now completely replaced by a proper architecture using React components, and there exists infrastructure to test this.

Some new features like better row filters and improvements have also been added based on a user study. But the main advantage is that it is now possible to open large tables in browsers without long waiting periods.

The studies and the rewrite were done by @bschor and a report about this is available as her [Bachelor's thesis](https://www.sosy-lab.org/research/bsc/2019.Bschor.Modern_Architecture_and_Improved_UI_for_Tables_of_BenchExec.pdf).

Users who whish to continue using the old tables can pass `--static-table` to `table-generator`, but we expect this to be removed in BenchExec 3.0.

Fixes #130 (sorting added)
Fixes #423 (summary tab with only header and footer of table added)
Closes #123 (all Javascript is now bundled in the HTML file)
Closes #195 (all Javascript is now bundled in the HTML file)
Closes #214 (filtering is now longer in an overlay, and much faster anyway)
Closes #237 ([react-table](https://www.npmjs.com/package/react-table) is used)
Closes #296 (main table has only small header, full header is shown on summary tab)
Closes #297 (main table has only small header, full header is shown on summary tab)
Closes #375 (no longer applicable)

.gitignore

@@ -1,6 +1,6 @@
 *.pyc
 *~
-build
+/build
 dist
 BenchExec.egg-info
 .coverage.*
@@ -11,3 +11,4 @@ coverage
 /.idea/vcs.xml
 /.idea/workspace.xml
 /benchexec/.idea
+node_modules/

.gitlab-ci.yml

@@ -102,6 +102,50 @@ flake8:
   allow_failure: true # need to fix existing code first
 
 
+# For HTML tables, test that bundled files are uptodate
+javascript-build:
+  stage: test
+  image: node
+  before_script:
+    - cd benchexec/tablegenerator/react-table
+    - npm install
+  script:
+    - npm run build
+    - git diff --stat --exit-code
+  cache:
+    key: "$CI_JOB_NAME"
+    paths:
+      - "benchexec/tablegenerator/react-table/node_modules"
+  allow_failure: true # not sure why build on CI produces different output than locally
+
+javascript-tests:
+  stage: test
+  image: node
+  before_script:
+    - cd benchexec/tablegenerator/react-table
+    - npm install
+  script:
+    - npm run test
+  cache:
+    key: "$CI_JOB_NAME"
+    paths:
+      - "benchexec/tablegenerator/react-table/node_modules"
+
+javascript-eslint:
+  stage: test
+  image: node
+  before_script:
+    - cd benchexec/tablegenerator/react-table
+    - npm install
+  script:
+    - npx eslint src --max-warnings 0
+    - 'npx prettier $(find . \( -name build -o -name node_modules \) -prune -type f -o -name "*.js" -o -name "*.json" -o -name "*.css" -o -name "*.scss") --ignore-path .gitignore --check'
+  cache:
+    key: "$CI_JOB_NAME"
+    paths:
+      - "benchexec/tablegenerator/react-table/node_modules"
+
+
 # Build Docker images
 # following this guideline: https://docs.gitlab.com/ee/ci/docker/using_kaniko.html
 .build-docker:

MANIFEST.in

@@ -1,6 +1,8 @@
 include *.md
 include LICENSE
-include benchexec/tablegenerator/template.*
+include benchexec/tablegenerator/template*
+recursive-include benchexec/tablegenerator/react-table/build *.min.js
+recursive-include benchexec/tablegenerator/react-table/build *.min.css
 recursive-include bin *
 recursive-include contrib *
 recursive-include doc *

README.md

@@ -76,6 +76,7 @@ Maintainer: [Philipp Wendler](https://www.philippwendler.de)
 Contributors:
 - [Aditya Arora](https://github.com/alohamora)
 - [Dirk Beyer](https://www.sosy-lab.org/people/beyer/)
+- [Laura Bschor](https://github.com/laurabschor)
 - [Thomas Bunk](https://github.com/TBunk)
 - [Montgomery Carter](https://github.com/MontyCarter)
 - [Andreas Donig](https://github.com/adonig)
@@ -91,6 +92,7 @@ Contributors:
 - [Stephan Lukasczyk](https://github.com/stephanlukasczyk)
 - [Alexander von Rhein](http://www.infosun.fim.uni-passau.de/se/people-rhein.php)
 - [Alexander Schremmer](https://www.xing.com/profile/Alexander_Schremmer)
+- [Dennis Simon](https://github.com/DennisSimon)
 - [Andreas Stahlbauer](http://stahlbauer.net/)
 - [Thomas Stieglmaier](https://stieglmaier.me/)
 - [Ilja Zakharov](https://github.com/IljaZakharov)

benchexec/tablegenerator/__init__.py

@@ -75,9 +75,17 @@
 LIB_URL = "https://cdn.jsdelivr.net"
 LIB_URL_OFFLINE = "lib/javascript"
 
-TEMPLATE_FILE_NAME = os.path.join(os.path.dirname(__file__), "template.{format}")
+
+REACT_FILES = [
+    os.path.join(os.path.dirname(__file__), "react-table", "build", path)
+    for path in ["vendors.min.", "bundle.min."]
+]
+
+TEMPLATE_NAME = "template"
+TEMPLATE_NAME_REACT = "template_react"
+TEMPLATE_FILE_NAME = os.path.join(os.path.dirname(__file__), "{template}.{format}")
+
 TEMPLATE_FORMATS = ["html", "csv"]
-TEMPLATE_ENCODING = "UTF-8"
 TEMPLATE_NAMESPACE = {
     "flatten": Util.flatten,
     "json": Util.to_json,
@@ -1862,13 +1870,38 @@ def create_tables(
     )
     template_values.version = __version__
 
+    # prepare data for js react application
+    if options.template_name == TEMPLATE_NAME_REACT:
+        template_values.tools = Util.prepare_run_sets_for_js(
+            template_values.run_sets, template_values.columns
+        )
+        template_values.rows = Util.prepare_rows_for_js(
+            rows, template_values.tools, outputPath, template_values.href_base
+        )
+
+        template_values.app_css = [
+            Util.read_bundled_file(path + "css") for path in REACT_FILES
+        ]
+        template_values.app_js = [
+            Util.read_bundled_file(path + "js") for path in REACT_FILES
+        ]
+        # template_values.stats = <see below>
+    else:
+        logging.warning("Option --static-table will be removed in BenchExec 3.0.")
+
     futures = []
 
     def write_table(table_type, title, rows, use_local_summary):
         # calculate statistics if necessary
         if not options.format == ["csv"]:
             local_summary = get_summary(runSetResults) if use_local_summary else None
             stats, stats_columns = get_stats(rows, local_summary, options.correct_only)
+
+            # prepare data for js react application (stats)
+            if options.template_name == TEMPLATE_NAME_REACT:
+                template_values.stats = Util.prepare_stats_for_js(
+                    stats, template_values.tools
+                )
         else:
             stats = stats_columns = None
 
@@ -1901,6 +1934,9 @@ def write_table(table_type, title, rows, use_local_summary):
                     outfile,
                     this_template_values,
                     options.show_table and template_format == "html",
+                    options.template_name
+                    if template_format == "html"
+                    else TEMPLATE_NAME,
                 )
             )
 
@@ -1919,15 +1955,15 @@ def write_table(table_type, title, rows, use_local_summary):
     return futures
 
 
-def write_table_in_format(template_format, outfile, template_values, show_table):
+def write_table_in_format(
+    template_format, outfile, template_values, show_table, template_name
+):
     # read template
     Template = tempita.HTMLTemplate if template_format == "html" else tempita.Template
-    template_file = TEMPLATE_FILE_NAME.format(format=template_format)
-    try:
-        template_content = __loader__.get_data(template_file).decode(TEMPLATE_ENCODING)
-    except NameError:
-        with open(template_file, mode="r") as f:
-            template_content = f.read()
+    template_file = TEMPLATE_FILE_NAME.format(
+        template=template_name, format=template_format
+    )
+    template_content = Util.read_bundled_file(template_file)
     template = Template(template_content, namespace=TEMPLATE_NAMESPACE)
 
     result = template.substitute(**template_values)
@@ -2025,6 +2061,16 @@ def create_argument_parser():
         choices=TEMPLATE_FORMATS,
         help="Which format to generate (HTML or CSV). Can be specified multiple times. If not specified, all are generated.",
     )
+    parser.add_argument(
+        "--static-table",
+        action="store_const",
+        dest="template_name",
+        const=TEMPLATE_NAME,
+        default=TEMPLATE_NAME_REACT,
+        help="Generate HTML table with static HTML code as known until BenchExec 2.2 "
+        "instead of the new React-based table. "
+        "This option will be removed in BenchExec 3.0.",
+    )
     parser.add_argument(
         "-c",
         "--common",
@@ -2057,7 +2103,7 @@ def create_argument_parser():
         const=LIB_URL_OFFLINE,
         default=LIB_URL,
         help="Expect JS libs in libs/javascript/ instead of retrieving them from a CDN. "
-        "Currently does not work for all libs.",
+        "Currently does not work for all libs, and only relevant for --static-table.",
     )
     parser.add_argument(
         "--show",

benchexec/tablegenerator/react-table/.gitignore

@@ -0,0 +1,28 @@
+# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.
+
+# generated files
+/src/data/dependencies.json
+
+# dependencies
+/node_modules
+/.pnp
+.pnp.js
+
+# testing
+/coverage
+
+# production - include bundle JS
+/build/*
+!/build/*.min.js
+!/build/*.min.css
+
+# misc
+.DS_Store
+.env.local
+.env.development.local
+.env.test.local
+.env.production.local
+
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*

benchexec/tablegenerator/react-table/README.md

@@ -0,0 +1,79 @@
+This project was bootstrapped with [Create React App](https://github.com/facebook/create-react-app).
+
+## Available Scripts
+
+In the project directory, you can run:
+
+### `npm start`
+
+Runs the app in the development mode.<br>
+Open [http://localhost:3000](http://localhost:3000) to view it in the browser.
+
+The page will reload if you make edits.<br>
+You will also see any lint errors in the console.
+
+### `npm test`
+
+Launches the test runner in the interactive watch mode.<br>
+See the section about [running tests](https://facebook.github.io/create-react-app/docs/running-tests) for more information.
+
+## Test Datasets
+
+There are 4 different Data sets to test the App in the folder data.
+Change the data:
+
+- Go to Overview.js line 2, there the data is imported
+- Change the filter to the one you want from the data folder (e.g. new_middle.json)
+- Save the file
+
+### next section is not important for the normal testing
+
+### `npm run build`
+
+Builds the app for production to the `build` folder.<br>
+It correctly bundles React in production mode and optimizes the build for the best performance.
+
+The build is minified and the filenames include the hashes.<br>
+Your app is ready to be deployed!
+
+See the section about [deployment](https://facebook.github.io/create-react-app/docs/deployment) for more information.
+
+### `npm run eject`
+
+**Note: this is a one-way operation. Once you `eject`, you can’t go back!**
+
+If you aren’t satisfied with the build tool and configuration choices, you can `eject` at any time. This command will remove the single build dependency from your project.
+
+Instead, it will copy all the configuration files and the transitive dependencies (Webpack, Babel, ESLint, etc) right into your project so you have full control over them. All of the commands except `eject` will still work, but they will point to the copied scripts so you can tweak them. At this point you’re on your own.
+
+You don’t have to ever use `eject`. The curated feature set is suitable for small and middle deployments, and you shouldn’t feel obligated to use this feature. However we understand that this tool wouldn’t be useful if you couldn’t customize it when you are ready for it.
+
+## Learn More
+
+You can learn more in the [Create React App documentation](https://facebook.github.io/create-react-app/docs/getting-started).
+
+To learn React, check out the [React documentation](https://reactjs.org/).
+
+### Code Splitting
+
+This section has moved here: https://facebook.github.io/create-react-app/docs/code-splitting
+
+### Analyzing the Bundle Size
+
+This section has moved here: https://facebook.github.io/create-react-app/docs/analyzing-the-bundle-size
+
+### Making a Progressive Web App
+
+This section has moved here: https://facebook.github.io/create-react-app/docs/making-a-progressive-web-app
+
+### Advanced Configuration
+
+This section has moved here: https://facebook.github.io/create-react-app/docs/advanced-configuration
+
+### Deployment
+
+This section has moved here: https://facebook.github.io/create-react-app/docs/deployment
+
+### `npm run build` fails to minify
+
+This section has moved here: https://facebook.github.io/create-react-app/docs/troubleshooting#npm-run-build-fails-to-minify