Merge pull request #30 from swerik-project/qes

quality dimension to the right place

.github/workflows/qe_mp-coverage.yml

@@ -0,0 +1,51 @@
+name: "Riksdagen Persons: Estimate MP coverage"
+
+on:
+  pull_request:
+    branches:
+      - 'dev'
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: [3.8]
+    steps:
+    - name: Checkout PR source branch
+      uses: actions/checkout@v4
+      with:
+        ref: ${{ github.head_ref }}
+
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v5
+      with:
+        python-version: ${{ matrix.python-version }}
+
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install pyriksdagen
+
+    - name: Estimate MP coverage
+      run: |
+        python quality/qe_mp-coverage.py
+
+    - name: Plot MP coverage
+      run: |
+        python quality/plot/mp-coverage/mp-coverage.py
+
+    - name: Add and commit changes
+      run: |
+        git config --local user.email "41898282+github-actions[bot]@users.noreply.github.com"
+        git config --local user.name "github-actions[bot]"
+        git add quality/estimates/mp-coverage/coverage-summary.txt
+        git add quality/estimates/mp-coverage/mp-coverage.csv
+        git add quality/estimates/mp-coverage/*.png
+        git commit --allow-empty -m "chore: update MP coverage"
+
+    - name: Push changes
+      uses: ad-m/github-push-action@master
+      with:
+        github_token: ${{ secrets.GITHUB_TOKEN }}
+        branch: ${{ github.head_ref }}

docs/dark-mode/syntax-highlighting.css

@@ -0,0 +1,80 @@
+/* monokai color scheme, see pdoc/template/README.md */
+pre { line-height: 125%; }
+span.linenos { color: inherit; background-color: transparent; padding-left: 5px; padding-right: 20px; }
+.pdoc-code .hll { background-color: #49483e }
+.pdoc-code { background: #272822; color: #f8f8f2 }
+.pdoc-code .c { color: #75715e } /* Comment */
+.pdoc-code .err { color: #960050; background-color: #1e0010 } /* Error */
+.pdoc-code .esc { color: #f8f8f2 } /* Escape */
+.pdoc-code .g { color: #f8f8f2 } /* Generic */
+.pdoc-code .k { color: #66d9ef } /* Keyword */
+.pdoc-code .l { color: #ae81ff } /* Literal */
+.pdoc-code .n { color: #f8f8f2 } /* Name */
+.pdoc-code .o { color: #f92672 } /* Operator */
+.pdoc-code .x { color: #f8f8f2 } /* Other */
+.pdoc-code .p { color: #f8f8f2 } /* Punctuation */
+.pdoc-code .ch { color: #75715e } /* Comment.Hashbang */
+.pdoc-code .cm { color: #75715e } /* Comment.Multiline */
+.pdoc-code .cp { color: #75715e } /* Comment.Preproc */
+.pdoc-code .cpf { color: #75715e } /* Comment.PreprocFile */
+.pdoc-code .c1 { color: #75715e } /* Comment.Single */
+.pdoc-code .cs { color: #75715e } /* Comment.Special */
+.pdoc-code .gd { color: #f92672 } /* Generic.Deleted */
+.pdoc-code .ge { color: #f8f8f2; font-style: italic } /* Generic.Emph */
+.pdoc-code .gr { color: #f8f8f2 } /* Generic.Error */
+.pdoc-code .gh { color: #f8f8f2 } /* Generic.Heading */
+.pdoc-code .gi { color: #a6e22e } /* Generic.Inserted */
+.pdoc-code .go { color: #66d9ef } /* Generic.Output */
+.pdoc-code .gp { color: #f92672; font-weight: bold } /* Generic.Prompt */
+.pdoc-code .gs { color: #f8f8f2; font-weight: bold } /* Generic.Strong */
+.pdoc-code .gu { color: #75715e } /* Generic.Subheading */
+.pdoc-code .gt { color: #f8f8f2 } /* Generic.Traceback */
+.pdoc-code .kc { color: #66d9ef } /* Keyword.Constant */
+.pdoc-code .kd { color: #66d9ef } /* Keyword.Declaration */
+.pdoc-code .kn { color: #f92672 } /* Keyword.Namespace */
+.pdoc-code .kp { color: #66d9ef } /* Keyword.Pseudo */
+.pdoc-code .kr { color: #66d9ef } /* Keyword.Reserved */
+.pdoc-code .kt { color: #66d9ef } /* Keyword.Type */
+.pdoc-code .ld { color: #e6db74 } /* Literal.Date */
+.pdoc-code .m { color: #ae81ff } /* Literal.Number */
+.pdoc-code .s { color: #e6db74 } /* Literal.String */
+.pdoc-code .na { color: #a6e22e } /* Name.Attribute */
+.pdoc-code .nb { color: #f8f8f2 } /* Name.Builtin */
+.pdoc-code .nc { color: #a6e22e } /* Name.Class */
+.pdoc-code .no { color: #66d9ef } /* Name.Constant */
+.pdoc-code .nd { color: #a6e22e } /* Name.Decorator */
+.pdoc-code .ni { color: #f8f8f2 } /* Name.Entity */
+.pdoc-code .ne { color: #a6e22e } /* Name.Exception */
+.pdoc-code .nf { color: #a6e22e } /* Name.Function */
+.pdoc-code .nl { color: #f8f8f2 } /* Name.Label */
+.pdoc-code .nn { color: #f8f8f2 } /* Name.Namespace */
+.pdoc-code .nx { color: #a6e22e } /* Name.Other */
+.pdoc-code .py { color: #f8f8f2 } /* Name.Property */
+.pdoc-code .nt { color: #f92672 } /* Name.Tag */
+.pdoc-code .nv { color: #f8f8f2 } /* Name.Variable */
+.pdoc-code .ow { color: #f92672 } /* Operator.Word */
+.pdoc-code .w { color: #f8f8f2 } /* Text.Whitespace */
+.pdoc-code .mb { color: #ae81ff } /* Literal.Number.Bin */
+.pdoc-code .mf { color: #ae81ff } /* Literal.Number.Float */
+.pdoc-code .mh { color: #ae81ff } /* Literal.Number.Hex */
+.pdoc-code .mi { color: #ae81ff } /* Literal.Number.Integer */
+.pdoc-code .mo { color: #ae81ff } /* Literal.Number.Oct */
+.pdoc-code .sa { color: #e6db74 } /* Literal.String.Affix */
+.pdoc-code .sb { color: #e6db74 } /* Literal.String.Backtick */
+.pdoc-code .sc { color: #e6db74 } /* Literal.String.Char */
+.pdoc-code .dl { color: #e6db74 } /* Literal.String.Delimiter */
+.pdoc-code .sd { color: #e6db74 } /* Literal.String.Doc */
+.pdoc-code .s2 { color: #e6db74 } /* Literal.String.Double */
+.pdoc-code .se { color: #ae81ff } /* Literal.String.Escape */
+.pdoc-code .sh { color: #e6db74 } /* Literal.String.Heredoc */
+.pdoc-code .si { color: #e6db74 } /* Literal.String.Interpol */
+.pdoc-code .sx { color: #e6db74 } /* Literal.String.Other */
+.pdoc-code .sr { color: #e6db74 } /* Literal.String.Regex */
+.pdoc-code .s1 { color: #e6db74 } /* Literal.String.Single */
+.pdoc-code .ss { color: #e6db74 } /* Literal.String.Symbol */
+.pdoc-code .bp { color: #f8f8f2 } /* Name.Builtin.Pseudo */
+.pdoc-code .fm { color: #a6e22e } /* Name.Function.Magic */
+.pdoc-code .vc { color: #f8f8f2 } /* Name.Variable.Class */
+.pdoc-code .vg { color: #f8f8f2 } /* Name.Variable.Global */
+.pdoc-code .vi { color: #f8f8f2 } /* Name.Variable.Instance */
+.pdoc-code .vm { color: #f8f8f2 } /* Name.Variable.Magic */

docs/dark-mode/theme.css

@@ -0,0 +1,20 @@
+:root {
+    --pdoc-background: #212529;
+}
+
+.pdoc {
+    --text: #f7f7f7;
+    --muted: #9d9d9d;
+    --link: #58a6ff;
+    --link-hover: #3989ff;
+    --code: #333;
+    --active: #555;
+
+    --accent: #343434;
+    --accent2: #555;
+
+    --nav-hover: rgba(0, 0, 0, 0.1);
+    --name: #77C1FF;
+    --def: #0cdd0c;
+    --annotation: #00c037;
+}

docs/index.html

@@ -0,0 +1,7 @@
+<!doctype html>
+<html>
+<head>
+    <meta charset="utf-8">
+    <meta http-equiv="refresh" content="0; url=./riksdagen-persons.html"/>
+</head>
+</html>

quality/README.md

@@ -0,0 +1,36 @@
+# Riksdagen Persons: Quality Estimates
+
+This directory contains code and data related to estimating the quality of the Riksdagen Persons metadata corpus, along with the estimates themselves.
+
+## What's here?
+
+### `./`
+
+Python code used to estimate various quality dimensions.
+
+- `qe_mp-coverage.py` : Estimates coverage of MPs over time
+
+
+Support files
+
+- `README.md` : this file
+- `__init__.py` : used for building the documentation
+
+
+### `data/`
+
+Contains data necessary to run the quality estimation code.
+
+
+### `docs/`
+
+Contains explanation and justifications for each quality dimension.
+
+
+### `estimates/`
+
+Contains versioned output of quality estimations.
+
+### `plot/`
+
+Contains code for generating plots from the estimate results

quality/docs/qe_mp-coverage.md

@@ -0,0 +1,20 @@
+# MP Coverage
+
+## Summary
+
+We want to know that we have the correct number of MPs in the metadata for each parliament year and chamber.
+
+
+## What is the problem
+
+It's straightforward to add known information to a database, but less so to estimate information that is missing. In this case we want to make sure that we have enough MPs per parliament year and chamber, but also not too many. Since there are a fixed number of seats in the parliament, we have a fixed baseline against which we can check that we have an adequate number of MPs for each day of conducted parliament business.
+
+
+## Estimation procedure
+
+An extensive list of MPs and their mandate periods have been compiled using Wikidata, Riksdagen Öppna data, and Biografi Banden with the help of students at Uppsala University and experts at the Riksdagenbibliotek.
+
+For each parliament day (date for which there is a Record), this dimension checks the number of MPs with an overlapping mandate and compares that number with the baseline number of MPs who should have been sitting in the parliament.
+
+
+## The code

quality/estimates/mp-coverage/coverage-summary.txt

@@ -0,0 +1,2 @@
+Of 34516 parliament days, 8122 have correct N MPs in metadata: 0.23531116004171979.
+   33783 pass or almost passed within the margin of error, i.e. +- 10%: 0.9787634720129795.