rerendered book

scicloj · Dec 8, 2023 · a3d6b67 · a3d6b67
1 parent f1d3303
commit a3d6b67
Show file tree

Hide file tree

Showing 12 changed files with 5,650 additions and 326 deletions.
diff --git a/docs/datasets.html b/docs/datasets.html
diff --git a/docs/image.html b/docs/image.html
diff --git a/docs/index.html b/docs/index.html
@@ -4040,7 +4040,12 @@ <h1 class="quarto-secondary-nav-title">Noj Documentation</h1>
 </li>
         <li class="sidebar-item">
   <div class="sidebar-item-container"> 
-  <a href="./visualization.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">5</span>&nbsp; <span class="chapter-title">Visualization</span></a>
+  <a href="./stats.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">5</span>&nbsp; <span class="chapter-title">Statistics</span></a>
+  </div>
+</li>
+        <li class="sidebar-item">
+  <div class="sidebar-item-container"> 
+  <a href="./visualization.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">6</span>&nbsp; <span class="chapter-title">Visualization</span></a>
   </div>
 </li>
     </ul>

diff --git a/docs/python.html b/docs/python.html
diff --git a/docs/python.qmd b/docs/python.qmd
diff --git a/docs/search.json b/docs/search.json
diff --git a/docs/stats.html b/docs/stats.html
diff --git a/docs/stats.qmd b/docs/stats.qmd
@@ -0,0 +1,302 @@
+
+---
+format:
+  html: {toc: true, toc-depth: 4, theme: spacelab, output-file: stats.html}
+monofont: Fira Code Medium
+highlight-style: solarized
+code-block-background: true
+include-in-header: {text: '<link rel = "icon" href = "data:," />'}
+embed-resources: true
+
+---
+<style>table {
+  border-style: thin;
+}
+th, td {
+  padding: 6px;
+}
+td {
+  text-align: left;
+}
+th {
+  text-align: center;
+  background-color: #ddd;
+}
+tr:nth-child(even) {
+  background-color: #f6f6f6;
+}
+</style><style>.printedClojure .sourceCode {
+  background-color: transparent;
+  border-style: none;
+}
+</style>
+<script src="stats_files/md-default0.js" type="text/javascript"></script><script src="stats_files/md-default1.js" type="text/javascript"></script>
+
+# Statistics
+
+
+<div class="sourceClojure">
+```clojure
+(ns stats
+  (:require [scicloj.noj.v1.datasets :as datasets]
+            [scicloj.noj.v1.stats :as stats]))
+```
+</div>
+
+
+
+## Correlation matrices
+The `stats/calc-correlations-matrix` function commputes the correlation
+matrix of selected columns of a given dataset,
+organizing the resulting data as a dataset.
+
+
+<div class="sourceClojure">
+```clojure
+(-> datasets/iris
+    (stats/calc-correlations-matrix
+     [:sepal-length :sepal-width :petal-length :petal-width]))
+```
+</div>
+
+
+_unnamed [16 3]:
+
+|        :col-1 |        :col-2 |       :corr |
+|---------------|---------------|------------:|
+| :sepal-length | :sepal-length |  1.00000000 |
+| :sepal-length |  :sepal-width | -0.11000000 |
+| :sepal-length | :petal-length |  0.87000000 |
+| :sepal-length |  :petal-width |  0.81000000 |
+|  :sepal-width | :sepal-length | -0.11000000 |
+|  :sepal-width |  :sepal-width |  1.00000000 |
+|  :sepal-width | :petal-length | -0.41999999 |
+|  :sepal-width |  :petal-width | -0.36000001 |
+| :petal-length | :sepal-length |  0.87000000 |
+| :petal-length |  :sepal-width | -0.41999999 |
+| :petal-length | :petal-length |  1.00000000 |
+| :petal-length |  :petal-width |  0.95999998 |
+|  :petal-width | :sepal-length |  0.81000000 |
+|  :petal-width |  :sepal-width | -0.36000001 |
+|  :petal-width | :petal-length |  0.95999998 |
+|  :petal-width |  :petal-width |  1.00000000 |
+
+
+
+
+## Multivariate regression
+The `stats/regression-model` function computes a regressiom model (using `scicloj.ml`)
+and adds some relevant information such as the `R^2` measure.
+
+
+<div class="sourceClojure">
+```clojure
+(-> datasets/iris
+    (stats/regression-model
+     :sepal-length
+     [:sepal-width :petal-length :petal-width]
+     {:model-type :smile.regression/elastic-net})
+    (dissoc :model-data))
+```
+</div>
+
+
+
+<div class="printedClojure">
+```clojure
+{:feature-columns [:sepal-width :petal-length :petal-width],
+ :target-columns [:sepal-length],
+ :explained #function[malli.core/-instrument/fn--38328],
+ :R2 0.8582120394597336,
+ :id #uuid "eac1ff31-bb6f-4970-86a4-ec7f8df8be67",
+ :predictions #tech.v3.dataset.column&lt;float64&gt;[150]
+:sepal-length
+[5.022, 4.724, 4.775, 4.851, 5.081, 5.360, 4.911, 5.030, 4.664, 4.903, 5.209, 5.098, 4.775, 4.572, 5.184, 5.522, 5.089, 4.970, 5.352, 5.217...],
+ :predict
+ #function[scicloj.noj.v1.stats/regression-model/predict--44543],
+ :options {:model-type :smile.regression/elastic-net}}
+
+```
+</div>
+
+
+
+<div class="sourceClojure">
+```clojure
+(-> datasets/iris
+    (stats/regression-model
+     :sepal-length
+     [:sepal-width :petal-length :petal-width]
+     {:model-type :smile.regression/ordinary-least-square})
+    (dissoc :model-data))
+```
+</div>
+
+
+
+<div class="printedClojure">
+```clojure
+{:feature-columns [:sepal-width :petal-length :petal-width],
+ :target-columns [:sepal-length],
+ :explained #function[malli.core/-instrument/fn--38328],
+ :R2 0.8586117200664085,
+ :id #uuid "9c0c6b79-ecd5-4d84-b3e4-e4d9586c540d",
+ :predictions #tech.v3.dataset.column&lt;float64&gt;[150]
+:sepal-length
+[5.015, 4.690, 4.749, 4.826, 5.080, 5.377, 4.895, 5.021, 4.625, 4.882, 5.216, 5.092, 4.746, 4.533, 5.199, 5.561, 5.094, 4.960, 5.368, 5.226...],
+ :predict
+ #function[scicloj.noj.v1.stats/regression-model/predict--44543],
+ :options {:model-type :smile.regression/ordinary-least-square}}
+
+```
+</div>
+
+
+The `stats/linear-regression-model` convenience function
+uses specifically the `:smile.regression/ordinary-least-square` model type.
+
+
+<div class="sourceClojure">
+```clojure
+(-> datasets/iris
+    (stats/linear-regression-model
+     :sepal-length
+     [:sepal-width :petal-length :petal-width])
+    (dissoc :model-data))
+```
+</div>
+
+
+
+<div class="printedClojure">
+```clojure
+{:feature-columns [:sepal-width :petal-length :petal-width],
+ :target-columns [:sepal-length],
+ :explained #function[malli.core/-instrument/fn--38328],
+ :R2 0.8586117200664085,
+ :id #uuid "ed2fe62e-945f-43fd-b849-8cb6e6bf030d",
+ :predictions #tech.v3.dataset.column&lt;float64&gt;[150]
+:sepal-length
+[5.015, 4.690, 4.749, 4.826, 5.080, 5.377, 4.895, 5.021, 4.625, 4.882, 5.216, 5.092, 4.746, 4.533, 5.199, 5.561, 5.094, 4.960, 5.368, 5.226...],
+ :predict
+ #function[scicloj.noj.v1.stats/regression-model/predict--44543],
+ :options {:model-type :smile.regression/ordinary-least-square}}
+
+```
+</div>
+
+
+
+## Adding regression predictions to a dataset
+The `stats/add-predictions` function
+models a target column using feature columns,
+adds a new prediction column with the model predictions.
+
+
+<div class="sourceClojure">
+```clojure
+(-> datasets/iris
+    (stats/add-predictions
+     :sepal-length
+     [:sepal-width :petal-length :petal-width]
+     {:model-type :smile.regression/ordinary-least-square}))
+```
+</div>
+
+
+_unnamed [150 6]:
+
+| :sepal-length | :sepal-width | :petal-length | :petal-width |  :species | :sepal-length-prediction |
+|--------------:|-------------:|--------------:|-------------:|-----------|-------------------------:|
+|           5.1 |          3.5 |           1.4 |          0.2 |    setosa |               5.01541576 |
+|           4.9 |          3.0 |           1.4 |          0.2 |    setosa |               4.68999718 |
+|           4.7 |          3.2 |           1.3 |          0.2 |    setosa |               4.74925142 |
+|           4.6 |          3.1 |           1.5 |          0.2 |    setosa |               4.82599409 |
+|           5.0 |          3.6 |           1.4 |          0.2 |    setosa |               5.08049948 |
+|           5.4 |          3.9 |           1.7 |          0.4 |    setosa |               5.37719368 |
+|           4.6 |          3.4 |           1.4 |          0.3 |    setosa |               4.89468378 |
+|           5.0 |          3.4 |           1.5 |          0.2 |    setosa |               5.02124524 |
+|           4.4 |          2.9 |           1.4 |          0.2 |    setosa |               4.62491347 |
+|           4.9 |          3.1 |           1.5 |          0.1 |    setosa |               4.88164236 |
+|           ... |          ... |           ... |          ... |       ... |                      ... |
+|           6.9 |          3.1 |           5.4 |          2.1 | virginica |               6.53429168 |
+|           6.7 |          3.1 |           5.6 |          2.4 | virginica |               6.50917327 |
+|           6.9 |          3.1 |           5.1 |          2.3 | virginica |               6.21025556 |
+|           5.8 |          2.7 |           5.1 |          1.9 | virginica |               6.17251376 |
+|           6.8 |          3.2 |           5.9 |          2.3 | virginica |               6.84264484 |
+|           6.7 |          3.3 |           5.7 |          2.5 | virginica |               6.65460564 |
+|           6.7 |          3.0 |           5.2 |          2.3 | virginica |               6.21608504 |
+|           6.3 |          2.5 |           5.0 |          1.9 | virginica |               5.97143313 |
+|           6.5 |          3.0 |           5.2 |          2.0 | virginica |               6.38302984 |
+|           6.2 |          3.4 |           5.4 |          2.3 | virginica |               6.61824630 |
+|           5.9 |          3.0 |           5.1 |          1.8 | virginica |               6.42341317 |
+
+
+
+It attaches the model's information
+to the metadata of that new column.
+
+
+<div class="sourceClojure">
+```clojure
+(-> datasets/iris
+    (stats/add-predictions
+     :sepal-length
+     [:sepal-width :petal-length :petal-width]
+     {:model-type :smile.regression/ordinary-least-square})
+    :sepal-length-prediction
+    meta
+    (update :model
+            dissoc :model-data :predict :predictions))
+```
+</div>
+
+
+
+<div class="printedClojure">
+```clojure
+{:name :sepal-length-prediction,
+ :datatype :float64,
+ :n-elems 150,
+ :column-type :prediction,
+ :model
+ {:feature-columns [:sepal-width :petal-length :petal-width],
+  :target-columns [:sepal-length],
+  :explained #function[malli.core/-instrument/fn--38328],
+  :R2 0.8586117200664085,
+  :id #uuid "4e30bd5d-4abb-4858-a69c-e3a307209958",
+  :options {:model-type :smile.regression/ordinary-least-square}}}
+
+```
+</div>
+
+
+
+## Histograms
+The `stats/histogram` function computes the necessary data
+to plot a histogram.
+
+
+<div class="sourceClojure">
+```clojure
+(-> (repeatedly 99 rand)
+    (stats/histogram {:bin-count 5}))
+```
+</div>
+
+
+_unnamed [5 3]:
+
+| :count |      :left |     :right |
+|-------:|-----------:|-----------:|
+|     15 | 0.00460189 | 0.20230362 |
+|     20 | 0.20230362 | 0.40000535 |
+|     31 | 0.40000535 | 0.59770709 |
+|     20 | 0.59770709 | 0.79540882 |
+|     13 | 0.79540882 | 0.99311055 |
+
+
+
+<div style="background-color:grey;height:2px;width:100%;"></div>
+
+<div><pre><small><small>source: <a href="https://github.com/scicloj/noj/blob/main/notebooks/stats.clj">notebooks/stats.clj</a></small></small></pre></div>
diff --git a/docs/stats_files/md-default0.js b/docs/stats_files/md-default0.js
diff --git a/docs/stats_files/md-default1.js b/docs/stats_files/md-default1.js
diff --git a/docs/visualization.html b/docs/visualization.html