add krr page in cookbook

doc/cookbook/source/examples/regression/kernel_ridge_regression.rst

@@ -0,0 +1,51 @@
+=======================
+Kernel Ridge Regression
+=======================
+
+Kernel ridge regression is a non-parametric form of ridge regression. The aim is to learn a function in the space induced by the respective kernel :math:`k` by minimizing a squared loss with a squared norm regularization term.
+
+The solution can be written in closed form as:
+
+.. math::
+    \alpha = \left({\bf K}+\tau{\bf I}\right)^{-1}{\bf y}
+
+where :math:`{\bf K}` is the kernel matrix and :math:`\alpha` is the vector of weights in the space induced by the kernel.
+The learned function can then be evaluated as :math:`f(x)=\sum_{i=1}^N\alpha_ik(x,x_i)`.
+
+See Chapter 17 in :cite:`barber2012bayesian` for a detailed introduction.
+
+-------
+Example
+-------
+
+Imagine we have files with training and test data. We create `CDenseFeatures` (here 64 bit floats aka RealFeatures) and :sgclass:`CRegressionLabels` as
+
+.. sgexample:: kernel_ridge_regression.sg:create_features
+
+Choose an appropriate :sgclass:`CKernel` and instantiate it. Here we use a :sgclass:`CGaussianKernel`.
+
+.. sgexample:: kernel_ridge_regression.sg:create_appropriate_kernel
+
+We create an instance of :sgclass:`CKernelRidgeRegression` classifier by passing it :math:`\tau`, the kernel and labels.
+
+.. sgexample:: kernel_ridge_regression.sg:create_instance
+
+Then we train the regression model and apply it to test data to get the predicted :sgclass:`CRegressionLabels`.
+
+.. sgexample:: kernel_ridge_regression.sg:train_and_apply
+
+After training, we can extract :math:`\alpha`.
+
+.. sgexample:: kernel_ridge_regression.sg:extract_alpha
+
+Finally, we can evaluate the :sgclass:`CMeanSquaredError`.
+
+.. sgexample:: kernel_ridge_regression.sg:evaluate_error
+
+----------
+References
+----------
+:wiki:`Kernel_method`
+
+.. bibliography:: ../../references.bib
+    :filter: docname in docnames

examples/meta/src/regression/kernel_ridge_regression.sg

@@ -0,0 +1,38 @@
+CSVFile f_feats_train("../../data/regression_1d_sinc_features_train.dat")
+CSVFile f_feats_test("../../data/regression_1d_sinc_features_test.dat")
+CSVFile f_labels_train("../../data/regression_1d_sinc_labels_train.dat")
+CSVFile f_labels_test("../../data/regression_1d_sinc_labels_test.dat")
+
+#![create_features]
+RealFeatures features_train(f_feats_train)
+RealFeatures features_test(f_feats_test)
+RegressionLabels labels_train(f_labels_train)
+RegressionLabels labels_test(f_labels_test)
+#![create_features]
+
+#![create_appropriate_kernel]
+Real width = 1
+GaussianKernel kernel(features_train, features_train, width)
+#![create_appropriate_kernel]
+
+#![create_instance]
+Real tau = 0.001
+KernelRidgeRegression krr(tau, kernel, labels_train)
+#![create_instance]
+
+#![train_and_apply]
+krr.train()
+RegressionLabels labels_predict = krr.apply_regression(features_test)
+#![train_and_apply]
+
+#![extract_alpha]
+RealVector alpha = krr.get_alphas()
+#![extract_alpha]
+
+#![evaluate_error]
+MeanSquaredError eval()
+Real mse = eval.evaluate(labels_predict, labels_test)
+#![evaluate_error]
+
+# integration testing variables
+RealVector output = labels_test.get_labels()