RooFit::MultiProcess & TestStatistics part 1: mathcore & Minuit2 chan…

…ges (#8369)

* add virtuals to IFunction to calculate second derivative and stepsize

Used in Minuit2 ExternalInternalGradientCalculator (introduced in a later commit) and in RooFit (also later commits) to enable calculation of all derivative and related numbers outside of Minuit2. Specifically, having second derivative and step size calculated outside of Minuit2 allows exact replication of the gradients as calculated in Minuit2 itself, which in turn enables external optimization and parallelization of this process without having to modify Minuit2.

* add second derivative and step size calculation to Minuit2 function adaptors

Like the changes to IFunction in the previous commit, this is also necessary for Minuit2-external gradient calculation.

* add second derivative and step size to Minuit2 parameter transformers and increase their precision

This commit changes two (functionally speaking strictly separate, but unified in their ultimate purpose) things. First, to further accomodate the added ability of external gradient calculators to pass in second derivative and step sizes, the internal to external parameter transformers in Minuit2 need to have added D2Int2Ext and GStepInt2Ext functions that perform these transformations for those added parts. Second, we increase the precision of the transformers to long double. This is necessary  to make sure that trigonometric transformations don't lose a few bits of precision on back-and-forth parameter transformations, which they do when using the double versions. Both changes are necessary for the external gradient calculation in Minuit2 which will be introduced in a next commit in ExternalInternalGradientCalculator.

* implement optional second derivative and step size calculation in AnalyticalGradientCalculator

* remove unnecessary temporary objects

* add ExternalInternalGradientCalculator to minuit2

Derived class from AnalyticalGradientCalculator. Allows Minuit2 using
codes to calculate gradients themselves, but in Minuit2 "internal"
parameter space, in order to preserve exact value correspondence with
the Minuit2 implementation.

* Update math/mathcore/inc/Math/IFunction.h

Co-authored-by: Axel Naumann <Axel.Naumann@cern.ch>

* revert unnecessary math/minuit2 changes

As highlighted by @lmoneta in PR review comments (#8369 (review)), the second derivative and step size information is not necessary at this point. The floating point exact minuit-external (roofit-internal) derivative duplication is currently done using ExternalInternalGradientCalculator and further made possible by the long double parameter transformation functions. This is all we need for now.

* fix previous revert commit + revert one additional thing

Fixes: some merge mistakes + some remaining layout diffs.

The additional reverted thing is the block from line 119 in MnSeedGenerator.cxx.

* fix additional layout

Just trying to keep the PR diff minimal.

* remove copyright line [skip ci]

* add ExternalInternalGradientCalculator doxygen

Co-authored-by: Axel Naumann <Axel.Naumann@cern.ch>

math/mathcore/inc/Math/IFunction.h

@@ -360,7 +360,7 @@ namespace ROOT {
             Gradient(x, df);
          }
 
-
+         virtual bool returnsInMinuit2ParameterSpace() const { return false; }
       };
 
 //___________________________________________________________________________________

math/minuit2/CMakeLists.txt

@@ -28,6 +28,7 @@ if(CMAKE_PROJECT_NAME STREQUAL ROOT)
       Minuit2/CombinedMinimumBuilder.h
       Minuit2/ContoursError.h
       Minuit2/DavidonErrorUpdator.h
+      Minuit2/ExternalInternalGradientCalculator.h
       Minuit2/FCNAdapter.h
       Minuit2/FCNBase.h
       Minuit2/FCNGradAdapter.h
@@ -128,6 +129,7 @@ if(CMAKE_PROJECT_NAME STREQUAL ROOT)
       src/CMakeLists.txt
       src/CombinedMinimumBuilder.cxx
       src/DavidonErrorUpdator.cxx
+      src/ExternalInternalGradientCalculator.cxx
       src/FitterUtil.h
       src/FumiliBuilder.cxx
       src/FumiliErrorUpdator.cxx

math/minuit2/inc/Minuit2/AnalyticalGradientCalculator.h

@@ -35,7 +35,7 @@ class AnalyticalGradientCalculator : public GradientCalculator {
 
    virtual bool CheckGradient() const;
 
-private:
+protected:
    const FCNGradientBase &fGradCalc;
    const MnUserTransformation &fTransformation;
 };

math/minuit2/inc/Minuit2/ExternalInternalGradientCalculator.h

@@ -0,0 +1,49 @@
+// @(#)root/minuit2:$Id$
+// Authors: M. Winkler, F. James, L. Moneta, A. Zsenei, E.G.P. Bos   2003-2017
+
+/**********************************************************************
+ *                                                                    *
+ * Copyright (c) 2005 LCG ROOT Math team,  CERN/PH-SFT                *
+ *                                                                    *
+ **********************************************************************/
+
+#ifndef ROOT_Minuit2_ExternalInternalGradientCalculator
+#define ROOT_Minuit2_ExternalInternalGradientCalculator
+
+#include "Minuit2/AnalyticalGradientCalculator.h"
+
+namespace ROOT {
+
+namespace Minuit2 {
+
+class FCNGradientBase;
+class MnUserTransformation;
+
+/// Similar to the AnalyticalGradientCalculator, the ExternalInternalGradientCalculator
+/// supplies Minuit with an externally calculated gradient. The main difference is that
+/// ExternalInternalGradientCalculator expects that the external gradient calculator does
+/// things in Minuit2-internal parameter space, which means many int2ext and ext2int
+/// transformation steps are not necessary. This avoids loss of precision in some cases,
+/// where trigonometrically transforming parameters back and forth can lose a few bits of
+/// floating point precision on every pass.
+
+class ExternalInternalGradientCalculator : public AnalyticalGradientCalculator {
+
+public:
+   ExternalInternalGradientCalculator(const FCNGradientBase &fcn, const MnUserTransformation &state)
+      : AnalyticalGradientCalculator(fcn, state)
+   {
+   }
+
+   ~ExternalInternalGradientCalculator() {}
+
+   virtual FunctionGradient operator()(const MinimumParameters &) const;
+
+   virtual FunctionGradient operator()(const MinimumParameters &, const FunctionGradient &) const;
+};
+
+} // namespace Minuit2
+
+} // namespace ROOT
+
+#endif // ROOT_Minuit2_ExternalInternalGradientCalculator

math/minuit2/inc/Minuit2/FCNGradAdapter.h

@@ -38,12 +38,12 @@ class FCNGradAdapter : public FCNGradientBase {
 
    ~FCNGradAdapter() {}
 
-   double operator()(const std::vector<double> &v) const { return fFunc.operator()(&v[0]); }
+   double operator()(const std::vector<double> &v) const override { return fFunc.operator()(&v[0]); }
    double operator()(const double *v) const { return fFunc.operator()(v); }
 
-   double Up() const { return fUp; }
+   double Up() const override { return fUp; }
 
-   std::vector<double> Gradient(const std::vector<double> &v) const
+   std::vector<double> Gradient(const std::vector<double> &v) const override
    {
       fFunc.Gradient(&v[0], &fGrad[0]);
 
@@ -56,7 +56,15 @@ class FCNGradAdapter : public FCNGradientBase {
    }
    // forward interface
    // virtual double operator()(int npar, double* params,int iflag = 4) const;
-   bool CheckGradient() const { return false; }
+   bool CheckGradient() const override { return false; }
+
+   GradientParameterSpace gradParameterSpace() const override {
+      if (fFunc.returnsInMinuit2ParameterSpace()) {
+         return GradientParameterSpace::Internal;
+      } else {
+         return GradientParameterSpace::External;
+      }
+   }
 
 private:
    const Function &fFunc;

math/minuit2/inc/Minuit2/FCNGradientBase.h

@@ -1,5 +1,5 @@
 // @(#)root/minuit2:$Id$
-// Authors: M. Winkler, F. James, L. Moneta, A. Zsenei   2003-2005
+// Authors: M. Winkler, F. James, L. Moneta, A. Zsenei, E.G.P. Bos   2003-2017
 
 /**********************************************************************
  *                                                                    *
@@ -31,6 +31,10 @@ namespace Minuit2 {
     "false".
  */
 
+enum class GradientParameterSpace {
+  External, Internal
+};
+
 class FCNGradientBase : public FCNBase {
 
 public:
@@ -39,6 +43,11 @@ class FCNGradientBase : public FCNBase {
    virtual std::vector<double> Gradient(const std::vector<double> &) const = 0;
 
    virtual bool CheckGradient() const { return true; }
+
+   virtual GradientParameterSpace gradParameterSpace() const {
+      return GradientParameterSpace::External;
+   };
+
 };
 
 } // namespace Minuit2

math/minuit2/inc/Minuit2/SinParameterTransformation.h

@@ -1,5 +1,5 @@
 // @(#)root/minuit2:$Id$
-// Authors: M. Winkler, F. James, L. Moneta, A. Zsenei   2003-2005
+// Authors: M. Winkler, F. James, L. Moneta, A. Zsenei, E.G.P. Bos   2003-2017
 
 /**********************************************************************
  *                                                                    *
@@ -28,9 +28,9 @@ class SinParameterTransformation {
 
    ~SinParameterTransformation() {}
 
-   double Int2ext(double Value, double Upper, double Lower) const;
-   double Ext2int(double Value, double Upper, double Lower, const MnMachinePrecision &) const;
-   double DInt2Ext(double Value, double Upper, double Lower) const;
+   long double Int2ext(long double Value, long double Upper, long double Lower) const;
+   long double Ext2int(long double Value, long double Upper, long double Lower, const MnMachinePrecision &) const;
+   long double DInt2Ext(long double Value, long double Upper, long double Lower) const;
 
 private:
 };

math/minuit2/inc/Minuit2/SqrtLowParameterTransformation.h

@@ -1,5 +1,5 @@
 // @(#)root/minuit2:$Id$
-// Authors: M. Winkler, F. James, L. Moneta, A. Zsenei   2003-2005
+// Authors: M. Winkler, F. James, L. Moneta, A. Zsenei, E.G.P. Bos   2003-2017
 
 /**********************************************************************
  *                                                                    *
@@ -35,13 +35,13 @@ class SqrtLowParameterTransformation /* : public ParameterTransformation */ {
    ~SqrtLowParameterTransformation() {}
 
    // transformation from internal to external
-   double Int2ext(double Value, double Lower) const;
+   long double Int2ext(long double Value, long double Lower) const;
 
    // transformation from external to internal
-   double Ext2int(double Value, double Lower, const MnMachinePrecision &) const;
+   long double Ext2int(long double Value, long double Lower, const MnMachinePrecision &) const;
 
    // derivative of transformation from internal to external
-   double DInt2Ext(double Value, double Lower) const;
+   long double DInt2Ext(long double Value, long double Lower) const;
 
 private:
 };

math/minuit2/inc/Minuit2/SqrtUpParameterTransformation.h

@@ -1,5 +1,5 @@
 // @(#)root/minuit2:$Id$
-// Authors: M. Winkler, F. James, L. Moneta, A. Zsenei   2003-2005
+// Authors: M. Winkler, F. James, L. Moneta, A. Zsenei, E.G.P. Bos   2003-2017
 
 /**********************************************************************
  *                                                                    *
@@ -36,13 +36,13 @@ class SqrtUpParameterTransformation /* : public ParameterTransformation */ {
    ~SqrtUpParameterTransformation() {}
 
    // transformation from internal to external
-   double Int2ext(double Value, double Upper) const;
+   long double Int2ext(long double Value, long double Upper) const;
 
    // transformation from external to internal
-   double Ext2int(double Value, double Upper, const MnMachinePrecision &) const;
+   long double Ext2int(long double Value, long double Upper, const MnMachinePrecision &) const;
 
    // derivative of transformation from internal to external
-   double DInt2Ext(double Value, double Upper) const;
+   long double DInt2Ext(long double Value, long double Upper) const;
 
 private:
 };

math/minuit2/src/CMakeLists.txt

@@ -17,6 +17,7 @@ set(MINUIT2_HEADERS
     CombinedMinimumBuilder.h
     ContoursError.h
     DavidonErrorUpdator.h
+    ExternalInternalGradientCalculator.h
     FCNAdapter.h
     FCNBase.h
     FCNGradAdapter.h
@@ -117,6 +118,7 @@ set(MINUIT2_SOURCES
     BFGSErrorUpdator.cxx
     CombinedMinimumBuilder.cxx
     DavidonErrorUpdator.cxx
+    ExternalInternalGradientCalculator.cxx
     FumiliBuilder.cxx
     FumiliErrorUpdator.cxx
     FumiliGradientCalculator.cxx

math/minuit2/src/ExternalInternalGradientCalculator.cxx

@@ -0,0 +1,53 @@
+// @(#)root/minuit2:$Id$
+// Authors: M. Winkler, F. James, L. Moneta, A. Zsenei, E.G.P. Bos   2003-2017
+
+/**********************************************************************
+ *                                                                    *
+ * Copyright (c) 2005 LCG ROOT Math team,  CERN/PH-SFT                *
+ *                                                                    *
+ **********************************************************************/
+
+#include <vector>
+#include "Minuit2/ExternalInternalGradientCalculator.h"
+#include "Minuit2/FCNGradientBase.h"
+#include "Minuit2/MnUserTransformation.h"
+#include "Minuit2/FunctionGradient.h"
+#include "Minuit2/MinimumParameters.h"
+
+namespace ROOT {
+namespace Minuit2 {
+
+FunctionGradient ExternalInternalGradientCalculator::operator()(const MinimumParameters &par) const
+{
+   // evaluate analytical gradient. take care of parameter transformations
+   std::vector<double> par_vec;
+   par_vec.resize(par.Vec().size());
+   for (std::size_t ix = 0; ix < par.Vec().size(); ++ix) {
+      par_vec[ix] = par.Vec()(ix);
+   }
+
+   std::vector<double> grad = fGradCalc.Gradient(par_vec);
+   assert(grad.size() == fTransformation.Parameters().size());
+
+   MnAlgebraicVector v(par.Vec().size());
+   for (unsigned int i = 0; i < par.Vec().size(); i++) {
+      unsigned int ext = fTransformation.ExtOfInt(i);
+      v(i) = grad[ext];
+   }
+
+#ifdef DEBUG
+   std::cout << "User given gradient in Minuit2" << v << std::endl;
+#endif
+
+   return FunctionGradient(v);
+}
+
+FunctionGradient
+ExternalInternalGradientCalculator::operator()(const MinimumParameters &par, const FunctionGradient &) const
+{
+   // needed from base class
+   return (*this)(par);
+}
+
+} // namespace Minuit2
+} // namespace ROOT

math/minuit2/src/ModularFunctionMinimizer.cxx

@@ -1,5 +1,5 @@
 // @(#)root/minuit2:$Id$
-// Authors: M. Winkler, F. James, L. Moneta, A. Zsenei   2003-2005
+// Authors: M. Winkler, F. James, L. Moneta, A. Zsenei, E.G.P. Bos   2003-2017
 
 /**********************************************************************
  *                                                                    *
@@ -10,6 +10,7 @@
 #include "Minuit2/ModularFunctionMinimizer.h"
 #include "Minuit2/MinimumSeedGenerator.h"
 #include "Minuit2/AnalyticalGradientCalculator.h"
+#include "Minuit2/ExternalInternalGradientCalculator.h"
 #include "Minuit2/Numerical2PGradientCalculator.h"
 #include "Minuit2/MinimumBuilder.h"
 #include "Minuit2/MinimumSeed.h"
@@ -148,7 +149,14 @@ FunctionMinimum ModularFunctionMinimizer::Minimize(const FCNGradientBase &fcn, c
    // Create the minuit FCN wrapper (MnUserFcn) containing the trasformation (int<->ext)
 
    MnUserFcn mfcn(fcn, st.Trafo());
-   AnalyticalGradientCalculator gc(fcn, st.Trafo());
+   AnalyticalGradientCalculator *gc;
+   if (fcn.gradParameterSpace() == GradientParameterSpace::Internal) {
+        //        std::cout << "-- ModularFunctionMinimizer::Minimize: Internal parameter space" << std::endl;
+        gc = new ExternalInternalGradientCalculator(fcn, st.Trafo());
+   } else {
+        //        std::cout << "-- ModularFunctionMinimizer::Minimize: External parameter space" << std::endl;
+        gc = new AnalyticalGradientCalculator(fcn, st.Trafo());
+   }
 
    unsigned int npar = st.VariableParameters();
    if (maxfcn == 0)
@@ -158,7 +166,11 @@ FunctionMinimum ModularFunctionMinimizer::Minimize(const FCNGradientBase &fcn, c
    Numerical2PGradientCalculator numgc(mfcn, st.Trafo(), strategy);
    MinimumSeed mnseeds = SeedGenerator()(mfcn, numgc, st, strategy);
 
-   return Minimize(mfcn, gc, mnseeds, strategy, maxfcn, toler);
+   auto minimum = Minimize(mfcn, *gc, mnseeds, strategy, maxfcn, toler);
+
+   delete gc;
+
+   return minimum;
 }
 
 FunctionMinimum ModularFunctionMinimizer::Minimize(const MnFcn &mfcn, const GradientCalculator &gc,

math/minuit2/src/SinParameterTransformation.cxx

@@ -1,5 +1,5 @@
 // @(#)root/minuit2:$Id$
-// Authors: M. Winkler, F. James, L. Moneta, A. Zsenei   2003-2005
+// Authors: M. Winkler, F. James, L. Moneta, A. Zsenei, E.G.P. Bos   2003-2017
 
 /**********************************************************************
  *                                                                    *
@@ -16,25 +16,25 @@ namespace ROOT {
 
 namespace Minuit2 {
 
-double SinParameterTransformation::Int2ext(double Value, double Upper, double Lower) const
+long double SinParameterTransformation::Int2ext(long double Value, long double Upper, long double Lower) const
 {
    // transformation from  to internal (unlimited) to external values (limited by Lower/Upper )
    return Lower + 0.5 * (Upper - Lower) * (std::sin(Value) + 1.);
 }
 
-double
-SinParameterTransformation::Ext2int(double Value, double Upper, double Lower, const MnMachinePrecision &prec) const
+long double
+SinParameterTransformation::Ext2int(long double Value, long double Upper, long double Lower, const MnMachinePrecision &prec) const
 {
    // transformation from external (limited by Lower/Upper )  to internal (unlimited) values given the lower/upper
    // limits
 
-   double piby2 = 2. * std::atan(1.);
-   double distnn = 8. * std::sqrt(prec.Eps2());
-   double vlimhi = piby2 - distnn;
-   double vlimlo = -piby2 + distnn;
+   long double piby2 = 2. * std::atan(1.);
+   long double distnn = 8. * std::sqrt(prec.Eps2());
+   long double vlimhi = piby2 - distnn;
+   long double vlimlo = -piby2 + distnn;
 
-   double yy = 2. * (Value - Lower) / (Upper - Lower) - 1.;
-   double yy2 = yy * yy;
+   long double yy = 2. * (Value - Lower) / (Upper - Lower) - 1.;
+   long double yy2 = yy * yy;
    if (yy2 > (1. - prec.Eps2())) {
       if (yy < 0.) {
          // Lower limit
@@ -45,13 +45,12 @@ SinParameterTransformation::Ext2int(double Value, double Upper, double Lower, co
          //       std::cout<<"SinParameterTransformation warning: is at its Upper allowed limit."<<std::endl;
          return vlimhi;
       }
-
    } else {
       return std::asin(yy);
    }
 }
 
-double SinParameterTransformation::DInt2Ext(double Value, double Upper, double Lower) const
+long double SinParameterTransformation::DInt2Ext(long double Value, long double Upper, long double Lower) const
 {
    // return the derivative of the transformation d Ext/ d Int
    return 0.5 * ((Upper - Lower) * std::cos(Value));