Knowing what to cast the model to is hard

[sfilipi]

I have been writing a few tests about saving the model and reloading the models, and casting the model members to the right types, so I can get to something nested.

Example

`(IEstimator<ITransformer> pipe, IDataView dataView) = GetBinaryClassificationPipeline();

 pipe = pipe.Append(ML.BinaryClassification.Trainers.LogisticRegression(
 new LogisticRegression.Options
 {
     ShowTrainingStatistics = true,
     ComputeStandardDeviation = new ComputeLRTrainingStdThroughMkl(),
 }));

 // SEE THE CASTS
  var transformer = pipe.Fit(dataView) as TransformerChain<BinaryPredictionTransformer<CalibratedModelParametersBase<LinearBinaryModelParameters, PlattCalibrator>>>;

  var linearModel = transformer.LastTransformer.Model.SubModel as LinearBinaryModelParameters;
  var stats = linearModel.Statistics;

 var modelPath = GetOutputPath("temp.zip");
 // Save model. 
 using (var file = File.Create(modelPath))
 transformer.SaveTo(ML, file);

 // Load model.
 TransformerChain<ITransformer> transformerChain;
 using (var file = File.OpenRead(modelPath))
 transformerChain = TransformerChain.LoadFrom(ML, file);

 // SEE THE CASTS
 var lastTransformer = transformerChain.LastTransformer as BinaryPredictionTransformer<IPredictorProducing<float>>;
  var model = lastTransformer.Model as ParameterMixingCalibratedModelParameters<IPredictorWithFeatureWeights<float>, ICalibrator>;
  linearModel = model.SubModel as LinearBinaryModelParameters;
  var stats = linearModel.Statistics;

`

Notice the casts
The only way to get to the Statistics (or weights, bias etc) is by casting to the right type.
It takes living in the Visual Studio debugger to figure out the right types..

[TomFinley]

Hi @sfilipi, doesn't it work if you just keep it as whatever the type was when it was originally created?

It is certainly true that if you upcast to nothing more specific than IEstimator<ITransformer>, you cannot know anything more that the Fit result is returning some transformer. But, of course, we can hardly help that you did that, and you didn't have to do that. You could have just fit on that thing directly, then you would have had the transformer result directly. (You wouldn't even have had to work through the LastTransformer property!)

I had a similar comment to @rogancarr about this. If you care about the individual discrete units of fitting and transformation, keep them discrete? I most recently described our advice here.

[rogancarr]

I still think this is a bit confusing. Ideally, @sfilipi could write a class that fit a model and handed it to me as a pipeline, then I inspect it and do things to it. With what we have here, I have to know beforehand what's inside so I can do manual casting.

[TomFinley]

OK. Could you possibly outline what your ideal scenario might look like? I'm afraid that my understanding of "Ideally @sfilipi could write a class that fit a model and handed it to me as a pipeline" is somewhat vague. Maybe with some thoughts to what the types would be that would enable the scenario you have in mind to actually work?

[sfilipi]

I think the problem is still there for when the model gets deserialized, if the user has saved the entire pipeline, which i think most users will, or for cases when they use the CrossValidate api, where the arguments are the entire pipeline (for cases when the data should get transformed post splitting into folds.)