Home
Overall architecture was influenced by the discussion found in this blog posting's Section Stacked Generalization & Blending.
| Level 0 Model | Algorithm | Feature Set |
|---|---|---|
| gbm21 | Gradient Boosted Tree | Features selected from expanded Boruta analysis,character attributes set as factor level numbers |
| gbm41 | Gradient Boosted Tree | Features selected from expanded Boruta analysis, numeric attributes set to raw values, NA set to -999,character attributes set one-hot encoding |
| xtc11 | Extra Tree Classifier | All Features in raw format including synthetic features, Categorical represented as integers,numeric NA set to -999 |
| xtc21 | Extra Tree Classifier | All Features in raw format including synthetic features, Categorical represented as integers,numeric NA set to -999 |
| xtc31 | Extra Tree Classifier | Feature set based on Rscipt w/ one synthetic feature, Categorical represented as integers,numeric NA set to -999 |
| xtc51 | Extra Tree Classifier | Numeric features with low correlation, number attributes set to raw values, NA set to -999,Boruta selected character attributes set as one-hot encoding |
| xgb21 | Extreme Gradient Boosting | All Features in raw format including synthetic features, Categorical represented as integers,numeric NA set to -999 |
| xbg31 | Extreme Gradient Boosting | Features selected from expanded Boruta analysis, character attributes set as factor level numbers, set numeric NA to -999 |
Features for this level are the predicted Class 1 probabilities from Level 0. Two Level 1 models are nnet11 (Neural Network) and xgb11 (Extreme Gradient Boosting). This page discusses the approach taken to generate features and train the Level 1 models.
Features for this level are the predicted Class 1 probabilities from Level 1. Level 1 probabilities are combined via weighted geometric mean. Weights were determined by trial and error.
The error metrics for the models used in the final submission are summarized on this page.
This page describes the steps required to build the model and make a submission to Kaggle.