This is an ultimate package for SOTA CI algorithmes
git clone https://github.com/tupoylogin/Neural_Net_Genetic_Alg.git
cd Neural_Net_Genetic_Alg
pip install . (or pip install -e . to enable edit mode)
- Multilayer Perceptron trained with Genetic Algorithm
- Multilayer Perceptron trained with SGD
- Multilayer Perceptron trained with Genetic Algorithm and then with SGD
- Multilayer Perceptron trained with Conjugate SGD
- ANFIS Neural Net trained with SGD
- GroupedMethod+Dense Neural Net with Layer Hypersearch trained with SGD
- GMDH Neural Net with batching on LSM
- Fuzzy GMDH Neural Net with batching on Linear Programming
- GMDH Neural Net on LSM on Time Series data
- Fuzzy GMDH Neural Net on Linear Programming on Time Series data
All experiments are carried on Boston dataset
Using such preprocessing:
- Quantile Transform on Target (
n_quantiles=300, output_distribution="normal") - Standard Scaling of features
Test/Train splitting:
- test size - 20%
- use histogram bins stratification
Metric - MSE on normalized data
| Experiment name | Train score | Test score |
|---|---|---|
| MLP+Genetic | 0.508 | 0.746 |
| MLP+SGD | 0.233 | 0.669 |
| MLP+(Genetic->SGD) | 0.244 | 0.636 |
| MLP+Conjugate SGD | 0.307 | 0.650 |
| ANFIS+SGD | 0.561 | 0.759 |
| GroupedMethod+Dense+SGD+LayerSearch | 0.193 | 0.344 |
| GMDH | 0.732 | 0.423 |
| FuzzyGMDH | 94.0 | 0.432 |