warm starting error because of empty array in the search space

[prabhant]

Extratree classifier has

       "min_samples_split": [],
        "min_samples_leaf": [],

It looks like this is the reason i can't add Individuals with any value for these parameters in GAMA during warm_start(these individuals were discovered by GAMA)

MWE:

12 
     13 automl = GamaClassifier(max_total_time=3000, n_jobs=1)
---> 14 Individual.from_string("ExtraTreesClassifier(SelectFwe(StandardScaler(data), SelectFwe.score_func=f_classif, SelectFwe.alpha=0.035), ExtraTreesClassifier.bootstrap=False, ExtraTreesClassifier.max_features=0.45, ExtraTreesClassifier.n_estimators=100, ExtraTreesClassifier.criterion='entropy', ExtraTreesClassifier.min_samples_split=6)", automl._pset, automl._operator_set._compile)
     15 
     16 d = openml.datasets.get_dataset(151)

3 frames
[/usr/local/lib/python3.7/dist-packages/gama/genetic_programming/components/primitive_node.py](https://localhost:8080/#) in find_terminal(primitive_set, terminal_string)
    120         if repr(terminal) == terminal_string:
    121             return terminal
--> 122     raise RuntimeError(f"Could not find Terminal of type '{terminal_string}'.")

RuntimeError: Could not find Terminal of type 'ExtraTreesClassifier.min_samples_split=6'.

[PGijsbers]

For the MWE please include relevant import statement and also post it as code snippet, not just stack trace. That makes running it locally easier.

The way the hyperparameters are defined means they rely on shared values at the top of the config:

    "min_samples_split": range(2, 21),
    "min_samples_leaf": range(1, 21),

which means they don't need to be prefixed with the classifier:

Individual.from_string("ExtraTreesClassifier(SelectFwe(StandardScaler(data), SelectFwe.score_func=f_classif, SelectFwe.alpha=0.035), ExtraTreesClassifier.bootstrap=False, ExtraTreesClassifier.max_features=0.45, ExtraTreesClassifier.n_estimators=100, ExtraTreesClassifier.criterion='entropy', 
-ExtraTreesClassifier.min_samples_split=6
+min_samples_split=6, min_samples_leaf=1
)", automl._pset, automl._operator_set._compile)

I am not sure why min_samples_leaf was not included in the original string representation, since it's also part of the (original) definition.

[prabhant]

Sorry missed the MWE code, here you go:

from gama.genetic_programming.components import Individual, PrimitiveNode
from gama import GamaClassifier

automl = GamaClassifier(max_total_time=3000, n_jobs=1)
Individual.from_string("ExtraTreesClassifier(SelectFwe(StandardScaler(data), SelectFwe.score_func=f_classif, SelectFwe.alpha=0.035), ExtraTreesClassifier.bootstrap=False, ExtraTreesClassifier.min_samples_leaf=3, ExtraTreesClassifier.max_features=0.45, ExtraTreesClassifier.n_estimators=100, ExtraTreesClassifier.criterion='entropy', ExtraTreesClassifier.min_samples_split=6)", automl._pset, automl._operator_set._compile)

[PGijsbers]

Yes, so this should work:

from gama.genetic_programming.components import Individual, PrimitiveNode
from gama import GamaClassifier

automl = GamaClassifier(max_total_time=3000, n_jobs=1)
-Individual.from_string("ExtraTreesClassifier(SelectFwe(StandardScaler(data), SelectFwe.score_func=f_classif, SelectFwe.alpha=0.035), ExtraTreesClassifier.bootstrap=False, ExtraTreesClassifier.min_samples_leaf=3, ExtraTreesClassifier.max_features=0.45, ExtraTreesClassifier.n_estimators=100, ExtraTreesClassifier.criterion='entropy', ExtraTreesClassifier.min_samples_split=6)", automl._pset, automl._operator_set._compile)
+Individual.from_string("ExtraTreesClassifier(SelectFwe(StandardScaler(data), SelectFwe.score_func=f_classif, SelectFwe.alpha=0.035), ExtraTreesClassifier.bootstrap=False, min_samples_leaf=3, ExtraTreesClassifier.max_features=0.45, ExtraTreesClassifier.n_estimators=100, ExtraTreesClassifier.criterion='entropy', min_samples_split=6)", automl._pset, automl._operator_set._compile)

[prabhant]

thanks closing this now