Code

Ss test mapie#41

Code

Open

ssorou1 wants to merge 79 commits intoNOAA-OWP:devNOAA-OWP/formulation-selector:devfrom ssorou1:ss_test_mapiessorou1/formulation-selector:ss_test_mapieCopy head branch name to clipboard

+264-72

Commits

Commits on Jan 22, 2025

• bring in the ci for rf in fs_algo_train_eval.py

  

  Soroush Sorourian

  committed
  31791cb

  Copy full SHA for 31791cb

• bring in ci to fs_proc_algo.py

  

  Soroush Sorourian

  committed
  dcc6c5a

  Copy full SHA for dcc6c5a

• bring in ci to fs_pred_algo.py

  

  Soroush Sorourian

  committed
  1805ec0

  Copy full SHA for 1805ec0

• apply only one n_estimators (grid selection bug)

  

  Soroush Sorourian

  committed
  93ede73

  Copy full SHA for 93ede73

• fix a syntax error in fs_algo_train_eval

  

  Soroush Sorourian

  committed
  2d3dbfa

  Copy full SHA for 2d3dbfa

• clean fs_algo_train_eval.py

  

  Soroush Sorourian

  committed
  1154eef

  Copy full SHA for 1154eef

• added unit test for std_Xtrain_path function

  

  Soroush Sorourian

  committed
  50e790c

  Copy full SHA for 50e790c

• Revert "Merge remote-tracking branch 'upstream/dev' into ss_test_fci_dev for parquet column features"

  Show description for ffbbc62
  

  Soroush Sorourian

  committed
  ffbbc62

  Copy full SHA for ffbbc62

• added unit test for fci function

  

  Soroush Sorourian

  committed
  bdb7d32

  Copy full SHA for bdb7d32

• Update fs_pred_algo.py

  Show description for cb0efe1
  glitt13

  authored
  cb0efe1

  Copy full SHA for cb0efe1

• brought back list of values in n_estimators in xssa_algo_config.yaml

  

  Soroush Sorourian

  committed
  5eb00e4

  Copy full SHA for 5eb00e4

• Merge branch 'ss_test_fci_dev3' of https://github.com/ssorou1/formulation-selector into ss_test_fci_dev3

  

  Soroush Sorourian

  committed
  c310236

  Copy full SHA for c310236

Commits on Jan 28, 2025

• Incorporate Bagging into mlp in fs_algo_train_eval

  

  Soroush Sorourian

  committed
  737c36a

  Copy full SHA for 737c36a

• rf n_estimators=400

  

  Soroush Sorourian

  committed
  88661ad

  Copy full SHA for 88661ad

• Incorporate Bagging into rf in fs_algo_train_eval

  

  Soroush Sorourian

  committed
  ea914f4

  Copy full SHA for ea914f4

Commits on Jan 31, 2025

• incorporated mapie for rf model

  

  Soroush Sorourian

  committed
  ad24727

  Copy full SHA for ad24727

• update the MAPIE to use the same fit as rf

  

  Soroush Sorourian

  committed
  983a662

  Copy full SHA for 983a662

• incorporated mapie for mlp model

  

  Soroush Sorourian

  committed
  710038a

  Copy full SHA for 710038a

• deleted unsed pred_rf variable inside tran_algos function

  

  Soroush Sorourian

  committed
  d24e19f

  Copy full SHA for d24e19f

Commits on Feb 6, 2025

• add number of bootstrap runs as a parameter to the yaml file

  

  Soroush Sorourian

  committed
  08deb3c

  Copy full SHA for 08deb3c

• Implemented multiple confidence intervals (90, 95 & 99%) for rf Bagging calculation. Convert rf Bagging ci as a separate function

  

  Soroush Sorourian

  committed
  6476710

  Copy full SHA for 6476710

• Implemented multiple confidence intervals (90, 95 & 99%) for mlp Bagging calculation. Convert mlp Bagging ci as a separate function

  

  Soroush Sorourian

  committed
  e94b32d

  Copy full SHA for e94b32d

• Update rf_Bagging_ci function to dynamically read the number of bootstrapping runs from the yaml file

  

  Soroush Sorourian

  committed
  746b172

  Copy full SHA for 746b172

• Update mlp_Bagging_ci function to dynamically read the number of bootstrapping runs from the yaml file

  

  Soroush Sorourian

  committed
  6fdb8d6

  Copy full SHA for 6fdb8d6

• develop a separate function for MAPIE and update fs_algo_train_eval accordingly

  

  Soroush Sorourian

  committed
  52f088f

  Copy full SHA for 52f088f

• Rename ci for rf model for clarification

  

  Soroush Sorourian

  committed
  048ca30

  Copy full SHA for 048ca30

Commits on Feb 7, 2025

• Update fs_pred to consider forestci only for rf model. Update saving prediction algorithms pipeline to remove ci

  

  Soroush Sorourian

  committed
  5c9e14b

  Copy full SHA for 5c9e14b

• added pkg/proc.attr.hydfab/.RData to the gitignore

  

  Soroush Sorourian

  committed
  275564f

  Copy full SHA for 275564f

• add confidence level to the yaml file

  

  Soroush Sorourian

  committed
  75a5140

  Copy full SHA for 75a5140

• update Bagging_ci files to calculate the confidence interval from user-specified confidence level

  

  Soroush Sorourian

  committed
  3bc23ea

  Copy full SHA for 3bc23ea

• delete .RData

  

  Soroush Sorourian

  committed
  ba8858d

  Copy full SHA for ba8858d

Commits on Feb 10, 2025

• add MAPIE alpha to the yaml file

  

  Soroush Sorourian

  committed
  3e65cd6

  Copy full SHA for 3e65cd6

• boolean variable for MAPIE

  

  Soroush Sorourian

  committed
  cfc0c24

  Copy full SHA for cfc0c24

• implement MAPIE dynamically into fs_proc and fs_train_eval from information from the config file

  

  Soroush Sorourian

  committed
  39174fc

  Copy full SHA for 39174fc

• clean up fs_algo_train_eval from regarding MAPIE

  

  Soroush Sorourian

  committed
  91cb971

  Copy full SHA for 91cb971

Commits on Feb 11, 2025

• add forestci boolean to yaml file

  

  Soroush Sorourian

  committed
  afaa087

  Copy full SHA for afaa087

• change forestci as an independent function outside of train_algos. Add Uncertainty object to the algs_dict to store the uncertainties of the models

  

  Soroush Sorourian

  committed
  aecf707

  Copy full SHA for aecf707

• added flag for Bagging enable/disable in yaml file

  

  Soroush Sorourian

  committed
  bf34bbf

  Copy full SHA for bf34bbf

• fixed a bug in calculate_Bagging_ci

  

  Soroush Sorourian

  committed
  7077843

  Copy full SHA for 7077843

Commits on Feb 12, 2025

• merged Bagging_ci functions into one and bring it out of train_algos() to the wrapper

  

  Soroush Sorourian

  committed
  7e4ee03

  Copy full SHA for 7e4ee03

• clean up fs_algo_train_eval from individual Bagging_ci(), and forestci and Bagging in train_algos()

  

  Soroush Sorourian

  committed
  6aaa3ee

  Copy full SHA for 6aaa3ee

• replace model_name with algo_str in Bagging_ci

  

  Soroush Sorourian

  committed
  4de7e19

  Copy full SHA for 4de7e19

• replace model_cfg with algo_cfg for consistency

  

  Soroush Sorourian

  committed
  15bbc04

  Copy full SHA for 15bbc04

• replace n_models with n_algos

  

  Soroush Sorourian

  committed
  5046c84

  Copy full SHA for 5046c84

• relace model with algo

  

  Soroush Sorourian

  committed
  1d8bba6

  Copy full SHA for 1d8bba6

• delete Bagging_ci flag from yaml and build the logic around n_algos for Bagging uncertainty calculation

  

  Soroush Sorourian

  committed
  adf4746

  Copy full SHA for adf4746

• delete MAPIE flag from yaml and build the logic around MAPIE_alpha for MAPIE calculation

  

  Soroush Sorourian

  committed
  3df0553

  Copy full SHA for 3df0553

• create a separate section in yaml for Bagging

  

  Soroush Sorourian

  committed
  7e5ce60

  Copy full SHA for 7e5ce60

Commits on Feb 13, 2025

• update fs_proc and fs_algo_train_eval to treat uncertainty as a separate section

  

  Soroush Sorourian

  committed
  9edb092

  Copy full SHA for 9edb092

• delete unused mapie boolean arg

  

  Soroush Sorourian

  committed
  7b616a3

  Copy full SHA for 7b616a3

• added some information about OPTIONAL/REQUIRED to yaml file

  

  Soroush Sorourian

  committed
  ddab11a

  Copy full SHA for ddab11a

• replace Bagging with bagging (lower case)

  

  Soroush Sorourian

  committed
  9ab4ccf

  Copy full SHA for 9ab4ccf

Commits on Feb 14, 2025

• fixed a bug in the range of mapie_alpha

  

  Soroush Sorourian

  committed
  8404b69

  Copy full SHA for 8404b69

• temporarily change the the COMID retrieval till update for pynhd package

  

  Soroush Sorourian

  committed
  52e73ce

  Copy full SHA for 52e73ce

• temporarily change the the COMID retrieval till update for pynhd package - 2

  

  Soroush Sorourian

  committed
  373986c

  Copy full SHA for 373986c

• Apply the changes in fs_prog_algo.py into fs_proc_algo_viz.py

  

  Soroush Sorourian

  committed
  685023e

  Copy full SHA for 685023e

• implement random sampling and recycle hyper-parameters into uncertainty calculation

  

  Soroush Sorourian

  committed
  99d8784

  Copy full SHA for 99d8784

• revert back the COMID retrieval section as pynhd is updated to >=0.19

  

  Soroush Sorourian

  committed
  9825210

  Copy full SHA for 9825210

• add additional comment for MAPIE_alpha

  

  Soroush Sorourian

  committed
  8129667

  Copy full SHA for 8129667

Commits on Feb 18, 2025

• resolve merge conflicts and apply fixes by Guy

  

  Soroush Sorourian

  committed
  9263a49

  Copy full SHA for 9263a49

• simplify the calculate_bagging_ci function

  

  Soroush Sorourian

  committed
  2390b90

  Copy full SHA for 2390b90

• update calculate_mapie() to write mapie to self.algs_dict

  

  Soroush Sorourian

  committed
  4ac5ce7

  Copy full SHA for 4ac5ce7

• clean up calculate_bagging_ci()

  

  Soroush Sorourian

  committed
  ef57c39

  Copy full SHA for ef57c39

• update train_algos_grid_search to calculate forestci with grid search

  

  Soroush Sorourian

  committed
  b3cbc62

  Copy full SHA for b3cbc62

Commits on Feb 19, 2025

• Merge remote-tracking branch 'upstream/dev' into ss_test_mapie

  

  Soroush Sorourian

  committed
  3336544

  Copy full SHA for 3336544

• add save_Xtrain_to_csv function to the model and update the code and fs_proc

  

  Soroush Sorourian

  committed
  e263848

  Copy full SHA for e263848

Commits on Feb 20, 2025

• update the save_algos() to output the X_train shape to the joblib file

  

  Soroush Sorourian

  committed
  36d6a62

  Copy full SHA for 36d6a62

• Write forestci parameters in terms of numpy arrays instead of dataframe

  

  Soroush Sorourian

  committed
  2563199

  Copy full SHA for 2563199

• updated fs_pred to read the X_train dimension from joblib file rather than the csv file

  

  Soroush Sorourian

  committed
  d3c6624

  Copy full SHA for d3c6624

• clean up unnecessary Xtrain related functions in fs_algo_train_eval after joblib update

  

  Soroush Sorourian

  committed
  1b3def2

  Copy full SHA for 1b3def2

• clean up save_algos function

  

  Soroush Sorourian

  committed
  3142576

  Copy full SHA for 3142576

• fix the forestci with grid search while keeping the forestci calculation under train_eval()

  

  Soroush Sorourian

  committed
  5913aed

  Copy full SHA for 5913aed

Commits on Feb 21, 2025

• implement calculate_bagging_ci to work with grid search

  

  Soroush Sorourian

  committed
  982ec5b

  Copy full SHA for 982ec5b

• update the keys and indices of tuples inside Bagging ci

  

  Soroush Sorourian

  committed
  6fee686

  Copy full SHA for 6fee686

• update the output format for mapie

  

  Soroush Sorourian

  committed
  80620a0

  Copy full SHA for 80620a0

• bring confidence_levels out of Bagging_uncertainty in the config file and update fs_algo_train_eval accordingly

  

  Soroush Sorourian

  committed
  6b3c5b8

  Copy full SHA for 6b3c5b8

• implement forestci with user-defined confidence intervals

  

  Soroush Sorourian

  committed
  c6f4554

  Copy full SHA for c6f4554

Commits on Feb 24, 2025

• update the code to save uncertainty dictionary

  

  Soroush Sorourian

  committed
  b17010b

  Copy full SHA for b17010b

Commits on Feb 25, 2025

• Generate n_algos unique random states using self.rs as the seed, ensuring reproducibility.

  

  Soroush Sorourian

  committed
  5b35735

  Copy full SHA for 5b35735