Add meta_init, enable it as default init process#84

Merged

lessw2020 merged 12 commits intopytorch:mainfrom lessw2020:curr_meta_init

Mar 5, 2024

+78-13

Commits on Mar 4, 2024

Merge branch 'main' into curr_meta_init
lessw2020
authored

Commits on Mar 5, 2024

remove llama7B as now llama_7b already added
lessw2020
committed
remove call to reset_params in model init, with comments for how it interplays with meta_init and when it should be re-activated.
lessw2020
committed
remove init call to reset_params in rmsnorm init to be consistent
lessw2020
committed