Sam Foreman 2024-12-25
- Computational scientist @ Argonne National Laboratory (ALCF)
- Working on:
- π§ͺ {AI, HPC} for science
- π training large models on supercomputers
[!TIP]
[!TIP]
style="height:1.25rem;vertical-align:text-bottom;" /> Now Playing
[!TIP]
<script> /** Developed by Prashant Shrestha + https://prashant.me */ var lastfmData = { baseURL: "https://ws.audioscrobbler.com/2.0/?method=user.getrecenttracks&user=", // Your Last.fm Username user: "saforem2", // Your API key api_key: "1dbc15037c1fe71ce06acbb3f73adc75", additional: "&format=json&limit=1" }; var getSetLastFM = function() { $.ajax({ type: "GET", url: lastfmData.baseURL + lastfmData.user + "&api_key=" + lastfmData.api_key + lastfmData.additional, dataType: "json", success: function(resp) { var recentTrack = resp.recenttracks.track[0]; var formatted = // "" + recentTrack.name; "πΆ " + recentTrack.name; $("a#tracktitle") .html(formatted) .attr("href", recentTrack.url) .attr("title", recentTrack.name + " by " + recentTrack.artist["#text"]) .attr("target", "_blank"); var artistFormatted = // "" + recentTrack.artist["#text"]; "π£οΈ " + recentTrack.artist["#text"]; $("a#trackartist") .html(artistFormatted) .attr("title", "Artist : " + recentTrack.artist["#text"]); $("img#trackart").attr("src", recentTrack.image[2]["#text"]); }, error: function(resp) { $("a#tracktitle").html( "" + "Silence!" ); $("img#trackart").attr("src", "π§π»βπ»"); var artistFormatted = "Sam Foreman"; $("a#trackartist") .html(artistFormatted) .attr("href", "https://samforeman.me"); } }); }; // Get the new one. getSetLastFM(); // Start the countdown. setInterval(getSetLastFM, 10 * 5000); </script>
[!TIP]
π If youβre curious
π How I got here
My current research focuses on using deep generative modeling to help build better sampling algorithms in lattice gauge theory. In particular, Iβm interested in building gauge equivariant neural network architectures and using inductive priors to incorporate physical symmetries into machine learning models.
I received my PhD in Physics from the University of Iowa in 2019 and my thesis was on Learning Better Physics: A Machine Learning Approach to Lattice Gauge Theory.
Prior to this, I completed two bachelors degrees (Engineering Physics and Applied Mathematics, 2015) at The University of Illinois at Urbana-Champaign. My undergraduate dissertation was titled Energy Storage in Quantum Resonators and was supervised by Professor Alfred HΓΌbler within the Center for Complex Systems Research at UIUC.
This work ultimately resulted in a patent !!
yellow pink green blue circle
import datetime from rich import print now = datetime.datetime.now() day = now.strftime("%Y-%m-%d") time = now.strftime("%H:%M:%S") print(' '.join([ "[#838383]Last Updated[/]:", f"[#E599F7]{day}[/]", "[#838383]@[/]", f"[#00CCFF]{time}[/]" ]))Last Updated: 2024-12-25 @ 01:59:51Β© Copyright Sam Foreman
[!NOTE]
You can find a full list of my publications on my Google Scholar
-
Intro to HPC Bootcamp: Engaging New Communities Through Energy Justice Projects
Journal of Computational Science, 2024 -
Thorough Characterization and Analysis of Large Transformer Model Training At-Scale
Proc. ACM Meas. Anal. Comput. Syst. 03/2024 -
MLMC: Machine Learning Monte Carlo for Lattice Gauge Theory
S. Foreman et al.Β Lattice, 2023 (Proceedings), 12/2023 -
Protein Generation via Genome-scale Language Models with Bio-physical Scoring
@ SCβ23, 11/2023 -
DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery [β¦]
@ NeurIPS 2023 AI For Science Workshop, 10/2023 -
Comprehensive Performance Study of LLMs on Novel AI Accelerators
M. Emani, S. Foreman, et al., IPDPS 2024, 10/2023 -
Exploratory Analysis of Climate Data with
ClimRR
S. Foreman, Intro to HPC Bootcamp @ NERSC, 08/2023 -
π GenSLMs: Genome-scale language models reveal SARS-Cov-2 evolutionary dynamics
@ SCβ22 10/2022 -
Lattice QCD and Particle Physics
A.S. Kronfeld et al., 07/2022 -
Applications of ML to Lattice QFT
D. Boyda, S. CalΓ, S. Foreman, et al., [arXiv:2202.05838], 02/2022 -
LeapFrogLayers: Trainable Framework for Effective Sampling
S. Foreman, X.Y. Jin, J.C. Osborn, Lattice, 2021 -
HMC with Normalizing Flows [slides]
S. Foreman et al., Lattice, 2021 -
Deep Learning Hamiltonian Monte Carlo [+ poster]
S. Foreman, X.Y. Jin, & J.C. Osborn, @ SimDL Workshop @ ICLR, 2021 -
Machine Learning and Neural Networks for Field Theory
S. Foreman, X.Y. Jin, & J.C. Osborn, SnowMass, 2020 -
Examples of renormalization group transformations for image sets
S. Foreman et al., Physical Review E., 2018 -
RG inspired Machine Learning for lattice field theory
S. Foreman et al., arXiv:1710.02079, 2017 -
Large Energy Density in Three-Plate Nanocapacitors due to Coulomb Blockade
S. Foreman et al., J. Appl. Phys, 2018
[!TIP]
- References:
- (Sam Foreman et al. 2018)
- (Hubler et al. 2018)
- (Samuel Foreman et al. 2018a)
- (Samuel Foreman et al. 2018b)
- (S. A. Foreman 2019)
- (Sam Foreman, Jin, and Osborn 2020)
- (Sam Foreman, Jin, and Osborn 2021b)
- (Sam Foreman et al. 2021)
- (Sam Foreman, Jin, and Osborn 2021a)
- (Liu et al. 2017)
- (Boyda et al. 2022)
- (Kronfeld et al. 2022)
- (Zvyagin et al. 2023)
- (Sam Foreman, Jin, and Osborn)
- (Deamont and Foreman 2014)
- (Emani et al. 2023)
- (Song et al. 2023)
- (Dharuman et al. 2023)
- (Shanahan, Terao, and Whiteson 2022)
- (Cheng et al. 2024)
- (Parete-Koon et al. 2024)
Boyda, Denis, Salvatore CalΔ±Μ, Sam Foreman, Lena Funcke, Daniel C Hackett, Yin Lin, Gert Aarts, et al. 2022. βApplications of Machine Learning to Lattice Quantum Field Theory.β arXiv Preprint arXiv:2202.05838.
Cheng, Scott, Jun-Liang Lin, Murali Emani, Siddhisanket Raskar, Sam Foreman, Zhen Xie, Venkatram Vishwanath, and Mahmut Taylan Kandemir. 2024. βThorough Characterization and Analysis of Large Transformer Model Training at-Scale.β Proceedings of the ACM on Measurement and Analysis of Computing Systems 8 (1): 1β25.
Deamont, George, and Sam Foreman. 2014. βSuperconductivity of in and Sn Samples.β
Dharuman, Gautham, Logan Ward, Heng Ma, Priyanka V Setty, Ozan Gokdemir, Sam Foreman, Murali Emani, et al. 2023. βProtein Generation via Genome-Scale Language Models with Bio-Physical Scoring.β In Proceedings of the SCβ23 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis, 95β101.
Emani, Murali, Sam Foreman, Varuni Sastry, Zhen Xie, Siddhisanket Raskar, William Arnold, Rajeev Thakur, Venkatram Vishwanath, and Michael E Papka. 2023. βA Comprehensive Performance Study of Large Language Models on Novel AI Accelerators.β arXiv Preprint arXiv:2310.04607.
Foreman, Sam, Joel Giedt, Yannick Meurice, and Judah Unmuth-Yockey. 2018. βRG-Inspired Machine Learning for Lattice Field Theory.β In EPJ Web of Conferences, 175:11025. EDP Sciences.
Foreman, Sam, Taku Izubuchi, Luchang Jin, Xiao-Yong Jin, James C Osborn, and Akio Tomiya. 2021. βHMC with Normalizing Flows.β arXiv Preprint arXiv:2112.01586.
Foreman, Sam, Xiao-Yong Jin, and James Osborn. βMLMC: Machine Learning Monte Carlo for Lattice Gauge Theory.β In 40th International Symposium on Lattice Field Theory (Lattice 2023) (Batavia, IL, United States, 07/31/2023 - 08/04/2023).
Foreman, Sam, Xiao-Yong Jin, and James C Osborn. 2020. βMachine Learning and Neural Networks for Field Theory.β
βββ. 2021a. βLeapfrogLayers: A Trainable Framework for Effective Topological Sampling.β arXiv Preprint arXiv:2112.01582.
Foreman, Sam, Xiao-Yong Jin, and James C. Osborn. 2021b. βDeep Learning Hamiltonian Monte Carlo.β https://arxiv.org/abs/2105.03418.
Foreman, Samuel Alfred. 2019. βLearning Better Physics: A Machine Learning Approach to Lattice Gauge Theory.β PhD thesis, University of Iowa.
Foreman, Samuel, Joel Giedt, Yannick Meurice, and Judah Unmuth-Yockey. 2018a. βExamples of Renormalization Group Transformations for Image Sets.β Physical Review E 98 (5): 052129.
βββ. 2018b. βMachine Learning Inspired Analysis of the Ising Model Transition.β In Lattice 2018.
Hubler, A, S Foreman, J Liu, and L Wortsmann. 2018. βLarge Energy Density in Three-Plate Nanocapacitors Due to Coulomb Blockade.β Journal of Applied Physics 123 (10).
Kronfeld, Andreas S, Tanmoy Bhattacharya, Thomas Blum, Norman H Christ, Carleton DeTar, William Detmold, Robert Edwards, et al. 2022. βLattice QCD and Particle Physics.β arXiv Preprint arXiv:2207.07641.
Liu, Jiaqi, Alfred W Hubler, Samuel Alfred Foreman, and Katharina Ott. 2017. βEnergy Storage in Quantum Resonators.β
Parete-Koon, Suzanne, Michael Sandoval, Kellen Leland, Subil Abraham, Mary Ann Leung, Rebecca Hartman-Baker, Paige Kinsley, et al. 2024. βIntro to HPC Bootcamp: Engaging New Communities Through Energy Justice Projects.β Journal of Computational Science Education 15 (1).
Shanahan, Phiala, Kazuhiro Terao, and Daniel Whiteson. 2022. βSnowmass 2021 Computational Frontier CompF03 Topical Group Report: Machine Learning.β arXiv Preprint arXiv:2209.07559.
Song, Shuaiwen Leon, Bonnie Kruft, Minjia Zhang, Conglong Li, Shiyang Chen, Chengming Zhang, Masahiro Tanaka, et al. 2023. βDeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery Through Sophisticated AI System Technologies.β arXiv Preprint arXiv:2310.04610.
Zvyagin, Maxim, Alexander Brace, Kyle Hippe, Yuntian Deng, Bin Zhang, Cindy Orozco Bohorquez, Austin Clyde, et al. 2023. βGenSLMs: Genome-Scale Language Models Reveal SARS-CoV-2 Evolutionary Dynamics.β The International Journal of High Performance Computing Applications 37 (6): 683β705.
[!NOTE]
See βem all, live: Talks
[!TIP]
Convert from HTML to slideshow version of a page by appending
/slides
to the end of its URL, e.g.
[!TIP]
<iframe class="slide-deck reveal-full-page" loading="lazy" allow="picture-in-picture" src="/talks/ai-for-science-2024/slides.html" title="Parallel Training Methods" align="center" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="aspect-ratio:1.3671875;"> </iframe>
[!TIP]
AuroraGPT @ 2024 ALCF Hands-On HPC Workshop [10/2024]
<iframe class="slide-deck" loading="lazy" allow="picture-in-picture" src="/talks/AuroraGPT/alcf-hpc-workshop-2024/slides.html" title="Machine Learning and Foundation Models at Scale" align="center" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="aspect-ratio:1.3671875;"> </iframe>
[!TIP]
<iframe class="slide-deck" loading="lazy" allow="picture-in-picture" src="https://samforeman.me/talks/alcf-hpc-workshop-2024/slides#/section" title="Machine Learning and Foundation Models at Scale" align="center" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="aspect-ratio:1.3671875;"> </iframe>
[!TIP]
<iframe class="slide-deck reveal-full-page" loading="lazy" allow="picture-in-picture" src="/talks/hpc-user-forum/slides.html" title="AuroraGPT" align="center" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="aspect-ratio:1.3671875;"> </iframe>AuroraGPT @ HPC User Forum, 2024 [09/2024]
[!TIP]
<iframe class="slide-deck" loading="lazy" allow="picture-in-picture" src="/talks/llms-at-scale/slides.html" title="Training LLMs at Scale" align="center" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="aspect-ratio:1.3671875;"> </iframe>Training LLMs at Scale @ ATPESC, 2024 [08/2024]
[!TIP]
<iframe class="slide-deck" loading="lazy" allow="picture-in-picture" src="/talks/llms-on-polaris/slides.html" title="LLMs on Polaris" align="center" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="aspect-ratio:1.3671875;"> </iframe>
[!TIP]
<iframe class="slide-deck" loading="lazy" allow="picture-in-picture" src="https://saforem2.github.io/parallel-training-slides" title="Parallel Training Techniques" align="center" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="aspect-ratio:1.3671875;"> </iframe>
[!TIP]
LLMs from Scratch @ LLM Tutorial Workshop [02/2024]
<iframe class="slide-deck" loading="lazy" allow="picture-in-picture" src="https://saforem2.github.io/llm-workshop-talk" title="LLMs from Scratch" align="center" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="aspect-ratio:1.3671875;"> </iframe>
[!TIP]
Creating Small(-ish) LLMs @ LLM Tutorial Workshop (1) [11/2023]
<iframe class="slide-deck" loading="lazy" allow="picture-in-picture" src="https://saforem2.github.io/LLM-tutorial" title="Creating Small(-ish) LLMs" align="center" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen> </iframe>
[!TIP]
<iframe class="slide-deck" loading="lazy" allow="picture-in-picture" src="https://saforem2.github.io/oneapi-talk" title="Exascale Science on Aurora" align="center" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen> </iframe>
[!TIP]
LLM Lunch Talk @ ALCF Hands On HPC Workshop [10/2023]
<iframe class="slide-deck" loading="lazy" allow="picture-in-picture" src="https://saforem2.github.io/llm-lunch-talk/#/section" title="LLMs on Polaris" align="center" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="aspect-ratio:1.3671875;"> </iframe>
[!TIP]
<iframe class="slide-deck" loading="lazy" allow="picture-in-picture" src="https://saforem2.github.io/scaling4science/#/section" title="Scaling LLMs for Science and Ongoing Collaborations" align="center" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="aspect-ratio:1.3671875;"> </iframe>
[!TIP]
MLMC: Machine Learning Monte Carlo @ Lattice 2023 [07/2023]
<iframe class="slide-deck" loading="lazy" allow="picture-in-picture" src="https://saforem2.github.io/lattice23/#/title-slide" title="MLMC: Machine Learning Monte Carlo" align="center" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="aspect-ratio:1.3671875;"> </iframe>
[!TIP]
Generative Modeling and Efficient Sampling @ PASC23 [07/2023]
<iframe class="slide-deck" loading="lazy" allow="picture-in-picture" src="https://saforem2.github.io/lqcd-pasc23/" title="Generative Modeling and Efficient Sampling" align="center" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="aspect-ratio:1.3671875;"> </iframe>
[!TIP]
Efficient Sampling for LGT @ Deep Fridays @ U. Bologna [04/2023]
<iframe class="slide-deck" loading="lazy" allow="picture-in-picture" src="https://saforem2.github.io/deep-fridays/" title="Efficient Sampling for LGT" align="center" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="aspect-ratio:1.3671875;"> </iframe>
[!TIP]
<iframe class="slide-deck" loading="lazy" allow="picture-in-picture" src="https://saforem2.github.io/ai4sci-large-scale-training/#" title="Large Scale Training" align="center" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="aspect-ratio:1.3671875;"> </iframe>
[!TIP]
Hyperparameter Management @ ALCF SDL Workshop [10/2022]
<iframe class="slide-deck" loading="lazy" allow="picture-in-picture" src="https://saforem2.github.io/hparam-management-sdl2022" title="Hyperparameter Management" align="center" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="aspect-ratio:1.3671875;"> </iframe>
[!TIP]
Statistical Learning @ ATPESC 2022 [08/2022]
<iframe class="slide-deck" loading="lazy" allow="picture-in-picture" src="https://saforem2.github.io/ATPESC-StatisticalLearning/#/" title="Statistical Learning" align="center" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="aspect-ratio:1.3671875;"> </iframe>
[!TIP]
Scientific Data Science: An Emerging Symbiosis @ ANL (05/2022)
<iframe class="slide-deck" loading="lazy" allow="picture-in-picture" src="https://saforem2.github.io/anl-job-talk" title="Scientific Data Science" align="center" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="aspect-ratio:1.3671875;"> </iframe>
[!TIP]
Machine Learning in HEP @ UNC Greensboro [03/2022]
- Machine Learning in HEP, at UNC Greensboro, March 2022
<iframe class="slide-deck" loading="lazy" allow="picture-in-picture" src="https://saforem2.github.io/physicsSeminar" title="Machine Learning in HEP" align="center" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="width:100%!important; border:none;border-radius:0.25rem;" style="aspect-ratio:1.3671875;"> </iframe>
[!TIP]
Accelerated Sampling Methods for LGT, @ DWQ @ 25 [BNL] [12/2021]
<iframe class="slide-deck" loading="lazy" allow="picture-in-picture" src="https://saforem2.github.io/l2hmc-dwq25/" title="Accelerated Sampling Methods for LGT" align="center" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="aspect-ratio:1.3671875;"> </iframe>
[!TIP]
<iframe class="slide-deck" loading="lazy" allow="picture-in-picture" src="https://saforem2.github.io/l2hmc_talk_ect2021" title="Training Topological Samplers for LGT" align="center" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="aspect-ratio:1.3671875;"> </iframe>
[!TIP]
l2hmc-qcd @ MIT Lattice Group Seminar [2021]
l2hmc-qcd at the MIT Lattice Group Seminar, 2021
[!TIP]
<iframe class="slide-deck" loading="lazy" allow="picture-in-picture" src="https://slides.com/samforeman/dlhmc/embed" title="Deep Learning HMC for Improved Gauge Generation" scrolling="no" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="aspect-ratio:1.3671875;"> </iframe>
[!TIP]
Machine Learning for Lattice QCD @ U. Iowa [2020]
<iframe class="slide-deck" loading="lazy" allow="picture-in-picture" src="https://slides.com/samforeman/l2hmc-qcd/embed" title="Machine Learning for Lattice QCD" align="center" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="aspect-ratio:1.3671875;"> </iframe>
[!TIP]
[!TIP]
π
saforem2/
-
Organizer for:
-
SC24 Workshop: High Performance Python for Science at Scale (HPPSS), November 2024
-
SC23 Workshop: High Performance Python for Science at Scale (HPPSS), November 2023
-
Machine Learning and Quantum Computing for Earth Sciences at 17th U. S. National Congress on Computational Mechanics, July 2023
-
TableΒ 1: π Experience
Position | @ | Start | End |
---|---|---|---|
Assistant Computational Scientist | ALCF | 2022 | β |
Postdoc | ALCF | 2019 | 2022 |
Graduate Researcher | ANL | 2018 | 2019 |
π Experience
TableΒ 2: π Education
Degree | In | @ | End |
---|---|---|---|
PhD | Physics | University of Iowa | 2019 |
B.Sc | Physics | UIUC | 2015 |
B.Sc | Math | UIUC | 2015 |
π Education
[!TIP]
<script> /** Developed by Prashant Shrestha + https://prashant.me */ var lastfmData = { baseURL: "https://ws.audioscrobbler.com/2.0/?method=user.getrecenttracks&user=", // Your Last.fm Username user: "saforem2", // Your API key api_key: "1dbc15037c1fe71ce06acbb3f73adc75", additional: "&format=json&limit=1" }; var getSetLastFM = function() { $.ajax({ type: "GET", url: lastfmData.baseURL + lastfmData.user + "&api_key=" + lastfmData.api_key + lastfmData.additional, dataType: "json", success: function(resp) { var recentTrack = resp.recenttracks.track[0]; var formatted = // "" + recentTrack.name; "πΆ " + recentTrack.name; $("a#tracktitle") .html(formatted) .attr("href", recentTrack.url) .attr("title", recentTrack.name + " by " + recentTrack.artist["#text"]) .attr("target", "_blank"); var artistFormatted = // "" + recentTrack.artist["#text"]; "π£οΈ " + recentTrack.artist["#text"]; $("a#trackartist") .html(artistFormatted) .attr("title", "Artist : " + recentTrack.artist["#text"]); $("img#trackart").attr("src", recentTrack.image[2]["#text"]); }, error: function(resp) { $("a#tracktitle").html( "" + "Silence!" ); $("img#trackart").attr("src", "π§π»βπ»"); var artistFormatted = "Sam Foreman"; $("a#trackartist") .html(artistFormatted) .attr("href", "https://samforeman.me"); } }); }; // Get the new one. getSetLastFM(); // Start the countdown. setInterval(getSetLastFM, 10 * 5000); </script>