Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Model documentation #436

Merged

Conversation

amandarichardsonn
Copy link
Contributor

This PR requests the merge of the Model documentation section into the full craylab SmartSim doc refactor branch.

Copy link

codecov bot commented Dec 11, 2023

Codecov Report

Attention: 20 lines in your changes are missing coverage. Please review.

Comparison is base (ea44a1e) 84.33% compared to head (ef25116) 90.47%.
Report is 147 commits behind head on smartsim-docs-refresh.

Additional details and impacted files

Impacted file tree graph

@@                    Coverage Diff                    @@
##           smartsim-docs-refresh     #436      +/-   ##
=========================================================
+ Coverage                  84.33%   90.47%   +6.14%     
=========================================================
  Files                         58       60       +2     
  Lines                       3223     3738     +515     
=========================================================
+ Hits                        2718     3382     +664     
+ Misses                       505      356     -149     
Files Coverage Δ
smartsim/__init__.py 100.00% <ø> (ø)
smartsim/_core/__init__.py 100.00% <ø> (ø)
smartsim/_core/config/__init__.py 100.00% <ø> (ø)
smartsim/_core/control/__init__.py 100.00% <ø> (ø)
smartsim/_core/generation/__init__.py 100.00% <ø> (ø)
smartsim/_core/generation/generator.py 96.00% <100.00%> (+2.38%) ⬆️
smartsim/_core/generation/modelwriter.py 100.00% <100.00%> (+15.06%) ⬆️
smartsim/_core/launcher/__init__.py 100.00% <100.00%> (ø)
smartsim/_core/launcher/launcher.py 100.00% <100.00%> (ø)
smartsim/_core/launcher/local/local.py 95.45% <100.00%> (-0.11%) ⬇️
... and 50 more

doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
Copy link
Contributor

@mellis13 mellis13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple of initial comments to get revisions started.

doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
@al-rigazzi
Copy link
Collaborator

@amandarichardsonn I'd suggest to bring the feature branch up to master, to limit the number of modified files shown in this (and parallel) PRs.

doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated

----------------
A Standard Model
----------------
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

total nitpick: could we reduce the total number of lines here by letting lines extend to 80 chars more often?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is good practice for sure, Ill go through the doc once Ive finished your review comments

doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
The sphinx-tabs documentation extension uses a white background 
for the tabs component. This causes readability issues with the theme 
that we have chosen. A custom CSS has been added to override
those components to inherit the overall theme color.

[ committed by @mellis13 ]
[ reviewed by @al-rigazzi ]
Copy link
Contributor

@mellis13 mellis13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly minor comments. The refactor of the ML model and TorchScript sections are much more readable -- thanks!

doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated
``Experiment`` workflow, such as launching compiled applications,
running scripts, or performing general computational operations. ``Models`` can be launched with
other SmartSim entities and :ref:`SmartRedis<dead_link>` to build AI-enabled workflows.
``Model`` objects can leverage ML capabilities by utilizing the SmartSim client (:ref:`SmartRedis<dead_link>`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Model objects can leverage ML capabilities by utilizing the SmartSim client (:ref:SmartRedis<dead_link>) to transfer data to a standard Orchestrator to enable other running Models to access the data. Additionally, clients can execute ML models (TF, TF-lite, PyTorch, or ONNX) and TorchScripts stored in the Orchestrator.

->

With the SmartSim client (:ref:SmartRedis<dead_link>), data can be transferred out of the Model into the standalone Orchestrator for use in an ML model (TF, TF-lite, PyTorch, or ONNX), online training process, or additional Model applications.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you mention ML model, online training process and additional Model apps - what do you mean by online training process? Im wondering if this is repeating ML model

doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
amandarichardsonn and others added 8 commits January 16, 2024 12:26
Adds infrastructure to fetch RedisAI's dependencies. This removes the
need to call RedisAI's `get_deps.sh` script so that we can fetch newer
versions of our machine learning backends than the ones officially
supported by RedisAI.

Additionally, this upgrades the machine learning python packages
required by SmartSim so that they stay up to date with the backends.
This in turn allows us to add Python3.10+ONNX support.

[ committed by @MattToast ]
[ reviewed by @ashao ]
Copy link
Contributor

@mellis13 mellis13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates. The examples of key collision prevention I think will really help users make sense of what is a complicated process. Mostly minor comments, but one large theme of using client.set_data_source when performing an operation from within the application that set the data. This applies to all of the sections. I didn't add the same comment to the copy/rename/delete section, but applies there too. After those edits I will do another pass through but we should be close to merging this.

doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
Copy link
Contributor

@mellis13 mellis13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly minor comments and questions. One general critique for the prefixing section, often the term "script" is used when a more general "application" might be better. I think it's technically correct because all of our examples are scripts, but we don't want users to get in the mindset that all Model are scripts. Otherwise, these are small changes and we are close to merging!

doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
Copy link
Contributor

@mellis13 mellis13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think just a couple of strange wordings that need to change resulting from the find/replace operation.

doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
doc/model.rst Outdated Show resolved Hide resolved
Copy link
Contributor

@mellis13 mellis13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@amandarichardsonn amandarichardsonn merged commit efcbd74 into CrayLabs:smartsim-docs-refresh Jan 25, 2024
26 checks passed
@amandarichardsonn amandarichardsonn deleted the model-docs branch January 25, 2024 17:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants