Include examples in readthedocs #2382

quaquel · 2024-10-17T14:09:20Z

This updates conf.py to create markdown files for including the examples in the documentation. It also cleans up some warning on RTD, which happened while debugging the code. In this process, this also fixes #2360.

It currently only covers the basic examples because the advanced examples are not fully cleaned yet.

tpike3

This is awesome!

This also epitomizes to me why we want those seminal models in the main repo so we can ensure they are current with whatever Mesa version we are on.

tpike3 · 2024-10-19T17:57:44Z

docs/examples/conways_game_of_life.md

+To run the model interactively, run ``mesa runserver`` in this directory. e.g.
+
+```
+    $ mesa runserver


This needs to be updated to the new visualization approach

tpike3 · 2024-10-19T17:58:33Z

docs/examples/conways_game_of_life.md

+* ``agents.py``: Defines the behavior of an individual cell, which can be in two states: DEAD or ALIVE.
+* ``model.py``: Defines the model itself, initialized with a random configuration of alive and dead cells.
+* ``portrayal.py``: Describes for the front end how to render a cell.
+* ``st_app.py``: Defines an interactive visualization using Streamlit.


Needs to be updated for new set agents and app

tpike3 · 2024-10-19T17:59:32Z

docs/examples/virus_on_network.md

+To run the model interactively, run ``mesa runserver`` in this directory. e.g.
+
+```
+    $ mesa runserver


Needs to be updated with Solara

docs/index.md

tpike3 · 2024-10-19T18:04:26Z

mesa/examples/basic/conways_game_of_life/app.py

+import altair as alt
+import numpy as np
+import pandas as pd
+import streamlit as st


Do we still want to use streamlit or update to Solara?

in my view, we standardize all examples on solara

tpike3 · 2024-10-19T18:05:07Z

mesa/examples/basic/virus_on_network/Readme.md

+To run the model interactively, run ``mesa runserver`` in this directory. e.g.
+
+```
+    $ mesa runserver


Update to Solara?

tpike3 · 2024-10-19T18:06:10Z

docs/examples/virus_on_network.md

+
+For more information about this model, read the NetLogo's web page: http://ccl.northwestern.edu/netlogo/models/VirusonaNetwork.
+
+JavaScript library used in this example to render the network: [d3.js](https://d3js.org/).


This could be removed

tpike3 · 2024-10-19T18:09:48Z

mesa/examples/basic/virus_on_network/Readme.md

@@ -0,0 +1,61 @@
+# Virus on a Network


Why are these in two places?

not sure what you mean?

The old one wasn’t removed, so there are now two Readme files.

quaquel · 2024-10-19T18:21:04Z

@tpike3, I agree with most of your comments regarding the content of the examples, but can we leave that for a separate PR and focus this one on just including the examples in the docs in the first place? The updated conf.py ensures that any updates in the readme, agent.py, model.py, and app.py result in an updated page in the docs.

EwoutH · 2024-10-20T17:57:51Z

One question, why does it add so much additional files? Aren't all/most of those files in the repo already?

quaquel · 2024-10-20T18:02:56Z

It updates conf.py, 6 associated autogenerated example markdown files in docs, and 2 templates to create these markdown files (9 files in total)
It renames the readme files to all have the same format (case sensitive) and makes similar name changes to some other files.
I updated some files here and there in the docs because there are many warnings on RTD. These changes removed most of those allowing me to better debug the updates to conf.py.

EwoutH · 2024-10-20T18:11:30Z

Thanks for the explanation. I will be honest, I don't like these monster PRs that do multiple things and are quite far away from an atomic change. There's a big chance something slips up in there. I also do recognize real-world development flows and practical limitations.

6 associated autogenerated example markdown files in docs

Is there anyway we can not keep these files in our history, since they are autogenerated from existing content and thus 100% duplicate. I suspect this will trigger (huge) CI issues and changesets when updating examples or working on other stuff.

I see two options:

Build then in Sphinx / RTD and don't generated them in git at all
(automatically) add them to the .gitignore.

For me 1 is preferred, but 2 is also acceptable.

The other files are fine, renames and cleanups look good. Ideally the warnings cleanup would be a separate PR, but I do recognize how this goes so for now its fine from my part.

quaquel · 2024-10-20T18:16:09Z

Is there anyway we can not keep these files in our history, since they are autogenerated from existing content and thus 100% duplicate. I suspect this will trigger (huge) CI issues and changesets when updating examples or working on other stuff.

conf.py handles all this so there should not be a problem. I prefer to have them in the history because otherwise rtd has to first regenerate the md files and then regenerate the html pages every single time. Having them in git speeds up the building of docs because they are only recreated by RTD if the associated example Python files are updated.

EwoutH · 2024-10-20T18:25:17Z

But that means every time I update an example, the all the changes will be duplicate into the PR diff. I find that to be a major downside.

Can't we cache them elsewhere, and just hash the source files and templates use to check for cache hits?

Sorry to make this PR more complicated, but this duplication is something I really really would like to avoid.

quaquel · 2024-10-20T18:30:30Z

But that means every time I update an example, the all the changes will be duplicate into the PR diff. I find that to be a major downside.

This only happen if you build the docs yourself locally and commit those changes. Also there is a single md file per example, so I am not sure what the problem is.

quaquel · 2024-10-20T18:31:37Z

Can't we cache them elsewhere, and just hash the source files and templates use to check for cache hits?

The simpler solution would be to just always rebuild the md files and ignore them via gitignore, but I fail to see what the problem is with the current solution as explained above.

EwoutH · 2024-10-20T18:35:36Z

The simpler solution would be to just always rebuild the md files and ignore them via gitignore

That's a totally acceptable solution for me right now. We can add caching later if that seems to be needed.

I fail to see what the problem is with the current solution as explained above.

Basically, each change to the examples would have a completely duplicate diff, as both the example itself and generated readme is changed in exactly the same way.

quaquel · 2024-10-20T18:38:57Z

Basically, each change to the examples would have a completely duplicate diff, as both the example itself and generated readme is changed in exactly the same way.

I guess you mean markdown rather than readme. As indicated, this only happens if you build the docs locally. Who does that other than when developing and testing doc-related things? Note that this approach has been used for the workbench for several years. Have you ever run into this potential diff problem there?

I am happy to change it but I still fail to see the problem here.

EwoutH · 2024-10-20T18:43:06Z

But if these files are tracked/included in git, but not updated in a PR, when are they updated? And why does this PR have them?

(signing off for today, will reply tomorrow)

quaquel · 2024-10-20T18:50:30Z

But if these files are tracked/included in git, but not updated in a PR, when are they updated? And why does this PR have them?

If not updated in the PR, conf.py during building the docs on RTD will overwrite the existing files (depending on timestamps, basically if agent.py, model.py, or readme.md is newer than the exising <example_name>.md file, the md file will be overwritten). They are here because I tested conf.py locally.

Regardless, I have made the change you requested. The example related markdown files are now in gitignore and conf.py just allways creates them on RTD when building docs.

as requested by ewout

EwoutH · 2024-10-21T02:33:05Z

They are here because I tested conf.py locally.

Okay, so they are only generated within Sphinx / Readthedocs, and not tracked in git by default. So it’s already like the preferred solution I proposed.

When committing, please check which files you want to include in the PR and which files are just local because you tested / experimented with some things.

Two remaining points, then we can properly review:

There are two Readme files that are now duplicated. I think instead of rename you did copy and rename.
The same happened to the st_app.py I think. I think it would be good to keep the different name for the Streamlit app, since then users recognize it’s different. We might add a SolaraViz in the future called app.py, and then it would be nice to include both. I would say: Check for both app.py and st_app.py if they exist, and if they do, include them.

quaquel · 2024-10-21T05:22:56Z

It's a bit more complicated than you make it out to be. But I don't want to continue arguing the point.

I don't see two readme files that are identical, neither in my IDE nor in my file explorer. Please can you be explicit which files according to you are duplicated?

Including both app and st_app if they both exist and only either one, if they exist, is tricky with string templates. I actually experimented with trying to get that to work but it did not render nicely. So happy to take yet another stab at it but it's not entirely trivial as far as I can tell. Moreover, from a future proofing point of view, my preference would be to standardize on app.py and only showcase solara visualizations.

If there is any file duplication, it happened after rebasing because of the examples --> mesa.examples move. This rebase broke somehow GitHub Desktop and I had to resort to the command line.

EwoutH · 2024-10-21T06:25:57Z

May I clean up the PR?

quaquel · 2024-10-21T06:28:40Z

go ahead

EwoutH · 2024-10-21T07:13:05Z

I tried doing a proper rebase, but the git threw a few errors I never have seem before. So I ported the changes over manually in #2392.

quaquel · 2024-10-21T08:30:09Z

solved via #2392

quaquel added 30 commits January 31, 2024 18:59

further updates

d90b0f7

Update benchmarks/WolfSheep/__init__.py

9586490

Merge remote-tracking branch 'upstream/main'

4aaa35d

Merge remote-tracking branch 'upstream/main'

d31478c

Merge remote-tracking branch 'upstream/main'

6e4c72e

Merge remote-tracking branch 'upstream/main'

70fbaf5

Merge remote-tracking branch 'upstream/main'

724c8db

Merge remote-tracking branch 'upstream/main'

45184a4

Merge remote-tracking branch 'upstream/main'

3d75d30

Update __init__.py

2759244

Merge remote-tracking branch 'upstream/main'

fc8aaea

Merge remote-tracking branch 'upstream/main'

1ba465d

Merge remote-tracking branch 'upstream/main'

2b5e822

Merge remote-tracking branch 'upstream/main'

3847799

Merge remote-tracking branch 'upstream/main'

301d80e

Merge remote-tracking branch 'upstream/main'

fe3d655

Merge remote-tracking branch 'upstream/main'

7d18880

Merge remote-tracking branch 'upstream/main'

6b49a3b

Merge remote-tracking branch 'upstream/main'

b9909e6

Merge remote-tracking branch 'upstream/main'

8ce3d83

Merge remote-tracking branch 'upstream/main'

88fbf74

Merge remote-tracking branch 'upstream/main'

077aca5

Merge remote-tracking branch 'upstream/main'

c771e3b

Merge remote-tracking branch 'upstream/main'

3fa0c27

Merge remote-tracking branch 'upstream/main'

8fdcf5c

Merge remote-tracking branch 'upstream/main'

475b060

Merge remote-tracking branch 'upstream/main'

fb93261

Merge remote-tracking branch 'upstream/main'

86f2f0c

Merge remote-tracking branch 'upstream/main'

dc1d843

Merge remote-tracking branch 'upstream/main'

9857ef1

trying to merge

824459f

tpike3 requested changes Oct 19, 2024

View reviewed changes

quaquel added 2 commits October 19, 2024 20:21

Update index.md

e0ac4ba

some typo fixes

8fa9f53

quaquel added 5 commits October 20, 2024 21:24

allways create md files

4be7dd2

as requested by ewout

Update conf.py

f92b37c

Update conf.py

5163352

Merge branch 'main' into docs

6a34561

Update conf.py

04e8763

EwoutH mentioned this pull request Oct 21, 2024

Include examples in readthedocs (port) #2392

Merged

quaquel closed this Oct 21, 2024

quaquel deleted the docs branch October 24, 2024 20:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Include examples in readthedocs #2382

Include examples in readthedocs #2382

quaquel commented Oct 17, 2024 •

edited

Loading

tpike3 left a comment

tpike3 Oct 19, 2024

tpike3 Oct 19, 2024

tpike3 Oct 19, 2024

tpike3 Oct 19, 2024

quaquel Oct 21, 2024

tpike3 Oct 19, 2024

tpike3 Oct 19, 2024

tpike3 Oct 19, 2024

quaquel Oct 19, 2024

EwoutH Oct 21, 2024

quaquel commented Oct 19, 2024

EwoutH commented Oct 20, 2024

quaquel commented Oct 20, 2024

EwoutH commented Oct 20, 2024

quaquel commented Oct 20, 2024 •

edited

Loading

EwoutH commented Oct 20, 2024

quaquel commented Oct 20, 2024

quaquel commented Oct 20, 2024

EwoutH commented Oct 20, 2024

quaquel commented Oct 20, 2024

EwoutH commented Oct 20, 2024 •

edited

Loading

quaquel commented Oct 20, 2024 •

edited

Loading

EwoutH commented Oct 21, 2024

quaquel commented Oct 21, 2024 •

edited

Loading

EwoutH commented Oct 21, 2024

quaquel commented Oct 21, 2024

EwoutH commented Oct 21, 2024

quaquel commented Oct 21, 2024


		For more information about this model, read the NetLogo's web page: http://ccl.northwestern.edu/netlogo/models/VirusonaNetwork.

		JavaScript library used in this example to render the network: [d3.js](https://d3js.org/).

Include examples in readthedocs #2382

Include examples in readthedocs #2382

Conversation

quaquel commented Oct 17, 2024 • edited Loading

tpike3 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

quaquel commented Oct 19, 2024

EwoutH commented Oct 20, 2024

quaquel commented Oct 20, 2024

EwoutH commented Oct 20, 2024

quaquel commented Oct 20, 2024 • edited Loading

EwoutH commented Oct 20, 2024

quaquel commented Oct 20, 2024

quaquel commented Oct 20, 2024

EwoutH commented Oct 20, 2024

quaquel commented Oct 20, 2024

EwoutH commented Oct 20, 2024 • edited Loading

quaquel commented Oct 20, 2024 • edited Loading

EwoutH commented Oct 21, 2024

quaquel commented Oct 21, 2024 • edited Loading

EwoutH commented Oct 21, 2024

quaquel commented Oct 21, 2024

EwoutH commented Oct 21, 2024

quaquel commented Oct 21, 2024

quaquel commented Oct 17, 2024 •

edited

Loading

quaquel commented Oct 20, 2024 •

edited

Loading

EwoutH commented Oct 20, 2024 •

edited

Loading

quaquel commented Oct 20, 2024 •

edited

Loading

quaquel commented Oct 21, 2024 •

edited

Loading