Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update into_tutorial #2372

Merged
merged 9 commits into from
Oct 22, 2024
Merged

Update into_tutorial #2372

merged 9 commits into from
Oct 22, 2024

Conversation

tpike3
Copy link
Member

@tpike3 tpike3 commented Oct 16, 2024

  • Update intro tutorial to be compatible with Mesa 3.0

Update intro tutorial for Mesa 3.0
@quaquel
Copy link
Member

quaquel commented Oct 16, 2024

A first quick comment. You currently have

class MoneyModel(mesa.Model):
    """A model with some number of agents."""

    def __init__(self, N):
        super().__init__()

This should be changed to

class MoneyModel(mesa.Model):
    """A model with some number of agents."""

    def __init__(self, N, seed=None):
        super().__init__(seed=seed)

The same change should be made in all other places where you have the MoneyModel class. This ensures that seed is handled correctly and is controllable.

@tpike3
Copy link
Member Author

tpike3 commented Oct 17, 2024

A first quick comment. You currently have

class MoneyModel(mesa.Model):
    """A model with some number of agents."""

    def __init__(self, N):
        super().__init__()

This should be changed to

class MoneyModel(mesa.Model):
    """A model with some number of agents."""

    def __init__(self, N, seed=None):
        super().__init__(seed=seed)

The same change should be made in all other places where you have the MoneyModel class. This ensures that seed is handled correctly and is controllable.

I added seed with None and added a challenge to set a seed and see the impact when setting up the initial model and agent interaction. (In this case it demonstrates that the agent order stays in whatever random configuration the seed is set too)

As the seed kwarg is set to none, I am thinking we should not put it in each one. Thoughts?

@EwoutH
Copy link
Member

EwoutH commented Oct 17, 2024

Maybe we need to make seeds / random numbers a separate section - but not sure that fits in an introductory tutorial.

@quaquel
Copy link
Member

quaquel commented Oct 18, 2024

As the seed kwarg is set to none, I am thinking we should not put it in each one. Thoughts?

Maybe we need to make seeds / random numbers a separate section - but not sure that fits in an introductory tutorial.

  1. Seed analysis is critical in developing and using ABMs
  2. So, we need to ensure that all examples ensure that the seed is user-controllable (hence my original comment) even if it's not used in each example.
  3. Boltzman is not an ideal model to showcase the impact of changing the seed. Something like wolf-sheep might be better for that because you can get 3 distinct dynamics: wolves eating all sheep and then also dying out, wolves dying out and sheep taking over (limited by grass), and meta-stable dynamic equilibrium between wolf and sheep over many steps. There might thus be room for a further tutorial, probably centered on experimentation (so also covering batch run and something very basic on the design of experiments). Here we might explicitly cover seed analysis.

- Updates descriptions to be consistent with 3.0
- Adds user challenges
- Corrects headers
@tpike3
Copy link
Member Author

tpike3 commented Oct 18, 2024

I think it is critical to have the seed in there and upfront, it is one of those oft overlooked and but critically important. Everyone wants to ignore the butterfly effect, because it is intuitively scary.

How about I bookend seed--- right up front and then as the last section is batchrun put it in there?

I got the writing part of the documentation done,

#TODO is add some of the new agentset functionalities.

@quaquel
Copy link
Member

quaquel commented Oct 18, 2024

How about I bookend seed--- right up front and then as the last section is batchrun put it in there?

Sounds good to me. We can keep an experimentation tutorial for later.

@Corvince
Copy link
Contributor

Also please check for changes between the Model in the tutorial and the updated example in mesa/examples. The latter for example uses "n" instead of "N" for the number of agents

- add AgentSet functionality section
- change N parameter to n to be consistent
- improve grammar and readability
@tpike3 tpike3 marked this pull request as ready for review October 20, 2024 11:28
Copy link

Performance benchmarks:

Model Size Init time [95% CI] Run time [95% CI]
BoltzmannWealth small 🔵 -2.7% [-4.9%, -0.4%] 🔵 -1.6% [-1.8%, -1.5%]
BoltzmannWealth large 🔵 +0.5% [+0.1%, +0.8%] 🔵 -2.7% [-4.1%, -1.2%]
Schelling small 🔵 -0.2% [-0.7%, +0.3%] 🔵 -1.3% [-1.6%, -1.0%]
Schelling large 🔵 +0.6% [-0.2%, +1.3%] 🔵 -2.0% [-3.1%, -1.1%]
WolfSheep small 🔵 -1.0% [-1.3%, -0.7%] 🔵 +0.2% [+0.0%, +0.4%]
WolfSheep large 🔵 -2.3% [-3.4%, -1.4%] 🔵 -3.7% [-6.8%, -1.2%]
BoidFlockers small 🔵 -1.7% [-2.5%, -0.9%] 🔵 +0.4% [-0.4%, +1.3%]
BoidFlockers large 🔵 -0.3% [-1.0%, +0.4%] 🔵 -0.6% [-1.0%, -0.1%]

- Fix build failure
- Make portions independent to avoid computational overhead
@quaquel
Copy link
Member

quaquel commented Oct 20, 2024

I am scanning the tutorial. A few quick points:

  1. There is a section titled "Making the Agents do", but we have a step function on the model and the agent, so this section probably needs a bit of a rewrite.
  2. In the "Making the Agents do" there is a link to the how to guide, but this is very outdated (see The How-To guide is outdated #2385).
  3. The section titled "Running the Model" says that running the model will result in printing the agent unique_id, but since there is no step method in MoneyAgent at that point, that's not correct.
  4. In the batchrun part there is still a mention of schedulers.

@tpike3
Copy link
Member Author

tpike3 commented Oct 20, 2024

Ok this is ready now that the build is working . As there is a lot of happies to glads, besides making compatible with 3.0 the three big parts to look at are:

1- I added an AgentSet functionality section. As I am always behind, I am a little worried I did code that worked vs good coding practices we want to instill in students
2 - I added a seed section at the beginning and then in the comparison portion of batch run with a quick overview of reproducibility and sensitivity analysis
3 - In the visual I made each part self contained. As the more dashboards being run the slower it gets. I also moved the stand alone histogram to after the last dashboard. This isn't elegant but I could not find an elegant way to end the process in one one cell with another cell.

This should however give us a very nice tutorial for CSSSA

@tpike3
Copy link
Member Author

tpike3 commented Oct 20, 2024

I am scanning the tutorial. A few quick points:

  1. There is a section titled "Making the Agents do", but we have a step function on the model and the agent, so this section probably needs a bit of a rewrite.
  2. In the "Making the Agents do" there is a link to the how to guide, but this is very outdated (see The How-To guide is outdated #2385).
  3. The section titled "Running the Model" says that running the model will result in printing the agent unique_id, but since there is no step method in MoneyAgent at that point, that's not correct.
  4. In the batchrun part there is still a mention of schedulers.

Thanks--- on it!

@quaquel
Copy link
Member

quaquel commented Oct 20, 2024

This should however give us a very nice tutorial for CSSSA

I agree, this looks like a great tutorial. Once the few points I had are addressed this is good to go as far as I am concerned.

@EwoutH
Copy link
Member

EwoutH commented Oct 20, 2024

I will review today!

- update agent "do" section
- remove schedule reference in batch_run
- other minor fixes
@tpike3
Copy link
Member Author

tpike3 commented Oct 20, 2024

I am scanning the tutorial. A few quick points:

  1. There is a section titled "Making the Agents do", but we have a step function on the model and the agent, so this section probably needs a bit of a rewrite.
  2. In the "Making the Agents do" there is a link to the how to guide, but this is very outdated (see The How-To guide is outdated #2385).
  3. The section titled "Running the Model" says that running the model will result in printing the agent unique_id, but since there is no step method in MoneyAgent at that point, that's not correct.
  4. In the batchrun part there is still a mention of schedulers.

Thanks--- on it!

Updates made

  1. I updated the "Making Agents Do" section
  2. Part of the pull request is updating the "How To" to add agentset features (just taken form @EwoutH migration guide
  3. Fixed
  4. Removed

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like there are now some duplicate parts in the Visualisation tutorial, is that intentional?

Copy link
Member

@EwoutH EwoutH left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really great job on the tutorial! I especially like all the challenges to get users to engage with the code!

Please check if the changes to the viz tutorial are intentional and as expected, but otherwise, good to go!

@tpike3
Copy link
Member Author

tpike3 commented Oct 21, 2024

Really great job on the tutorial! I especially like all the challenges to get users to engage with the code!

Please check if the changes to the viz tutorial are intentional and as expected, but otherwise, good to go!

The repetition is intentional; although very brute force

The problem is when you run three dashboards at the same the last one barely moves. So each part is self contained with instructions to restart the kernel. So far I haven't found a good solution, with but this was better than users getting frustrated with the visuals. I imagine our tutorial is a bit of an edge case for jupyter and solara.

@EwoutH
Copy link
Member

EwoutH commented Oct 21, 2024

@Corvince or @maartenbreddels any suggestions? It's about this changeset in the visualisation tutorial (rendered: old | new).

@tpike3
Copy link
Member Author

tpike3 commented Oct 21, 2024

Really great job on the tutorial! I especially like all the challenges to get users to engage with the code!

Please check if the changes to the viz tutorial are intentional and as expected, but otherwise, good to go!

Awesome!! Thanks @EwoutH Very curious to see their thoughts!

@tpike3 tpike3 merged commit 470ab0a into projectmesa:main Oct 22, 2024
11 of 12 checks passed
@EwoutH EwoutH added the docs Release notes label label Oct 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs Release notes label
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants