Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

replace model with random in AgentSet init #2350

Merged
merged 2 commits into from
Oct 13, 2024
Merged

Conversation

quaquel
Copy link
Member

@quaquel quaquel commented Oct 13, 2024

This PR replaces the model argument in AgentSet.__init__ with random as a keyword argument as argued in #2323.

Motivation:

  • the only thing retrieved from model is the random number generator. Passing model serves no other function and there is no other use case for having model in the agentset.
  • For advanced use cases, explicitly passing random allows users to control which random number generator is being used. The most common reason for wanting this is for variance reduction through common random numbers. See Remove "model" from agentset initialization #2323 and links provided there for a further elaboration on this argument.
  • I cannot currently change CellCollection.agents to returning an agentset because CellCollection does not have a reference to model even though it does have access to its random number generator.

Some have argued that this change creates extra load on the user. I disagree. First, In none of the examples in mesa-examples is the user creating an AgentSet object themselves. This highlights that it will be rare for users to create AgentSet themselves directly in all but the most advanced use cases. It also implies that this change is almost excusively confined to the inner workings of mesa. Second, this API is identical to the new gridspaces were we pass random explicitly instead of passing model only to get access to the central random number generator. So, this PR in fact makes the code base internally more coherent and therefore easier to use.

Another argument against this change is that we might potentially expand in the future AgentSet to use other model attributes. Again, I am not convinced by this unless clear examples of useful AgentSet functionality are given that require access to Model. We have had AgentSet for almost a year. No example has come up, I cannot come up with a convincing example, and None have not been offered.

@quaquel quaquel requested a review from EwoutH October 13, 2024 08:07
@quaquel quaquel requested a review from Corvince October 13, 2024 08:07
Copy link

Performance benchmarks:

Model Size Init time [95% CI] Run time [95% CI]
BoltzmannWealth small 🔵 -3.3% [-4.0%, -2.5%] 🔵 +0.3% [+0.2%, +0.4%]
BoltzmannWealth large 🔵 -0.7% [-1.1%, -0.4%] 🔵 +0.4% [-0.1%, +0.8%]
Schelling small 🔵 -0.1% [-0.7%, +0.5%] 🔵 -0.5% [-0.7%, -0.3%]
Schelling large 🔵 -0.7% [-1.5%, +0.3%] 🔵 -1.3% [-1.9%, -0.7%]
WolfSheep small 🔵 -0.7% [-0.9%, -0.5%] 🔵 +0.2% [-0.0%, +0.4%]
WolfSheep large 🔵 -0.5% [-2.1%, +0.6%] 🔵 -0.5% [-2.0%, +1.0%]
BoidFlockers small 🔵 -0.1% [-0.4%, +0.2%] 🔵 +1.8% [+1.1%, +2.5%]
BoidFlockers large 🔵 -0.3% [-1.4%, +0.8%] 🔵 +0.9% [+0.4%, +1.4%]

Copy link
Member

@EwoutH EwoutH left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this change. Thanks especially for the extensive motivation, that’s really appreciated.

Another argument against this change is that we might potentially expand in the future AgentSet to use other model attributes.

This was the first thing I was thinking about. However, the AgentSet has been with us since 2.2, is very actively integrated and developed and so far we never needed it. So a few months back I might give this argument a lot of weight, but now I think we will perfectly manage without it.

Since this is technically a breaking change, I would like a second maintainer to approve, but from my perspective, good to go!

@EwoutH EwoutH added breaking Release notes label enhancement Release notes label labels Oct 13, 2024
@rht
Copy link
Contributor

rht commented Oct 13, 2024

I cannot currently change CellCollection.agents to returning an agentset because CellCollection does not have a reference to model even though it does have access to its random number generator.
So, this PR in fact makes the code base internally more coherent and therefore easier to use.

Having access to the model everywhere is arguably also coherent and easy to use. So it could go either way.

We have had AgentSet for almost a year. No example has come up, I cannot come up with a convincing example, and None have not been offered.

This was the first thing I was thinking about. However, the AgentSet has been with us since 2.2, is very actively integrated and developed and so far we never needed it. So a few months back I might give this argument a lot of weight, but now I think we will perfectly manage without it.

It's because the use cases (the "experimental evidences") are limited to the existing mesa-examples, and we haven't received feedback from advanced users yet. This is in contrast to the datacollector where we received lots of requests for multi-agent data collection.

But I'm fine with this PR. It can always be changed back in the future when needed. The added bonus with this PR is that I have observed that AgentSet's utility may extend beyond ABM, as an arbitrary object DataFrame, and so tying it to an ABM model might actually be limiting it.

@quaquel quaquel merged commit e255a2d into projectmesa:main Oct 13, 2024
11 of 14 checks passed
@quaquel quaquel deleted the agentset branch October 13, 2024 13:32
@EwoutH
Copy link
Member

EwoutH commented Oct 13, 2024

Apparently this broke our docs build: https://readthedocs.org/projects/mesa/builds/25927550/

@EwoutH
Copy link
Member

EwoutH commented Oct 15, 2024

@quaquel could you check what's going wrong in the tutorial?

@quaquel
Copy link
Member Author

quaquel commented Oct 15, 2024

The tutorial is still using a scheduler, so the tutorial needs a bigger overhaul.

I have no idea what causes the error and I can't seem to replicate it locally.

@EwoutH
Copy link
Member

EwoutH commented Oct 15, 2024

Okay, @tpike3 is working on a tutorial update, let's first wait if he already has fixed the issue (feel free to open a draft PR with your current progress, Tom).

@quaquel
Copy link
Member Author

quaquel commented Oct 15, 2024

should I split the fix in time.py into a seperate PR then?

EwoutH pushed a commit that referenced this pull request Oct 15, 2024
As pionted out in by @EwoutH, #2350 broke the tutorial. This is due to not updating the schedulers to pass model.random. This fixes that bug (and does a few minor updates to the tutorial).

The tutorial needs a complete update to remove schedulers and use (in my view) the new grid spaces. I am not going to do that in this PR.
@quaquel quaquel mentioned this pull request Oct 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking Release notes label enhancement Release notes label
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants