Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test scenarios #4

Merged
merged 1 commit into from
Aug 22, 2022
Merged

Conversation

ivan-gavran
Copy link
Contributor

Adding the list here in the form of PR.
The goal is that one the list is finalized, agreed upon, and implemented, we remove the .md document.
I chose the PR form because of its good infrastructure for discussing individual lines.

@ivan-gavran ivan-gavran linked an issue Aug 19, 2022 that may be closed by this pull request
@hvanz
Copy link
Member

hvanz commented Aug 22, 2022

Great list though I have the feeling that it describes many of all the possible scenarios in the model, and thus the model itself, which makes it a bit pointless as we want to let the model-checker explore all possible behaviours. If we write a TLA+ predicate for each test scenario in the list, there's not too much difference with writing them manually in Go.

Apart from that, I think we can try to cover different sets of behaviours by restricting the model. For example, we can cover all items in the Granting section by just disabling the Exec action, and explore the model until, let's say, depth 5. And for scenarios that are independent of the authorization logic, we can restrict the model to one authorization per time, and generate all the traces for each three kinds of authorizations.

@hvanz
Copy link
Member

hvanz commented Aug 22, 2022

We have also to take into account the fact that executing a single trace takes time (at least ~10 seconds?) which is quite a lot if we want to cover hundreds of scenarios, including trivial ones.

@ivan-gavran ivan-gavran marked this pull request as ready for review August 22, 2022 14:12
@ivan-gavran
Copy link
Contributor Author

As we agreed synchronously, I am merging this list into the repo.
The remarks on implementing it still hold:

  • there are probably more elegant ways of implementing those checks, without enumerating them one by one. E.g., by manipulating the View predicate.
  • when generating all possible cases for a particular invariant, we have to be careful because tests take time so there is no much space for having trivial tests

@ivan-gavran ivan-gavran merged commit 13808f1 into hernan/authz-model Aug 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Authz: create a list of properties to test
2 participants