Use when
and make eager checking for spec fn requires configurable
#764
Replies: 8 comments 10 replies
-
I generally like this new proposal! The more errors detected earlier, the better! A small part of it is a bit confusing though. Specifically, the example immediately after the line “ If that is indeed the case, then the encoding is already including the Also, as another note, rather than |
Beta Was this translation helpful? Give feedback.
-
@Chris-Hawblitzel asked about functions like The intent of the proposal is to have a single user-facing concept which interacts clearly with decreases and ensures. I think important questions for these functions are:
|
Beta Was this translation helpful? Give feedback.
-
My understanding of this proposal is that it intends to maximize the benefits of catching mistakes early (particularly mistakes related to passing incorrect arguments to spec functions), while minimizing the costs to the SMT solver. I definititely agree with this intent. I fear, though, that "when" clauses, as proposed here, may be significantly more expensive to the SMT solver than recommends clauses, even in proof and exec functions, and even though the proposal strives to avoid extra overhead in proof and exec functions. Preconditions have a tendency to accumulate in long series of nested definitions. In the following example, g needs to declare P1 as a precondition because f needs it, and h needs to declare P1 && P2 as a precondition because g needs it:
When the SMT solver expands h(arg) = body_h[x := arg], including expanding g and f, it has to jump through many hoops, proving P1 three times, P2 twice, and P3 once. If we used recommends instead:
then the SMT solver can simply substitute the bodies of f, g, and h directly, with no additional proof obligations, when expanding h(arg). This overhead is necessary for recursive definitions. But for most definitions, I think this overhead is all SMT cost and little programmer benefit. If our goal is to check spec function preconditions more eagerly and in more places, I'd rather just check "arg" at the call site to h, instead of adding extra SMT overhead to the expansion of h, g, and f. Perhaps we could do this with recommends if the recommends checking is sufficiently convenient to customize (e.g. #![recommends(warn|allow)]). |
Beta Was this translation helpful? Give feedback.
-
The distinction between checking spec functions preconditions in spec function bodies but not in exec and proof function bodies is interesting. I agree that we should strive to avoid checking exec/proof obligations and spec precondition obligations together in one giant verification condition. But if we wanted to eagerly check spec function preconditions in exec/proof code (I'm not saying we do, necessarily), we could potentially check them separately, with one VC for the spec precondition checking and one for the exec/proof obligations. And the VC for spec precondition checking could run first. |
Beta Was this translation helpful? Give feedback.
-
The following is the first revised proposal, #764 (comment) is the latest. First revised proposal.Here is a proposed revision, based on the feedback so far:
spec fn some_gt_0(s: Option<int>) -> bool
when s.is_some()
{
s.unwrap() > 0
} When clauses take up the role of recommends, but are checked by default on calls from other spec functions. spec fn gt_0(s: Option<int>) -> bool
{
some_gt_0(s) // ERROR: failed when
} This is a hard error (unlike current recommends). Optionally, the user can disable when-checking for spec functions:
spec(unchecked) fn gt_0(s: Option<int>) -> bool
{
some_gt_0(s) // no error
}
spec fn minus2(i: int) -> int
when i >= 2
{
i - 2
}
proof fn test() {
assert(minus2(2) == 0); // succeeds
assert(minus2(0) == -2); // succeeds !!! do we think this would be confusing? !!!
} If a recursive function only decreases with certain arguments, spec fn minus2_rec(n: nat) -> nat
decreases n
when n >= 2
{
if n == 2 {
0
} else {
1 + minus2(n - 1)
}
} With the upcoming spec-ensures, if a spec function has
An attribute Avoiding hard errors in common cases where they don't matterWe can keep an (advanced) affordance for functions for which we do not want strict precondition checks but we do want warnings when arguments may be outside a certain domain (like pub open spec fn spec_index(self, i: int) -> A
recommends 0 <= i < self.len()
{
self.index(i)
}
How to silence recommends failuresIf they don't want to introduce a
Calls in a function marked Recursive functions
spec fn sum_many(s: Seq<nat>, indices: Seq<int>) -> nat
decreases indices.len()
{
if forall|i| 0 <= i < indices.len() ==> 0 <= #[trigger] indices[i] < s.len() {
if indices.len() > 0 { s[indices[0]] + sum_many(s, indices.drop_first()) } else { 0 }
} else { arbitrary() }
} Spec-ensuresIf a non-recursive spec function has |
Beta Was this translation helpful? Give feedback.
-
I'd like to argue against defaulting to "hard errors" when checking spec functions. I have to admit that I have a bias here: I was the one who implemented the far opposite, of defaulting to no checks at all, and having to opt in just to get warnings ("spec(checked)"). I do like the idea of making this option easily configurable at a function level, module level, and crate level via something like Here's my reason for suggesting "warn by default". Verus's ultimate goal is to verify exec functions. For these, verification is mandatory; it's unsound to disable verification. Proof functions can support exec functions, and for soundness, proof functions must also be verified. Finally, for soundness, recursive spec functions need to have their decreases clauses verified. Beyond this, verifying spec functions is not strictly necessary, but can be helpful in catching bug in spec definitions early. As long as it's helpful, people should enable checking inside spec function definitions. However, this checking is not an end in itself, but rather a tool for easing the verification of proof and exec code, which is the ultimate goal. If checking spec function definitions ever starts to become more of a burden than a benefit, people should simply not enable this checking in places where the pain exceeds the gain. This is why I like the idea of making this checking easily configurable, making it convenient to promote to "error" to demote to "warn" or "allow" as is appropriate to each situation. I'm worried that if the default for checking spec functions is "error", people will get the impression that demoting the checking to "warn" or "allow" is unsound and will be afraid to do this, even when they should, and spec functions will simply pile up more and more proof obligations out of a sense of dutiful completeness. To continue my earlier example, suppose someone implements a series of functions f, g, h, and wants to use h to implement
The programmer might see a warning or error when their implementation of
I'm afraid that with a default of "error", people will assume that they have to keep propagating the error, and think that
I think this would be the wrong conclusion; I would consider I think warn-by-default is a nice balance here: it makes people aware of checking spec functions, so that they can decide for themselves whether to be more strict (e.g. "deny(when)") or more lenient (e.g. "allow(when)"). |
Beta Was this translation helpful? Give feedback.
-
The central issue here is our mental model of the people we're trying to design the language for. The concept of deliberately using functions outside their intended domain to improve verification performance is a very advanced one, a subtle compile-time optimization. When I sort the concepts my students and engineering colleagues have acquired and need to acquire, that one is very late in the list. One could build a big project and never need it. Code (proof) that exploits it is much less readable, so even if one engineer gets it, they're setting a trap for the next unsuspecting person that joins the team. I'm not persuaded by the argument that "someone might think it unsound". Verus should have a mode akin to the By the way, as a sorta-advanced user, there are places in our Dafny corpus where we explicitly deployed this advanced mode by writing
...so I think of Another way for an advanced user to mitigate verification performance creep is to roll a long If you're worried that default-deny will prevent anyone from discovering the performance crimes they're committing: I propose we address that with good tooling. A verification-perf linter might say something like "in this project, when-checking these three spec fns is costing 470ms, which is 2.6% of total verification time. To reduce it, read this chapter (link) on In summary: my intuition is that the language design shouldn't be targeting people with Hawblitzel-level comprehension of what's happening under the covers. Default |
Beta Was this translation helpful? Give feedback.
-
Here is an alternative revision, that allows both styles (error-by-default and warn-by-default), using a flag, like Rust's lint levels (as suggested by @jaybosamiya): Shorter suggestions for It does so trying to fulfill as best I can the priorities of both: (a) ensuring that users get early, actionable, and easy-to-parse feedback to avoid "wild goose chases", and (b) making it clear to users that fulfilling The default level is
spec fn some_gt_0(s: Option<int>) -> bool
when s.is_some()
{
s.unwrap() > 0
} When clauses are checked by default on calls from other spec functions, and produce a warning or error depending on the spec fn gt_0(s: Option<int>) -> bool
{
some_gt_0(s) // WARNING or ERROR: unfulfilled when
} Optionally, the user can disable when-checking for spec functions: #[allow(unfulfilled_when)]
spec fn gt_0(s: Option<int>) -> bool
{
some_gt_0(s) // no error
}
spec fn minus2(i: int) -> int
when i >= 2
{
i - 2
}
proof fn test() {
assert(minus2(2) == 0); // succeeds
assert(minus2(0) == -2); // succeeds
} If a recursive function only decreases with certain arguments, spec fn minus2_rec(n: nat) -> (r: nat)
decreases n
when n >= 2
ensures r == n - 2
{
if n == 2 {
0
} else {
1 + minus2(n - 1)
}
} With the upcoming spec-ensures, if a spec function has proof fn caller(i: nat) {
let j = minus2_rec(i); // WARNING: unfulfilled when
assert(j == i - 2); // no error
} Proofs for
|
Beta Was this translation helpful? Give feedback.
-
NOTE: This is the original proposal. See the new proposal #764 (comment) based on feedback in this discussion.
Original proposal
Based on the discussion on the Slack, and on offline discussions with @Chris-Hawblitzel and @jonhnet, here is a proposal to change the user experience around "requires" for `spec` functions (by @jonhnet and @utaal). Proposal first, the rationale follows.spec
functions can optionally have awhen
clause.When clauses take up the role of recommends, but are checked by default on calls from other spec functions.
This is a hard error (unlike current recommends).
Optionally, the user can disable when-checking for spec functions:
when
always restricts where the function is defined, so:If a recursive function only decreases with certain arguments,
when
takes up the same role as before: restricting where the function is defined, and for which values of the arguments one needs to prove termination.With the upcoming spec-ensures, if a spec function has
when W ensures E
, then we learn thatW ==> E
. We are not going to have hard-checks onwhen
forspec
functions inproof
andexec
code, so we cannot simply learnE
after a call to the function.Spec functions are otherwise treated as before in proof and exec contexts (modulo the restriction of the definition to
when
). When a proof fails, we'd check thewhen
s of functions invoked by the proof, and report failures as warnings.An attribute
#![when(warn|allow)]
(or a similar name) disables hard errors for whens inside a module or crate, and optionally turns them into warnings instead. This does not change semantics / soundness, due to how termination andensures
are treated. This restores (most) of the previousrecommends
behavior for folks/projects that prefer it.Rationale
This does not require emitting hard-precondition checks in proof and exec functions. But this helps catch mistakes in
spec
functions early, which @jonhnet has found to be very valuable (rather, delaying these findings can be very costly during development).Patterns supported by a mix of
decreases when
andrecommends
can still be supported, but a bit more verbosely. See below.The name
when
is used instead ofrequires
because inproof
andexec
contexts (and inspec(unchecked)
) we don't emit a verification condition forwhen
but allow the proof to fail due to it being not satisfied.when
is "when" the function is defined.Behavior of spec-ensures: in Dafny, failure to satisfy the
requires
appears as an error and if the remaining proof text relies on the assumption of the ensures, there would be no further errors. Under this proposal, the failure to satisfy thewhen
appears as a warning, and then an additional error would appear related to the unavailable ensures conclusion. To recover the Dafny behavior, which we believe is more desirable, we can hold off on reporting errors for a failing proof, but re-run the query with hard requires checks, and strong ensures assumptions; more thought may be required in handling "nested" unchecked specs.Support for advanced patterns
It still allows advanced patterns when needed.
If we want to avoid a check at a call-site for certain functions where a hard-
when
check would be burdensome (e.g.spec_index
), we can write something like:And we can similarly support cases in which the hard-
when
would be cumbersome in recursive functions, like in @matthias-brun's example (with the current meaning ofrecommends
):which can still be represented as:
Beta Was this translation helpful? Give feedback.
All reactions