Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ns/fix/versionable bad bounds #1551

Merged
merged 7 commits into from
Sep 23, 2024
Merged

Conversation

nsarlin-zama
Copy link
Contributor

@nsarlin-zama nsarlin-zama commented Sep 18, 2024

needed by: #1482

PR content/description

Fix the bounds added in the derived traits for the Versionize macro. These bounds were too restrictive and there were cases where they were not satisfiable, so the versionize method on a type was actually not callable. This was the case for example when a type was defined with a generic that is not itself used, but only through associated types. For example:

trait WithAssoc {
  type Assoc;
}

struct MyStruct<T: WithAssoc> {
  inner: T::Assoc
}

Using the #[derive(Versionize)] on this type before the PR would require T: Versionize which is actually not necessary and sometimes not possible. This was caused by the proc macro trying to apply bounds directly on the generics of the type.

The main idea behind this PR is to add bounds closer to what is actually used, and not necessarily to the generics. For example, If I have 3 traits:

trait TraitA {}

trait TraitB {
  fn method_b(&self);
}

trait TraitC {
  fn method_c(&self);
}

and these 2 types:

struct Type1<T> {
  value: T
}

impl<T: TraitA> TraitB for Type1<T> {
  fn method_b(&self) {}
}

struct Type2<T> {
  inner: Type1<T>
}

impl<T: ??> TraitC for Type2<T> where ?? {
  fn method_c(&self) {
    self.inner.method_b();
  }
}

I have 2 ways of expressing the bound:

  • impl<T: TraitA> TraitC for Type2<T>
  • impl<T> TraitC for Type2<T> where Type1<T>: TraitB

This PR works by using the second method instead of the first one. The advantage is that theses bounds are easier to generate by looking only at Type2, without knowing the internals of Type1.

As a side effect, the #[versionize(..., bound = "...")] attribute for the proc macro is not necessary anymore to express the bounds present in tfhe-rs, so it have been removed. This is a breaking change. There might still be specific corner cases that will require it, but as I don't have any example to test it I preferred to removed it.

This PR also adds the versioning of vector of small tuples of arbitrary versioned types.

Check-list:

  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)
  • Relevant issues are marked as resolved/closed, related issues are linked in the description
  • Check for breaking changes (including serialization changes) and add them to commit message following the conventional commit specification

@cla-bot cla-bot bot added the cla-signed label Sep 18, 2024
@nsarlin-zama nsarlin-zama force-pushed the ns/fix/versionable_bad_bounds branch 2 times, most recently from 2ebd7b0 to f1548d1 Compare September 18, 2024 11:56
@nsarlin-zama nsarlin-zama force-pushed the ns/fix/versionable_bad_bounds branch 3 times, most recently from a7628ea to b0c32a9 Compare September 18, 2024 15:54
Over-restrictive derived bounds were in some cases unsatisfiable, making the
`versionize` method uncallable.

BREAKING_CHANGE:
- The `#[versionize(bound = ...)]` attribute is not needed anymore, so it has
been removed.
@nsarlin-zama nsarlin-zama force-pushed the ns/fix/versionable_bad_bounds branch from b0c32a9 to 2e8c436 Compare September 19, 2024 07:48
@nsarlin-zama
Copy link
Contributor Author

nsarlin-zama commented Sep 19, 2024

For reference, the rust std lib and serde have the same problem this PR is trying to solve: rust-lang/rust#26925.
Serde even implemented it in 0.7 and reverted in 0.8 (2016). dtolnay explained the 3 cases where this approach did not work at the time, mostly because of bugs in the trait solver: dtolnay/syn#370 (comment):

  1. Private in public: when a pub type has a private field, the bound expose this private type
  2. Overflow evaluating requirements: when there are recursions in the type definitions, the trait solver fails to resolve the bounds
  3. Lifetime deduplication inference is broken: a bug when a type has 2 attributes with references to the same type but with different lifetimes

It looks like 1. has been fixed on the compiler side and does not raise an error anymore. I will try to test 2 and 3 but I think that if they still fail we should adopt this PR. The macros of tfhe-versionable create several types and implement different traits, so it can be a bit more difficult to fix than what serde does (ie: having a #[serde(bound = ...)] argument, which I tried in the previous version but did not truly work).
The errors 2. and 3 are not encountered in the tfhe-rs codebase, and even if that was the case it would just mean that we have to do a manual impl of the trait.

@nsarlin-zama
Copy link
Contributor Author

I have done a few tests on these issues:

  1. is fixed
  2. still fails to compile due to the circular dependency in the type. If we ever use this kind of type we will need to do a manual impl
  3. fails to compile but I think the problem is not only caused by the rust compiler, lifetime generics are not very well handled by the proc macro. Since we avoid them in tfhe-rs I have not properly checked them but that is something that I should fix when I have more time.

Copy link
Member

@IceTDrinker IceTDrinker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I won't pretend I understand everything that's going on, but the assurance it works with the test runs gives reassurance on that end (and the fact our lib and others are using this successfully)

some small stuff I saw

utils/tfhe-versionable/README.md Outdated Show resolved Hide resolved
utils/tfhe-versionable/README.md Outdated Show resolved Hide resolved
utils/tfhe-versionable/src/lib.rs Show resolved Hide resolved
utils/tfhe-versionable/README.md Outdated Show resolved Hide resolved
Copy link
Contributor

@mayeul-zama mayeul-zama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@nsarlin-zama nsarlin-zama merged commit 3ff81c3 into main Sep 23, 2024
86 of 88 checks passed
@nsarlin-zama nsarlin-zama deleted the ns/fix/versionable_bad_bounds branch September 23, 2024 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants