Migrate to pydantic v2 #129
Conversation
|
Looks like a great start Nikita! Thanks so much for looking into this, keep up the good work! |
|
Update: 2.1 has resolved some bugs, including above issue, but there are corner cases remaining with custom serialization. Will continue tracking main for now. |
nikitakuklev
force-pushed
the
pydantic2
branch
from
49f3b89 to
5afa1c7
Compare
|
Thanks @nikitakuklev on your hard work RE this issue! What are your thoughts on how close we are to transitioning Xopt to pydantic v2 (accepting this PR)? |
|
We are close. I have unfortunately confirmed that some of the design changes in v2 are not bugs, so serialization is unlikely to ever work in the 'intended' way with decorators. I implemented parallel methods for those. |
|
Tests now pass. I plan to test if orjson is still necessary, and then prepare for merge. |
|
Note that development notebooks were kept at v1, not sure if needed anymore. |
|
|
||
| DECODERS = {"torch.float32": torch.float32, "torch.float64": torch.float64} | ||
| MIN_INFERRED_NOISE_LEVEL = 1e-4 | ||
|
|
||
|
|
||
| class StandardModelConstructor(ModelConstructor): | ||
| name = "standard" | ||
| name: str = Field("standard", frozen=True) |
There was a problem hiding this comment.
this should probably be a class variable instead, if you could update this and the base constructor class that would be great
There was a problem hiding this comment.
although on second thought, maybe not since the name isn't dumped during serialization. What do you think?
There was a problem hiding this comment.
Yes, classvars not getting dumped was the issue so kept them as standard fields. In case of generator, where name is classvar, we manually add that key during dump. There is another option of having a hidden discriminator field but that can't be (directly) used because they are incompatible with pre-validators.
|
@ChristopherMayes do we need the developer folder in docs/examples? I have a separate dev folder that is gitignored |
We should keep the logger example somewhere. The others look outdated. |
|
Ok, also @nikitakuklev and @ChristopherMayes I will hold off on incorporating this until after 9/22 since we have some runtime for Badger on Thursday that I want to keep xopt stable for |
|
@nikitakuklev so it looks like the examples are failing due to an update in matplotlib that I fixed in PR #147 . If you want to merge those changes into your branch then we can accept this PR |
|
@roussel-ryan ready to merge |
This PR begins work on pydantic v2 migration.
In general, documentation for v2 is still lacking, and many changes are not called out. On first pass, several issues were encountered due to changes in pydantic internals (v2 uses Rust core, has native json implementation, and thus less information is accessible/customizable). Namely:
type_attribute to acquire resolved inner type seems to have no direct replacement. Workaround with typing works ok for simple case of Optional[X], but requires further testing.**dumps_kwargs- need to rework savingNext step is to explore breaking out field serialization logic to take advantage of per-field decorators.