Feat!: Dev-only VDE mode #5087
Feat!: Dev-only VDE mode #5087
Conversation
izeigerman
force-pushed
the
feat-non-virtual-prod
branch
from
2a32f29 to
afcef50
Compare
| apply_effective_from = effective_from_ts > 0 and self.identifier != other.identifier | ||
| for start, end in other.intervals: | ||
| # If the effective_from is set, then intervals that come after it must come from | ||
| # the current snapshost. |
There was a problem hiding this comment.
(historical) nit: snapshost
sqlmesh/core/snapshot/definition.py
Outdated
| @@ -1013,15 +1035,18 @@ def categorize_as(self, category: SnapshotChangeCategory) -> None: | |||
| category: The change category to assign to this snapshot. | |||
| """ | |||
| self.dev_version_ = self.fingerprint.to_version() | |||
| reuse_previous_version = category in ( | |||
| is_no_rebuild = category in ( | |||
There was a problem hiding this comment.
Is this logic duplicated here and in the self.is_no_rebuild() property?
As a sidenote - thanks for renaming this and _should_force_breaking_change, the new names are much clearer as to the intent compared to the original names
There was a problem hiding this comment.
Is this logic duplicated here and in the self.is_no_rebuild() property?
yes, but only because _should_force_breaking_change relies on self.change_category which hasn't been set yet at this point.
There was a problem hiding this comment.
Oh, I was more thinking along the lines of adding a constant something like NON_REBUILDABLE_CHANGE_CATEGORIES={SnapshotChangeCategory.FORWARD_ONLY, ...} and then referencing it in both places rather than maintaining two identical lists
izeigerman
force-pushed
the
feat-non-virtual-prod
branch
from
4971cc6 to
caddf38
Compare
izeigerman
force-pushed
the
feat-non-virtual-prod
branch
from
caddf38 to
b268cea
Compare
This PR adds support for the
dev_onlyVirtual Data Environment (VDE) mode.In this mode VDE is applied only in development environments, while in production, model tables and views are updated directly, bypassing the virtual layer. This also means that physical tables in production will be created using the original, unversioned model names. Users will continue to benefit from VDE and data reuse across development environments.
The new mode can be enabled in the project configuration:
Switching the mode for an existing project will result in a complete rebuild of all models in the project.