[query] Remove jackson string length restriction in Spark/Local backends #14651
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ehigham
requested changes
Aug 2, 2024
chrisvittal
force-pushed
the
query/fix-other-stringlength-constraints
branch
from
9e89f35
to
fd10c0b
Compare
|
@ehigham Think I've addressed everything. |
ehigham
approved these changes
Aug 5, 2024
patrick-schultz
requested changes
Aug 7, 2024
There was a problem hiding this comment.
I'm a bit concerned about the replication, and in the future making a change and forgetting to apply it in all three places. Instead of doing this in the apply methods for each backend (and main for the service backend), could we move it to the constructor of the Backend superclass?
patrick-schultz
approved these changes
Aug 7, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As mentioned in #14580, IR can get quite big, especially as it can contain an arbitrary amount of encoded literals from the user's python session.
Tested manually, by making a very very large literal, running a pipeline with it on 0.2.132, observing the failure seen in #14650, then running the same pipeline with this change, and it succeeds as normal.
Resolves #14650