Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade jackson version #1739

Merged
merged 1 commit into from
Jul 22, 2024
Merged

Conversation

Polber
Copy link
Contributor

@Polber Polber commented Jul 18, 2024

Apache Beam depends on Jackson 2.15.4 as of apache/beam#31473 which is breaking at least the JMS To PubSub template when deserializing pipeline options due to

java.lang.NoSuchFieldError: READ_UNKNOWN_ENUM_VALUES_USING_DEFAULT_VALUE
  at com.fasterxml.jackson.databind.deser.std.EnumDeserializer.createContextual(EnumDeserializer.java:211)
  ...
  at org.apache.beam.sdk.options.PipelineOptionsFactory.computeDeserializerForMethod(PipelineOptionsFactory.java:1739)
  ...

which was a field added in 2.15.4

@Polber Polber requested review from Abacn and fozzie15 July 18, 2024 23:29
@Polber Polber self-assigned this Jul 18, 2024
Abacn
Abacn previously approved these changes Jul 18, 2024
@Polber Polber marked this pull request as draft July 18, 2024 23:32
@Polber
Copy link
Contributor Author

Polber commented Jul 18, 2024

Not sure why checks aren't running. Looking into it

@Polber
Copy link
Contributor Author

Polber commented Jul 18, 2024

Might be this https://www.githubstatus.com/

@Abacn
Copy link
Contributor

Abacn commented Jul 18, 2024

wait, at least need to add this line in several places per #1681

RowJsonUtils.increaseDefaultStreamReadConstraints(100 * 1024 * 1024);

@Polber
Copy link
Contributor Author

Polber commented Jul 18, 2024

Job is queued: https://github.com/GoogleCloudPlatform/DataflowTemplates/actions/runs/10000088771
Definitely a github issue. Don't love that it doesn't block submission

Signed-off-by: Jeffrey Kinard <jeff@thekinards.com>
@Polber Polber marked this pull request as ready for review July 19, 2024 19:12
@Abacn
Copy link
Contributor

Abacn commented Jul 19, 2024

there will be integration test failures which I observed during 2.57.0rc1 validation, need to fix some code path, see #1739 (comment)

Copy link

codecov bot commented Jul 19, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 42.31%. Comparing base (d7db191) to head (84d72c6).

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #1739      +/-   ##
============================================
- Coverage     42.32%   42.31%   -0.01%     
+ Complexity     3182     3180       -2     
============================================
  Files           794      794              
  Lines         46244    46244              
  Branches       4951     4951              
============================================
- Hits          19572    19568       -4     
- Misses        25074    25077       +3     
- Partials       1598     1599       +1     
Components Coverage Δ
spanner-templates 63.69% <ø> (-0.01%) ⬇️
spanner-import-export 64.42% <ø> (-0.03%) ⬇️
spanner-live-forward-migration 74.97% <ø> (ø)
spanner-live-reverse-replication 51.84% <ø> (ø)
spanner-bulk-migration 82.79% <ø> (ø)

see 2 files with indirect coverage changes

@Polber Polber requested a review from Abacn July 22, 2024 16:58
@Polber
Copy link
Contributor Author

Polber commented Jul 22, 2024

there will be integration test failures which I observed during 2.57.0rc1 validation, need to fix some code path, see #1739 (comment)

@Abacn I already included these changes when I upgraded beam in #1695.

IT's passed other than a couple Kafka IT's which have already been failing - I'm looking into those, but error is unrelated to the jackson upgrade.

@Polber Polber merged commit 5be5817 into GoogleCloudPlatform:main Jul 22, 2024
11 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants