Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bump azure core dependency to mitigate woodstox CVE #15432

Closed
wants to merge 38 commits into from

Conversation

tomrijntjes
Copy link

@tomrijntjes tomrijntjes commented Dec 16, 2022

Description

Bump azure-core dependency to address woodstox CVE.

Additional context and related issues

Two plugins rely on azure-core:

  • trino-delta-lake
  • trino-exchange-filesystem

Version 1.25 depends on Woodstox 6.2.6 which has a known CVE that showed up in our vulnerability scan. This CVE has been mitigated in azure-core 1.34 and its transitive dependency woodstox 6.4.0

Release notes

(x) This is not user-visible or docs only and no release notes are required.
( ) Release notes are required, please propose a release note for me.
( ) Release notes are required, with the following suggested text:

dain and others added 30 commits December 11, 2022 17:32
When refresh token is retrieved for UI, currently we were sending
HTTP Status 303, assuming that all the request will just repeat the
call on the Location header. When this works for GET/PUT verbs, it does
not for non-idempotent ones like POST, as every js http client should
do a GET on LOCATION after 303 on POST. Due to that I change it to 307, that
should force every client to repeat exactly the same request,
no matter the verb.

Co-authored-by: s2lomon <s2lomon@gmail.com>
Actual work is done in `pageProjectWork.process()` call while
`projection.project` only performs setup of projection.
So both `expressionProfiler` and `metrics.recordProjectionTime`
needed to be around that method.
Removes outdated comments and unnecessary methods in local exchange
PartitioningExchanger since the operator is no longer implemented
in a way that attempts to be thread-safe.
- Change ColumnHandle to BigQueryColumnHandle in BigQueryTableHandle
- Extract buildColumnHandles in BigQueryClient
The new field allows the table function to declare during
analysis which columns from the input tables are necessary to
execute the function.

The required columns can be then validated by the analyzer.
This declaration can be also used by the optimizer to prune
any input columns that are not used by the table function.
Change the way how DirectExchangeClient.scheduleRequestIfNecessary calculates
the number of clients to be requested on the exchange phase to use an average
request size of specific client instead of aggregated average of all clients.
According to https://www.rfc-editor.org/rfc/rfc6749#section-4.1.3 scope
parameter in the get token request is actually redundant as it was
already provided in the authorization request. Refresh token request on
the other hand should still provide it.
mx123 and others added 7 commits December 16, 2022 07:49
The Paruqet reader does not support pushdown on fields of a Row type.
The checks in `IcebergPageSourceProvider#getParquetTupleDomain` used
to prevent this, but they stopped working when dereference pushdown
was implemented.

If a row field had the same name as a top level column this would
have resulted in a correctness issue.
Add syntax for defining how stale an MV can be and still be queryable.
Example:

 Fragment 2 [SOURCE]
     CPU: 618.74ms, Scheduled: 1.23s, Blocked 2.95s (Input: 0.00ns, Output: 0.00ns), Input: 6001215 rows (51.51MB); per task: avg.: 6001215.00 std.dev.: 0.00, Output: 3 row
     Output buffer active time: 5.66ms, buffer utilization distribution (%): {p01=0.00, p05=0.00, p10=0.00, p25=0.00, p50=0.00, p75=0.00, p90=0.00, p95=0.00, p99=0.00, max=
     Task output distribution: {count=1.00, p01=12.87MB, p05=12.87MB, p10=12.87MB, p25=12.87MB, p50=12.87MB, p75=12.87MB, p90=12.87MB, p95=12.87MB, p99=12.87MB, max=12.87MB
     Task input distribution: {count=1.00, p01=12.87MB, p05=12.87MB, p10=12.87MB, p25=12.87MB, p50=12.87MB, p75=12.87MB, p90=12.87MB, p95=12.87MB, p99=12.87MB, max=12.87MB}
Example:

 Fragment 1 [HASH]
     Amount of input data processed by the workers for this stage might be skewed
@cla-bot cla-bot bot added the cla-signed label Dec 16, 2022
@ebyhr
Copy link
Member

ebyhr commented Dec 16, 2022

Please take a look at CI failure https://github.com/trinodb/trino/actions/runs/3712280909/jobs/6293641194.

@findepi
Copy link
Member

findepi commented Dec 19, 2022

The build seems not green. I think this is the reason

Warning:  Rule 1: org.apache.maven.plugins.enforcer.RequireUpperBoundDeps failed with message:
Failed while enforcing RequireUpperBoundDeps. The error(s) are [
Require upper bound dependencies error for com.fasterxml.jackson.core:jackson-databind:2.13.4 paths to dependency are:
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-com.fasterxml.jackson.core:jackson-databind:2.13.4
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-io.trino:trino-hive:404-SNAPSHOT
    +-com.fasterxml.jackson.core:jackson-databind:2.13.4 (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.13.4
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-io.trino:trino-plugin-toolkit:404-SNAPSHOT
    +-com.fasterxml.jackson.core:jackson-databind:2.13.4 (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.13.4
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-io.airlift:event:219
    +-com.fasterxml.jackson.core:jackson-databind:2.13.4 (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.13.4
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-io.airlift:json:219
    +-com.fasterxml.jackson.core:jackson-databind:2.13.4 (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.13.4
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-com.amazonaws:aws-java-sdk-core:1.12.261
    +-com.fasterxml.jackson.core:jackson-databind:2.13.4 (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.12.6.1
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-io.trino:trino-hive:404-SNAPSHOT:tests [test]
    +-com.fasterxml.jackson.core:jackson-databind:2.13.4 [test] (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.13.4 [test]
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-io.trino:trino-jmx:404-SNAPSHOT [test]
    +-com.fasterxml.jackson.core:jackson-databind:2.13.4 [test] (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.13.4 [test]
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-io.trino:trino-main:404-SNAPSHOT [test]
    +-com.fasterxml.jackson.core:jackson-databind:2.13.4 [test] (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.13.4 [test]
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-io.trino:trino-main:404-SNAPSHOT:tests [test]
    +-com.fasterxml.jackson.core:jackson-databind:2.13.4 [test] (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.13.4 [test]
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-io.trino:trino-tpcds:404-SNAPSHOT [test]
    +-com.fasterxml.jackson.core:jackson-databind:2.13.4 [test] (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.13.4 [test]
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-io.trino:trino-tpch:404-SNAPSHOT [test]
    +-com.fasterxml.jackson.core:jackson-databind:2.13.4 [test] (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.13.4 [test]
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-com.azure:azure-core:1.34.0 [test]
    +-com.fasterxml.jackson.core:jackson-databind:2.13.4 [test] (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.13.4.2 [test]
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-io.airlift:json:219
    +-com.fasterxml.jackson.datatype:jackson-datatype-jdk8:2.13.4 (managed) <-- com.fasterxml.jackson.datatype:jackson-datatype-jdk8:2.13.4
      +-com.fasterxml.jackson.core:jackson-databind:2.13.4 (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.13.4
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-io.airlift:json:219
    +-com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.13.4 (managed) <-- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.13.4
      +-com.fasterxml.jackson.core:jackson-databind:2.13.4 (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.13.4
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-io.airlift:json:219
    +-com.fasterxml.jackson.datatype:jackson-datatype-guava:2.13.4 (managed) <-- com.fasterxml.jackson.datatype:jackson-datatype-guava:2.13.4
      +-com.fasterxml.jackson.core:jackson-databind:2.13.4 (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.13.4
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-io.airlift:json:219
    +-com.fasterxml.jackson.datatype:jackson-datatype-joda:2.13.4 (managed) <-- com.fasterxml.jackson.datatype:jackson-datatype-joda:2.13.4
      +-com.fasterxml.jackson.core:jackson-databind:2.13.4 (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.13.4
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-io.airlift:json:219
    +-com.fasterxml.jackson.module:jackson-module-parameter-names:2.13.4 (managed) <-- com.fasterxml.jackson.module:jackson-module-parameter-names:2.13.4
      +-com.fasterxml.jackson.core:jackson-databind:2.13.4 (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.13.4
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-com.amazonaws:aws-java-sdk-core:1.12.261
    +-com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:2.13.4 (managed) <-- com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:2.12.6
      +-com.fasterxml.jackson.core:jackson-databind:2.13.4 (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.13.4
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-com.amazonaws:aws-java-sdk-glue:1.12.261
    +-com.amazonaws:jmespath-java:1.12.261
      +-com.fasterxml.jackson.core:jackson-databind:2.13.4 (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.12.6.1
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-io.trino:trino-main:404-SNAPSHOT [test]
    +-io.trino:trino-client:404-SNAPSHOT [test] (managed) <-- io.trino:trino-client:404-SNAPSHOT [test]
      +-com.fasterxml.jackson.core:jackson-databind:2.13.4 [test] (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.13.4 [test]
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-io.trino:trino-main:404-SNAPSHOT [test]
    +-io.trino:trino-geospatial-toolkit:404-SNAPSHOT [test] (managed) <-- io.trino:trino-geospatial-toolkit:404-SNAPSHOT [test]
      +-com.fasterxml.jackson.core:jackson-databind:2.13.4 [test] (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.13.4 [test]
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-io.trino:trino-main:404-SNAPSHOT [test]
    +-io.airlift:jaxrs:219 [test] (managed) <-- io.airlift:jaxrs:219 [test]
      +-com.fasterxml.jackson.core:jackson-databind:2.13.4 [test] (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.13.4 [test]
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-io.trino:trino-main:404-SNAPSHOT [test]
    +-io.airlift:jmx-http:219 [test] (managed) <-- io.airlift:jmx-http:219 [test]
      +-com.fasterxml.jackson.core:jackson-databind:2.13.4 [test] (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.13.4 [test]
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-io.trino:trino-main:404-SNAPSHOT [test]
    +-io.airlift.discovery:discovery-server:1.30 [test] (managed) <-- io.airlift.discovery:discovery-server:1.30 [test]
      +-com.fasterxml.jackson.core:jackson-databind:2.13.4 [test] (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.10.0 [test]
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-io.trino:trino-main:404-SNAPSHOT [test]
    +-io.jsonwebtoken:jjwt-jackson:0.11.2 [test] (managed) <-- io.jsonwebtoken:jjwt-jackson:0.11.2 [test]
      +-com.fasterxml.jackson.core:jackson-databind:2.13.4 [test] (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.9.10.4 [test]
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-io.trino:trino-testing-containers:404-SNAPSHOT [test]
    +-io.minio:minio:7.1.4 [test] (managed) <-- io.minio:minio:7.1.4 [test]
      +-com.fasterxml.jackson.core:jackson-databind:2.13.4 [test] (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.10.3 [test]
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-com.azure:azure-core:1.34.0 [test]
    +-com.fasterxml.jackson.dataformat:jackson-dataformat-xml:2.13.4 [test]
      +-com.fasterxml.jackson.core:jackson-databind:2.13.4 [test] (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.13.4 [test]
and
+-io.trino:trino-delta-lake:404-SNAPSHOT
  +-io.trino:trino-main:404-SNAPSHOT [test]
    +-io.airlift:jaxrs:219 [test] (managed) <-- io.airlift:jaxrs:219 [test]
      +-com.fasterxml.jackson.dataformat:jackson-dataformat-smile:2.13.4 [test] (managed) <-- com.fasterxml.jackson.dataformat:jackson-dataformat-smile:2.13.4 [test]
        +-com.fasterxml.jackson.core:jackson-databind:2.13.4 [test] (managed) <-- com.fasterxml.jackson.core:jackson-databind:2.13.4 [test]
]

@findepi
Copy link
Member

findepi commented Dec 19, 2022

@tomrijntjes there was apparently a force push to the master branch (per #15365 (comment)), so your PR appears as if containing some unrelated commits.
Can you please rebase to remove them?

After you do that, please make sure the CI passes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.