Add release notes for 0.289

[tdcmeehan]

Missing Release Notes

Abe Varghese

• [native] Enhance CSV Parsing in ContainerQueryRunner to Use MaterializedResult #23382 [native] Enhance CSV Parsing in ContainerQueryRunner to Use MaterializedResult (Merged by: Timothy Meehan)

Amit Dutta

• [native] Advance velox. #23263 [native] Advance velox. (Merged by: Amit Dutta)
• [native] Fix crash in httpclient when accessing null eventbase (follo… #23259 [native] Fix crash in httpclient when accessing null eventbase (follo… (Merged by: Ge Gao)
• [native] Fix crash in httpclient when accessing null eventbase #23258 [native] Fix crash in httpclient when accessing null eventbase (Merged by: Amit Dutta)
• [native] Advance velox. #23117 [native] Advance velox. (Merged by: Aditi Pandit)
• [native] Advance velox. #23036 [native] Advance velox. (Merged by: Aditi Pandit)
• [native] Advance velox. #23007 [native] Advance velox. (Merged by: Amit Dutta)

Auden Woolfson

• Add warning for IGNORE NULL with Window agggregates #23325 Add warning for IGNORE NULL with Window agggregates (Merged by: Auden Woolfson)

Bikramjeet Vig

• [native] Handle scan filter with NaN limits during plan conversion #23289 [native] Handle scan filter with NaN limits during plan conversion (Merged by: Amit Dutta)
• [native] Remove spark specific query config from BaseVeloxQueryConfig #23046 [native] Remove spark specific query config from BaseVeloxQueryConfig (Merged by: Ge Gao)
• [native] Include additional context from velox exception #23025 [native] Include additional context from velox exception (Merged by: Aditi Pandit)

Deepak Majeti

• [native] Add CMake options to enable GCS and ABFS #23290 [native] Add CMake options to enable GCS and ABFS (Merged by: Deepak Majeti)

Elliotte Rusty Harold

• Decouple from HiveErrorCode.HIVE_TOO_MANY_OPEN_PARTITIONS #23168 Decouple from HiveErrorCode.HIVE_TOO_MANY_OPEN_PARTITIONS (Merged by: Elliotte Rusty Harold)

Jialiang Tan

• [native] Add metrics for memory pushback mechanism #23422 [native] Add metrics for memory pushback mechanism (Merged by: Xiaoxuan)

Jimmy Lu

• [native] Fix HTTP client connection pool shutdown order #23321 [native] Fix HTTP client connection pool shutdown order (Merged by: Amit Dutta)
• [native] Fix exchange HTTP executors shutdown order #23286 [native] Fix exchange HTTP executors shutdown order (Merged by: Amit Dutta)
• [native] Implement bucket conversion for Hive splits #23028 [native] Implement bucket conversion for Hive splits (Merged by: Xiaoxuan)

Karteekmurthys

• [native] Expose REST API to fetch worker stats in Prometheus format #22360 [native] Expose REST API to fetch worker stats in Prometheus format (Merged by: Deepak Majeti)

Ke

• [native] Refactor TaskManagerTest to not create common::WriterOptions instance #23480 [native] Refactor TaskManagerTest to not create common::WriterOptions instance (Merged by: Ke)

Kevin Wilfong

• array_min/array_max throw if first element is an inner Array with a null #23323 array_min/array_max throw if first element is an inner Array with a null (Merged by: Xiaoxuan)

Miguel Blanco Godón

• Add support to be able to query new Delta protocols #22596 Add support to be able to query new Delta protocols (Merged by: Timothy Meehan)

Pratik Joseph Dabre

• Introduce a new CoordinatorPlugin interface for native clusters #23144 Introduce a new CoordinatorPlugin interface for native clusters (Merged by: Timothy Meehan)
• Add coordinator sidecar process #22854 Add coordinator sidecar process (Merged by: Timothy Meehan)
• [native] Add e2e tests for normalize scalar function #23035 [native] Add e2e tests for normalize scalar function (Merged by: Amit Dutta)

Yiqun (Ethan) Zhang

• 815012f Update .gitignore to include more IDE generated files

abhinavmuk04

• Support query retry for HBO #23147 Support query retry for HBO (Merged by: abhinavmuk04)
• Add a function for splitting array into slices of given length #23264 Add a function for splitting array into slices of given length (Merged by: abhinavmuk04)
• Treat Low Confidence, 0 estimations as unknown during joins #23047 Treat Low Confidence, 0 estimations as unknown during joins (Merged by: feilong-liu)
• Prioritize high confidence stats during broadcast joins #23016 Prioritize high confidence stats during broadcast joins (Merged by: abhinavmuk04)

aditi-pandit

• [Native] Add missing plumbing for Cte support #22780 [Native] Add missing plumbing for Cte support (Merged by: Aditi Pandit)
• [Native] Advance Velox #23078 [Native] Advance Velox (Merged by: Amit Dutta)

misterjpapa

• Prepend jdbc: to DatabaseMetaData.getURL result #23397 Prepend jdbc: to DatabaseMetaData.getURL result (Merged by: Timothy Meehan)

wypb

• Fix Iceberg read failing for Decimal type #23305 Fix Iceberg read failing for Decimal type (Merged by: Timothy Meehan)
• Add support for decimal batch reader #22636 Add support for decimal batch reader (Merged by: Ying)

Extracted Release Notes

• Add property to skip empty files in Hive Connector #22727 (Author: cvarelad-denodo): Add property to skip empty files in Hive Connector
  • Add support to skip empty files using configuration property hive.skip_empty_files :pr:22727.
• Pass full session to avoid Unknown connector errors #22936 (Author: Denodo Research Labs): Pass full session to avoid Unknown connector errors
  • Pass full session to avoid Unknown connector errors :pr:22936.
• Use distinct semantics for distincting array functions #22938 (Author: Rebecca Schlussel): Use distinct semantics for distincting array functions
  • Fix an issue where array_distinct,array_except, array_intersect, array_union, set_union, and set_agg would return an error if the input was an array of rows with null fields or array of arrays with null elements. These functions now use full "IS DISTINCT FROM" semantics for comparison.
• Fix serialization of aggregation node for HBO #22949 (Author: Feilong Liu): Fix serialization of aggregation node for HBO
  • • Fix serialization of aggregation node in HBO plan hash to output consistent hash :pr:22949 .
• [Verifier] Wrap checksum queries with partition predicate for reused output table #22965 (Author: Ge Gao): [Verifier] Wrap checksum queries with partition predicate for reused output table
  • Add control.reuse-table and test.reuse-table configuration properties for the Presto Verifier to reuse the output tables of the source query for control and test.
• Fix map_top_n to use keys to break ties on NULL values.  #22979 (Author: Krishna Pai): Fix map_top_n to use keys to break ties on NULL values.
  • Make map_top_n function more deterministic by using keys to break ties.
• Fix hash calculation for Timestamp in HiveBucketing to be Hive Compatible #22980 (Author: Ke): Fix hash calculation for Timestamp in HiveBucketing to be Hive Compatible
  • Fix hash calculation for Timestamp column to be hive compatible when writing to a table bucketed by Timestamp. :pr:22890.
  • Add config hive.legacy-timestamp-bucketing and session property hive.legacy_timestamp_bucketing to use the original hash function for Timestamp column, which is not hive compatible. :pr:22890.
• Fix wrong results bug with count over mixed aggregation #23013 (Author: Rebecca Schlussel): Fix wrong results bug with count over mixed aggregation
  • Fix a bug where count queries on top of aggregations with mixed global and grouped grouping sets could return wrong results.
• Update Joda-Time to 2.12.7 #23027 (Author: Amit Dutta): Update Joda-Time to 2.12.7
  • Update Joda-Time to 2.12.7 to use 2024a tzdata. Note: a corresponding update to the Java runtime should also be made to ensure consistent timezone data. For example, Oracle JDK 8u381, tzdata2024a rpm for OpenJDK, or use Timezone Updater Tool to apply 2024a tzdata to existing JVM. :pr:23027.
• Add analyzer warning for common NaN pitfalls #23059 (Author: Rebecca Schlussel): Add analyzer warning for common NaN pitfalls
  • Add configuration property warn-on-possible-nans and session property warn_on_possible_nans to produce a warning on division operations or comparison operations involving double or real types. Division operations are common causes of accidental creation of nans, and the semantics of comparison operations involving nans changed considerably in the most recent Presto release.
• Fix hash for double and real types when new nan is disabled #23060 (Author: Rebecca Schlussel): Fix hash for double and real types when new nan is disabled
  • Fix a bug in the hash code of -0 for double and real types when use-new-nan-definition is false :pr:23060.
• Remove cross join if one side of input is single row constant #23081 (Author: Feilong Liu): Remove cross join if one side of input is single row constant
  • Add an optimization to remove cross join when one side of inputs is single row of constant values. The optimization is controlled by session property remove_cross_join_with_constant_single_row_input :pr:23081.
• Upgrade json-path to 2.9.0 due CVE-2023-1370 #23104 (Author: Denodo Research Labs): Upgrade json-path to 2.9.0 due CVE-2023-1370
  • Upgrade json-path to 2.9.0 due CVE-2023-1370 :pr:23104.
• Upgrade guava to 32.1.0-jre due CVE-2023-2976 #23127 (Author: Denodo Research Labs): Upgrade guava to 32.1.0-jre due CVE-2023-2976
  • Upgrade guava to 32.1.0-jre due CVE-2023-2976 :pr:23127.
• Upgrade avro to 1.11.3 due CVE-2023-39410 #23142 (Author: Denodo Research Labs): Upgrade avro to 1.11.3 due CVE-2023-39410
  • Upgrade avro to 1.11.3 due CVE-2023-39410 :pr:23142.
• [Iceberg] Add table refresh retry configurations #23174 (Author: Zac Blanco): [Iceberg] Add table refresh retry configurations
  • The Iceberg connector when configured with Hive, can now configure table metadata refresh timeouts. See documentation for details.
• Use data size for directory listing cache #23176 (Author: Nikhil Collooru): Use data size for directory listing cache
  • Add support for setting the max size in bytes for directory listing cache. This can be set via the new hive.file-status-cache.max-retained-size configuration property. hive.file-status-cache-size is now deprecated :pr:23176.
• Fix Resource Group State Info #23178 (Author: Swapnil Tailor): Fix Resource Group State Info
  • Fix resource group state info endpoint to return correct value when caching is disabled. :pr:23178.
• [docs] Add DBeaver documentation to /clients/ #23189 (Author: Steve Burnett): [docs] Add DBeaver documentation to /clients/
  • ... Add :doc:/clients/dbeaver documentation :pr:23189.
• [native] Add support for cache periodic full persistence #23192 (Author: Zac Wen): [native] Add support for cache periodic full persistence
  • Add support for persisting full memory cache to SSD periodically on Presto C++ worker. This can be enabled by setting cache.velox.full-persistence-interval to a non-zero value. :pr:23192.
• [docs] Add Superset documentation to /clients/ #23194 (Author: Steve Burnett): [docs] Add Superset documentation to /clients/
  • ... Add :doc:/clients/superset documentation :pr:23194.
• Send periodic query progress events #23195 (Author: Arjun Gupta): Send periodic query progress events
  • Add a new configuration property event.query-progress-publish-interval to specify the time interval at which progress events should be generated. Default is every 1 minute. :pr:23195.
• Add plan validator for timestamp with timezone check for prestissimo #23200 (Author: Feilong Liu): Add plan validator for timestamp with timezone check for prestissimo
  • Add a plan checker so that queries having timestamp with timezone type will fail in prestissimo :pr:23200.
• Add session property to track latency breakdown for HBO optimizer #23241 (Author: Feilong Liu): Add session property to track latency breakdown for HBO optimizer
  • Add a session property enable_verbose_history_based_optimizer_runtime_stats to track latency of HBO optimizer :pr:23241.
• Add iceberg table properties for maintenance of previous metadata files #23260 (Author: wangd): Add iceberg table properties for maintenance of previous metadata files
  • Add table properties metadata_previous_versions_max and metadata_delete_after_commit to maintain the previous metadata files :pr:23260.
  • Enable Iceberg with hive catalog to support deleting old metadata files after commit based on the table properties :pr:23260.
• [Iceberg] Support procedure remove_orphan_files #23267 (Author: wangd): [Iceberg] Support procedure remove_orphan_files
  • Add procedure remove_orphan_files to remove orphan files that are not referenced in any metadata files for Iceberg. :pr:23267.
• Allow Verifier to validate varchar as floating point #23312 (Author: Ge Gao): Allow Verifier to validate varchar as floating point
  • Add verifier config --validate-string-as-double to control applying floating point validation to the column composed of varchar, if it is detected that the varchar column is derived from casting floating points. :pr:23312.
• Fix assignments for equijoin on TimestampWithTimeZone #23319 (Author: Tim Meehan): Fix assignments for equijoin on TimestampWithTimeZone
  • Fix equijoin over timestamp with timezone types (:pr:23319).
  • Add equalValuesAreIdentical to Type. Override this method to return false when the values of the type may have more than one canonical representation.
• Add session property to enforce timeout for query registration in HBO #23354 (Author: Feilong Liu): Add session property to enforce timeout for query registration in HBO
  • Add session property enforce_history_based_optimizer_register_timeout to enforce the maximum time HBO query registration can take :pr:23354 .
• Upgrade Airlift to 0.215 #23356 (Author: Nikhil Collooru): Upgrade Airlift to 0.215
  • Upgrade Airlift to 0.215 :pr:23356.
• Switch default Parquet writer version to PARQUET_1_0 #23369 (Author: Deepak Majeti): Switch default Parquet writer version to PARQUET_1_0
  • Switch default Parquet writer version to PARQUET_1_0. Please set the hive config hive.parquet.writer.version = PARQUET_2_0 for old behavior. :pr:23369.
• Move timezone with timestamp plan checker to intermediate plan stage #23374 (Author: Feilong Liu): Move timezone with timestamp plan checker to intermediate plan stage
  • Improve the plan check for timestamp with timezone type to cover filter pushdown :pr:12345.
• Fix like matching with newlines and no wildcard #23404 (Author: Rebecca Schlussel): Fix like matching with newlines and no wildcard
  • Fix a bug where like predicates would only match to a newline when there was no wildcard at the end. :pr:23404.
• Apply info column filters during split generation #23411 (Author: Mahadevuni Naveen Kumar): Apply info column filters during split generation
  • Fix filtering by info columns $file_size and $file_modified_time, which were ignored before. :pr:23411.
• Function to aggregate IP prefixes #23445 (Author: Matt Calder): Function to aggregate IP prefixes
  • Add function :func:ip_prefix_collapse ... :pr:23445.
• Improve correctness of stddev and variance with partial aggregation #23447 (Author: Rebecca Schlussel): Improve correctness of stddev and variance with partial aggregation
  • Improve stddev and variance functions to always return correct results when input is constant. :pr:23447.
• Fix failure with enabling JDBC autocommit #23453 (Author: Rebecca Schlussel): Fix failure with enabling JDBC autocommit
  • Fix failure when setting autoCommit from false to true :pr:23453.

All Commits

• b2974a7 [native] Advance velox. (Amit Dutta)
• 797019c [native] Refactor TaskManagerTest to not create common::WriterOptions instance (Ke)
• 67e423a [Iceberg]Fix cleanup files behavior on expiring snapshots (wangd)
• 1ab72db [native] Fix docker-compose and add etc config into native worker docker image (wypb)
• 8106613 Fix analyze table whose column name ends with _<number> (wangd)
• d69f1f8 Refactor dead links in the documents (Reetika Agrawal)
• 171e57d Fix sql block in Iceberg document (Reetika Agrawal)
• 73f58cc [native] Minor renames and spell fixes. (Amit Dutta)
• 14da6b7 [native] Advance velox. (Amit Dutta)
• 7cb9526 [native] Add a sample plan validator and e2e tests. (Amit Dutta)
• 6530afc [native] Fix Velox connector configs update (Zac Wen)
• a868469 Init w/ a random UUID (Zuyu ZHANG)
• 0e6d3ae [native] Init UuidParse in TaskManager due to boost UUID non-POD changes (Zuyu ZHANG)
• dec6c14 [native] Prestissimo worker metrics documentation (Karteekmurthys)
• 61a01f4 initial commit for ip_prefix_collapse function (Matt Calder)
• a37fea2 Prefer assertSame (Elliotte Rusty Harold)
• fdd2059 [native] Retrieve presto native session properties (Pramod Satya)
• 56223eb Aggressively prune statistics from query info after completion (Zac Blanco)
• c4e1ab3 [native] Advance velox. (Amit Dutta)
• a707af8 Fix a build link in README.md (zuyu)
• ae87032 [native] Add new shared arbitrator configs (Jialiang Tan)
• b4eca42 [native] Add new arbitration configs (Jialiang Tan)
• 34e73df Reenabling recently disabled E2E tests. (Sergey Pershin)
• c75087f [native] Add Velox factory registrations (Deepak Majeti)
• 0f7aa2e [native] Fill protocol::TaskStats::completedDrivers with finished splits (Jialiang Tan)
• bd444ec Fix failure with enabling JDBC autocommit (Rebecca Schlussel)
• 4dab583 [native] Comment out failing tests temporarely. (Sergey Pershin)
• ddc33d6 Move QueuedStatementResource helpers to QueryResourceUtil (prithvip)
• 6072d37 Separate ExecutingStatement response from QueuedStatement (prithvip)
• 0c273df Separate QueryPreparer from QueryAnalyzer (prithvip)
• bdf89f9 Extract interface from FailedDispatchQueryFactory (prithvip)
• 330fa47 array_min/array_max throw if first element is an inner Array with a null element (Kevin Wilfong)
• be29974 [native]Advance velox version (xiaoxmeng)
• 0d0c2ea Make hashCode consistent with equals (Elliotte Rusty Harold)
• 4f74f52 Improve correctness of stddev and variance with partial aggregation (Rebecca Schlussel)
• c969f30 Update airlift to 0.215 (Nikhil Collooru)
• 7cf06c3 [native] Update gtest targets (Zuyu ZHANG)
• d3dd60a Fix formatting in rest/node.rst and statement.rst (Steve Burnett)
• c8a4c55 Apply info column filters during split generation (Mahadevuni Naveen Kumar)
• 78a8d18 Use assertEquals instead of assertTrue when testing equality (Elliotte Rusty Harold)
• c624968 [native] Cleanup dependencies from old velox::core::Config (Jialiang Tan)
• 868e721 [native] Advance velox. (Amit Dutta)
• fbc0af3 Clean up language (Elliotte Rusty Harold)
• cef7fc6 Apply suggestions from code review (Linsong Wang)
• ee89abe Minor changes in Iceberg Connector Docs (Jalpreet Singh Nanda (:imjalpreet))
• 045a80c [native] Fix Meta internal build caused by folly::StringPiece (Jialiang Tan)
• 03e5e50 [native] Advance velox. (Amit Dutta)
• 4b920ca [native] Add hook for Velox plan validation. (Amit Dutta)
• 64219fb Clean up exception handling and avoid shadowing mutable field (Elliotte Rusty Harold)
• 28e84a0 Refine the CircleCI criteria (Yihong Wang)
• fba1b89 [native] Advance velox (Amit Dutta)
• e782f02 [native] Add metrics for memory pushback mechanism (Jialiang Tan)
• 9db7e1b Fix like matching with newlines and no wildcard (Rebecca Schlussel)
• 4e6678c Fix formatting in admin/verifier.rst (Steve Burnett)
• 6c8fd1f Fix issue with identifying column nullability (Ramesh Kanna S)
• 88ef79d [native] Enhance CSV Parsing in ContainerQueryRunner to Use MaterializedResult (Abe Varghese)
• 54dbe56 [native] Switch to use new config base class from Velox (Jialiang Tan)
• 2855762 [native] Advance velox. (Amit Dutta)
• 5e75ca7 [native] Advance velox. (Amit Dutta)
• d512070 [native] Advance velox. (Amit Dutta)
• 23477ad Warn when a non-window value function is specified to ignore nulls (Auden Woolfson)
• c61eade [native] Fix node select condition for cache retention (Zac Wen)
• 6d55f58 [iceberg] Refactor IcebergWritableTableHandle's schema and partitionSpec fields (Jalpreet Singh Nanda (:imjalpreet))
• 6566e7f [iceberg] Add wrapper class PrestoIcebergPartitionSpec for org.apache.iceberg.PartitionSpec (Jalpreet Singh Nanda (:imjalpreet))
• 31e8151 [iceberg] Add wrapper classes for org.apache.iceberg.Schema and Types.NestedField (Jalpreet Singh Nanda (:imjalpreet))
• fedb846 [native] Disable cache retention with NO_PREFERENCE node selection (Zac Wen)
• 39e74cc Add session property to enforce timeout for query register in HBO optimizer (Feilong Liu)
• f8f4a83 Add presto-openapi module and QueryResource definition (ymmarissa)
• 9ce424c Send periodic query progress events (Arjun Gupta)
• 6e4896d Prepend jdbc: to DatabaseMetaData.getURL result (misterjpapa)
• fb100f3 [native] Add e2e tests for corelated subqueries. (Amit Dutta)
• 83a82cd [native] Advance velox. (Amit Dutta)
• 937e863 Allow Verifier to validate varchar as floating point (Ge Gao)
• 768efa3 Upgrade avro to 1.11.3 due CVE-2023-39410 (Denodo Research Labs)
• 31aad37 Improve CTE materialization execution tests (Christian Zentgraf)
• 71addc7 Add array_split_into_chunks query tests (abhinavmuk04)
• 46f9545 Switch default Parquet writer version to PARQUET_1_0 (Deepak Majeti)
• ce85e97 Move timezone with timestamp check to intermediate stage (Feilong Liu)
• 45a52a7 Introduce a new CoordinatorPlugin interface for native clusters (Pratik Joseph Dabre)
• 62c5791 [native] Add support for running functional tests on Mac M1/ARM64. (Abe Varghese)
• 1fb7e92 Fix assignments for equijoin on TimestampWithTimeZone (Tim Meehan)
• 580988b [native] Advance velox (Amit Dutta)
• b94cca5 [Native] Disable recording http.client.presto_exchange_source.on_body_bytes metric (Karteekmurthys)
• 106cc4b Support query retry for HBO (abhinavmuk04)
• 27a3384 Fix formatting for map_key_exists in functions/map.rst (Steve Burnett)
• 4f5c431 Update docker compose scripts for docker compose v2 (Rebecca Schlussel)
• d41ba1c Clean up arraysql functions tests (abhinavmuk04)
• 052c5d9 Fix a typo of windowFunctions (Zac Wen)
• d2efd89 [Native] Keep presto_protocol files consistent (Abhisek Saikia)
• 8e95218 Fix error message for invalid real cast (Rebecca Schlussel)
• 97eed74 Provide info about new HBO retry feature (abhinavmuk04)
• 1e5d099 Include relevant retry info in hbo rst (abhinavmuk04)
• c2290c7 [native] Add AbstractTestNativeTpchConnectorQueries (Joe Abraham)
• 438b3b5 [native] Add initial basic native container test cases (Joe Abraham)
• 7f01d04 [native] Create ContainerQueryRunner and utility class (Joe Abraham)
• eebcb25 [native] Added functional testing framework using TestContainers (Abe Varghese)
• 7640fe6 [Iceberg]Fix tests failure caused by concurrently handle the same table (wangd)
• 90d1c0d Refractor Throwable Catch (abhinavmuk04)
• 703e739 [native] Advance velox. (Amit Dutta)
• 5f76bb5 Add query tests for array_sum function (abhinavmuk04)
• 78c41ff Fix round for input out of long range (Zac Wen)
• 39d419e [Native] Add runtime metric for bytes processed (Jacob Khaliqi)
• 86fc085 [Iceberg] Support procedure remove_orphan_files (wangd)
• 2384402 Add support to read uppercase partition columns (Miguel Blanco Godón)
• 2dc0a94 Add support to be able to query new Delta protocols (Miguel Blanco Godón)
• 1289eb0 [native] Fix build break for file not found in Utils.cpp (Christian Zentgraf)
• c9d1e8a Fix cross join with constant input optimizer (Feilong Liu)
• 772213c Add additional type support for KLL Sketches (Zac Blanco)
• 9d38c40 Fix Iceberg read failing for Decimal type (wypb)
• 5e9baaf [native] Fix HTTP client connection pool shutdown order (Jimmy Lu)
• 980ca70 [native] Switch to Velox based signal handler for Linux (Christian Zentgraf)
• 28d7111 Support delete old metadata files after commit for HiveTableOperations (wangd)
• 21ee3ae Add iceberg table properties for maintenance of previous metadata files (wangd)
• a1152bf Disable cross join with constant input optimizer (Feilong Liu)
• cf79ae9 Fix for Pull request 23127 (Devesh Agrawal)
• 0467afb Add additional scheduler instrumentation (Andrii Rosa)
• 03a2bd8 Fix explain plan to include materialized info for CTEs when verbose optimizer is enabled (jaystarshot)
• 72e771b Fixing NAN in coordinator UI on load (Anil Gupta Somisetty)
• 64eedc3 Provide various updates with new properties to optimizer (abhinavmuk04)
• 8c4fe23 Add documentation on temp table compression codec (jaystarshot)
• 9a3c695 Improve clarity of function behavior and potential usecases (abhinavmuk04)
• 0a4f2c5 [native] Advance velox. (Amit Dutta)
• 506fee2 [native] Add CMake options for GCS and ABFS (Deepak Majeti)
• e63a57a [native] Handle scan filter with NaN limits during plan conversion (Bikramjeet Vig)
• 31d3bd4 Fix CODEOWNERS (Timothy Meehan)
• 578aec4 Upgrade guava to 32.1.0-jre due CVE-2023-2976 (Denodo Research Labs)
• dd86e0c Upgrade json-path to 2.9.0 due CVE-2023-1370 (Denodo Research Labs)
• 1caebcc Create and test array split udf (abhinavmuk04)
• ca11b2f [native] Fix exchange HTTP executors shutdown order (Jimmy Lu)
• 9d55bda [native] Advance velox. (Amit Dutta)
• 9a364cd Add worker type to query context (Zac Wen)
• 51d3f69 Add support for cache periodic full persistence (Zac Wen)
• e884a44 Add session property to track latency breakdown for HBO optimizer (Feilong Liu)
• 06e7d50 [native] Advance velox. (Amit Dutta)
• 30c4804 fix thrift remote funciton (Wills Feng)
• 7bd2b82 [Docs] Fix typo in iceberg configuration property iceberg.file-format (wangd)
• c24aab3 Remove some debugging printlns from tests (Elliotte Rusty Harold)
• 08eb057 Resolved performance problem with the grammar. (Emanuel F.)
• d2d8642 Add Sql-invoked function support for column statistics (Zac Blanco)
• 00646ac [native] Fix crash in httpclient when accessing null eventbase (follow up) (Amit Dutta)
• 2fdbc69 [native] Fix crash in httpclient when accessing null eventbase (Amit Dutta)
• 632eb58 [Verifier] Wrap checksum queries with partition predicate for reused output table (Ge Gao)
• 753e8e1 [iceberg] Add IcebergInsertTableHandle and IcebergOutputTableHandle (Jalpreet Singh Nanda (:imjalpreet))
• 637024f [iceberg] Add compression codec in IcebergWritableTableHandle (Jalpreet Singh Nanda (:imjalpreet))
• 72b7689 Do not capitalize version and environment on web UI (Andrii Rosa)
• 96c0361 Update exchange protocol documentation (Andrii Rosa)
• 0521740 Use data size for directory listing cache (Nikhil Collooru)
• 1a08d34 Use path string as cache key instead of Path (Nikhil Collooru)
• 26d6738 [native] Ensure inheritance of fbos version for proxygen (Christian Zentgraf)
• d292e13 Fix bug when property exist in both connector,catalog session properties (Nikhil Collooru)
• a38da7a [native] Advance velox. (Amit Dutta)
• e789515 Add cost calculation for cte reference node (jaystarshot)
• 538fb0e Fix cte consumer cost calculation (jaystarshot)
• b3746ef More precise language (Elliotte Rusty Harold)
• 19bb370 update user info with geospatial type (abhinavmuk04)
• 904f9a1 [native] Update presto protocol for adding ConfidenceLevel in PlanNodeStatsEstimate (Jalpreet Singh Nanda (:imjalpreet))
• 90c27cf [native] Fix path for CacheQuotaRequirement and CacheQuotaScope java classes (Jalpreet Singh Nanda (:imjalpreet))
• 68c6fb9 Introduce feature config for low confidence and zero estimations (abhinavmuk04)
• 43b89cd Remove public from non-public classes (Elliotte Rusty Harold)
• bc8b6ff [native] Default to dummy reporter when prometheus is disabled (Karteekmurthys)
• 6fbfc31 [native] Add files for clang format and header checks from Velox (Deepak Majeti)
• 0a80251 Prefer more explicit assert methods (Elliotte Rusty Harold)
• 69d28c0 Fix docs links (Elliotte Rusty Harold)
• 66a632b [native] Fix driver count reporting from worker (Jialiang Tan)
• d3393a2 [native] Update clang check CI image (Deepak Majeti)
• 96711d6 Fix formatting in clients/superset.rst (Steve Burnett)
• 051c1d6 [native] Add support for Ubuntu aarch64 build on Mac M1 (Deepak Majeti)
• 991e508 Fix @VisibleForTesting method visibility (Ajay George)
• 6c7abcb [docs] Supplement the descriptions for configuration properties in HBO (wangd)
• bb9e2cd Add new join property to documentation (abhinavmuk04)
• 1c8b695 Treat low confidence, zero estimations as unknown during joins (abhinavmuk04)
• 3d25cd9 Fix doc links in READMEs (Steve Burnett)
• fd5ba48 Fix testcase name (Joe Abraham)
• 14e4fe5 Remove cross join if one side of input is single row constant (Feilong Liu)
• 6d6c8d2 Ensure connector specific session properties be valid in test framework (wangd)
• 3095fac Fix javadoc pointing to itself (Ajay George)
• b745e76 Fix multiple properties error handling (Ajay George)
• 3f778e9 Fix doc build error in release-0.288.rst (Steve Burnett)
• 6b77a1b Use specified strategy to get input table statistics (wangd)
• 4ca9f8b [native] Advance velox. (Amit Dutta)
• b693975 Add plan validator for timestamp with timezone check for prestissimo (Feilong Liu)
• 0e7a336 Return predicate columns in table statistics (Zac Blanco)
• ba08036 Pluralize parenthesis (Elliotte Rusty Harold)
• d5094e3 [native] Advance velox. (Amit Dutta)
• bfe68e6 Fix typo in LifespanAndStageStateTracker (wangd)
• 9cd54a0 Add Superset documentation to /clients/ (Steve Burnett)
• caa35d8 [Native] Add missing plumbing for CTE support (aditi-pandit)
• f3f060f [Iceberg] Add table refresh retry configurations (Zac Blanco)
• d76c2d3 Add support for decimal batch reader. (wypb)
• 66a7fc9 Update CODEOWNERS (Timothy Meehan)
• 960138f [native] Limit format and header checks to presto-native-execution (Deepak Majeti)
• ae788f3 Fix invalid limit clause (Ajay George)
• 8022abd Add DBeaver documentation to /clients/ (Steve Burnett)
• 852432d Fix Resource Group State Info (Swapnil Tailor)
• 9898886 Integrate config and session property for history matching threshold (wangd)
• 683dd61 Remove unused SecurityConfig from SystemSessionProperties (wangd)
• 8babf0a Decouple from com.facebook.presto.hive.HiveErrorCode.HIVE_TOO_MANY_OPEN_PARTITIONS (Elliotte Rusty Harold)
• 9c2d18a [native] Make ssd save behavior system config (Zac Wen)
• affcc94 [native] Advance Velox (Jialiang Tan)
• e0b76e6 Add more time for query runner to start (Elliotte Rusty Harold)
• 07d6f77 Use property to deactivate UI profile (Zac Blanco)
• 366a7fa Convert cast to substr when target is capped varchar (Ge Gao)
• bd47c2a Add confidence based broadcast to feature config (abhinavmuk04)
• cd68435 Fix View not found error classification (Ajay George)
• 6a0b194 Fix map_top_n to use keys to break ties on NULL values. Improve map_top_n to consistently break ties for null values  #22778 (Krishna Pai)
• b73ab7d [native]Add data cache related server operations (xiaoxmeng)
• 72f4af6 Improve the documentation for two memory counters (yingsu00)
• 01b70e0 [native] Revert coordinator and native process logs merging in e2e tests. (Amit Dutta)
• eb53161 Add positional deletes to Iceberg native tpcds e2e tests (Mahadevuni Naveen Kumar)
• 3a31ddc Fix bug in selective page source when row IDs are needed but row numbers aren't (Elliotte Rusty Harold)
• fd57488 [native] Expose REST API to fetch worker stats in Prometheus format (Karteekmurthys)
• 68c86e8 Let connector metadata existing in presto-main utilize newer SPI method (wangd)
• 8991db7 Remove json serialization annotation for TableWriterNode.UpdateTarget (wangd)
• 37a9cb3 [native] Minor cosmetic changes. (Amit Dutta)
• 6fe54ea Use buffered bytes in parquet written bytes metric (Zac Blanco)
• 27ece7e Pass full session to avoid Unknown connector errors (Denodo Research Labs)
• 7e199c8 Delete unused Docker class (Elliotte Rusty Harold)
• c9e3b06 [native] Advance velox. (Amit Dutta)
• 5b1cde7 Update Properties documentation with new Optimizer property (abhinavmuk04)
• 461a31a Prioritize high confidence stats during broadcast joins (abhinavmuk04)
• bccb13c Upgrade maven-shade-plugin to 3.6.0 (Elliotte Rusty Harold)
• 9ea981f Remvoe outdated workaround (Elliotte Rusty Harold)
• cd3968a Modify and sync DATA_DIR for IcebergQueryRunner (Reetika Agrawal)
• d022fdb Correct application name for HiveExternalWorkerQueryRunner (Reetika Agrawal)
• 3adf975 Enable ifNotExists flag while tpch copy in IcebergQueryRunner (Reetika Agrawal)
• e123d5e [native] Merge coordinator and native process logs in e2e tests. (Amit Dutta)
• dd42f26 [native] Fix TaskManagerTest.buildSpillDirectoryFailure (Sergey Pershin)
• 0a17a67 [native] Increase default request timeout for Exchange (Andrii Rosa)
• 0d48041 [native] Send explicit ack only when PrestoExchangeSource is paused (Andrii Rosa)
• d011944 [Native] Advance Velox (Andrii Rosa)
• c3e4c8e Prioritize Facts over HBO stats (abhinavmuk04)
• a150ea6 Update SingleStore image tag. (oyeliseiev-ua)
• 5e92be3 Fix SingleStore tests. Add metrics. (oyeliseiev-ua)
• d07c874 Fix doc build errors (Steve Burnett)
• dcb4ed5 [native] Allow running Driver threads with SCHED_BATCH policy (Andrii Rosa)
• 74fb8ce [native] Use dedicated thread pool for Exchange requests (Andrii Rosa)
• 2344876 Add analyzer warning for common NaN pitfalls (Rebecca Schlussel)
• c70fd3c Remove hard-coded values from TestNodeMemoryConfig to make tests independent of runtime heap size (Gary Helmling)
• 1d28c05 Fix error in 0.287 release notes (Steve Burnett)
• b7905c7 Exclude presto-ui from shaded jar (Elliotte Rusty Harold)
• 1947f9e [Native] Advance Velox (aditi-pandit)
• d412d86 Fix doc build errors (Steve Burnett)
• 3b03849 Fix doc build errors (Steve Burnett)
• ae7d90a Simplify code (Elliotte Rusty Harold)
• 45e3128 Fix typo in HiveSplitManager (wangd)
• e979ef0 Add coordinator sidecar process (Pratik Joseph Dabre)
• 435e177 Improve array function documentation (Rebecca Schlussel)
• 6d2e8cb Use distinct semantics for set_agg and set_union (Rebecca Schlussel)
• 0c8fbb7 Use distinct for array_intersect, array_except, array_union (Rebecca Schlussel)
• 1722ea7 Move array_union tests to TestArrayUnionFunction (Rebecca Schlussel)
• e7eb016 Consolidate array_intersect tests (Rebecca Schlussel)
• 01f5c65 Use distinct semantics for array_distinct (Rebecca Schlussel)
• dbfb6ac [native]Add to config largest size class pages (xiaoxmeng)
• 95da85b [native] Remove spark specific query config from BaseVeloxQueryConfig (Bikramjeet Vig)
• 30bd7fe Fix serialization of HBO plan nodes to output consistent hash (Feilong Liu)
• 5008458 Rename split --> splitRunner (Elliotte Rusty Harold)
• 5ed9da6 Fix hash for double and real types when new nan is disabled (Rebecca Schlussel)
• 794cbae [native] Add e2e tests for normalize scalar function (Pratik Joseph Dabre)
• 289ecb9 [Native] Advance velox (Karteekmurthys)
• 210876e [native] add entrypoint script into native worker docker image (Linsong Wang)
• 8f26ac6 Fix for previous correctness issue with TableScanConfidence (abhinavmuk04)
• 3fd0a02 Update Iceberg documentation with SQL queries (Reetika Agrawal)
• b7204ee [native]Add system config for global memory arbitration in velox (xiaoxmeng)
• 21667e2 [native] Advance velox. (Amit Dutta)
• 815012f Update .gitignore to include more IDE generated files (Yiqun (Ethan) Zhang)
• c3e4114 [native]Simplify test error message check (xiaoxmeng)
• 7d7d89d [native]Support no shuffle data copy to speedup query execution (xiaoxmeng)
• 0168e16 [native] Implement bucket conversion for Hive splits (Jimmy Lu)
• c60f472 [native] Advance Velox (Jimmy Lu)
• 6110456 Update Joda-Time to 2.12.7 (Amit Dutta)
• c542429 [native] Update CI dependency image to latest (Christian Zentgraf)
• 7727beb Fix wrong results bug with count over mixed aggregation (Rebecca Schlussel)
• 1286ae4 [native] Include additional context from velox exception (Bikramjeet Vig)
• 1ab45c2 Change getExpectedOrdersTableDescription() back to protected (Jialiang Tan)
• 5acdcd5 Update Jenkins pipeline per dockerfile name update (Linsong Wang)
• 35eab5d Add more documentation on Presto committers (Tim Meehan)
• 7164087 [native] Update dockerfiles to use Centos9 (Christian Zentgraf)
• 1e80972 Implement array_sum function in Java from SQL (abhinavmuk04)
• fd8d572 Fix hash calculation for Timestamp in HiveBucketing to be Hive Compatible (Ke)
• c55dad6 [native] Advance velox. (Amit Dutta)
• 7f8ff95 [docs] remove method toc icons (Zac Blanco)
• 416d3b5 Make private temporaryDirectory variable in TestHiveSkipEmptyFiles (cvarelad-denodo)
• 2597e60 Resolve TestHiveClientConfig conflicts (cvarelad-denodo)
• 6ed1e6f Fix TestHiveSkipEmptyFiles methods refactor (cvarelad-denodo)
• 24d3b6c Fix TestHiveSkipEmptyFiles temporal files and directories creation (cvarelad-denodo)
• b712b34 Add tests for bucketed tables in TestHiveSkipEmptyFiles (cvarelad-denodo)
• 4ac8108 Fixes in TesHiveSkipEmptyFiles and HiveClientConfig (cvarelad-denodo)
• 27bd90e Add property hive.skip_empty_files to session properties in TestQuickStatsProvider (cvarelad-denodo)
• da3d7c2 Fix code style in TestHiveSkipEmptyFiles (cvarelad-denodo)
• 176acba Add property to skip empty files in Hive Connector (cvarelad-denodo)
• 2ef2ba8 [Verifier] Support reuse of source query output table (Ge Gao)
• 4cbd3f4 Fix precedence (Elliotte Rusty Harold)
• 364d5ba Fill in missing spaces (Elliotte Rusty Harold)
• 0f99fe4 Remove unneeded suppression (Elliotte Rusty Harold)
• 7fec8c3 [native] Advance Velox (Christian Zentgraf)

[tdcmeehan]

@karteekmurthys
Can you suggest a release note for the below?
[native] Expose REST API to fetch worker stats in Prometheus format #22360 [native] Expose REST API to fetch worker stats in Prometheus format (Merged by: Deepak Majeti)

[karteekmurthys]

@karteekmurthys Can you suggest a release note for the below? [native] Expose REST API to fetch worker stats in Prometheus format #22360 [native] Expose REST API to fetch worker stats in Prometheus format (Merged by: Deepak Majeti)

"Add a REST API in Presto C++ worker to fetch worker stats in Prometheus Data Format."

Is this sufficient?

[steveburnett]

Usual pull branch, local doc build, review local doc build.

What I did in this review:

• Added missing PR numbers to release note entries.
• Minor formatting of property names following Formatting in the Release Notes entries.
• Nit of punctuation, moving the . from after the PR # to the end of the sentence. Edit applied throughout for consistency with the 0.288, 0.287, and 0.286 release notes.
• A very few changes to the text of entries.

What I did not do:

• Review any of the Missing Release Notes items.
• Revise all of the entries that do not conform to the Order of changes in the Release Notes Guidelines. Specifically, the items in a set in the General Changes section beginning with Make, Pass, Switch, The Iceberg should be rewritten. Others are welcome to, or I can address these in a second review after the changes in this review are applied.
• Because of the previous point, I did not sort the entries in General Changes to follow the Order of changes in the Release Notes Guidelines. Again, that can be addressed in a later review pass. I largely wanted to address formatting issues in this first pass.

[steveburnett]

Pull updated branch, local doc build, review local doc build.

What I did in this review:

• Checked all links to PR numbers to verify the linked PR had something to do with the release note entry. (Caught two typos in the PR numbers this way.)
• Checked all links in the release note entries.
• Reviewed the current draft as a whole again, looking for any nits in formatting.

[steveburnett]

In addition to my current open review above, one more thing: the release note entry for PR #23174 appears to be missing from the latest update. Was it intentionally removed?

#23174 (Author: Zac Blanco): [Iceberg] Add table refresh retry configurations

The Iceberg connector when configured with Hive, can now configure table metadata refresh timeouts. See documentation for details.

It should be revised from its current wording if it is to be included. Maybe something like the following:

Add configuration for retries of table metadata refresh timeouts, for the Iceberg connector when configured with Hive. :pr:23174

[tdcmeehan]

In addition to my current open review above, one more thing: the release note entry for PR #23174 appears to be missing from the latest update. Was it intentionally removed?

#23174 (Author: Zac Blanco): [Iceberg] Add table refresh retry configurations

The Iceberg connector when configured with Hive, can now configure table metadata refresh timeouts. See documentation for details.

It should be revised from its current wording if it is to be included. Maybe something like the following:

Add configuration for retries of table metadata refresh timeouts, for the Iceberg connector when configured with Hive. :pr:23174

I excluded this because I deemed it non-user facing and mostly for testing. @ZacBlanco do you agree?

[tdcmeehan]

@rschlussel does this need to be backported to 0.288?

• Fix a bug in the hash code of -0 for double and real types when use-new-nan-definition is false. :pr:23060

[tdcmeehan]

@swapsmagic

Fix Resource Group State Info

Does this need to be backported to 0.288?

[rschlussel]

Member

yes, that should be backported if the other nan changes are in 0.288

[steveburnett]

Only a few minor comments.

[tdcmeehan]

@pgupta2 I changed the release note for the query progress events:

• I moved it to the SPI section
• I mentioned that a new method was added to the EventListener class.

Please check this and let me know what you think:

• Add publishQueryProgress to EventListener to publish regular progress of queries in a Presto cluster. The event.query-progress-publish-interval config property can be used to specify the time interval at which progress events should be generated. Default is 0 (disabled). :pr:23195

[steveburnett]

One formatting nit I should have caught earlier, I think this should be everything.

[ZacBlanco]

I excluded this because I deemed it non-user facing and mostly for testing. @ZacBlanco do you agree?

That PR originated from a user from the community who was having issues with operations timing out and taking too long to fail, causing a hanging behavior. The configurations were added to the documentation because they may be desirable for a user to configure to decrease the amount of time spent "hanging". Personally I would keep it in the release notes, but don't have strong opinions.

[steveburnett]

LGTM! (docs)

Pull updated branch, local doc build, looks good. Thanks!

[tdcmeehan]

@ZacBlanco how does this sound?

Add configuration properties <https://prestodb.io/docs/current/connector/hive.html#aws-glue-catalog-configuration-properties> to tune table metadata refresh timeouts for the Iceberg connector when configured with the Hive or Glue catalog. :pr:23174

[ZacBlanco]

@ZacBlanco how does this sound?

Add configuration properties <https://prestodb.io/docs/current/connector/hive.html#aws-glue-catalog-configuration-properties> to tune table metadata refresh timeouts for the Iceberg connector when configured with the Hive or Glue catalog. :pr:23174

Is that link right? The configuration properties added are at connector/iceberg.html#glue-catalog

[tdcmeehan]

@ZacBlanco how does this sound?

Add configuration properties <https://prestodb.io/docs/current/connector/hive.html#aws-glue-catalog-configuration-properties> to tune table metadata refresh timeouts for the Iceberg connector when configured with the Hive or Glue catalog. :pr:23174

Is that link right? The configuration properties added are at connector/iceberg.html#glue-catalog

Oops, how about this:

Add configuration properties <https://prestodb.io/docs/current/connector/iceberg.html#glue-catalog> to tune table metadata refresh timeouts for the Iceberg connector when configured with the Hive or Glue catalog. :pr:23174

I think the documentation you added will be anchored to the Glue section.

[tdcmeehan]

I have incorporated @steveburnett and @elharo's feedback, and am now waiting on @pgupta2 on this item: #23513 (comment)

Additionally, @swapsmagic it would be helpful if you could comment on #23513 (comment)

[tdcmeehan]

I have incorporated @steveburnett and @elharo's feedback, and am now waiting on @pgupta2 on this item: #23513 (comment)

Additionally, @swapsmagic it would be helpful if you could comment on #23513 (comment)

@swapsmagic @pgupta2 ping

[pgupta2]

waiting on @pgupta2 on this item: #23513 (comment)

@tdcmeehan : The changes looks good to me. Thanks.

[swapsmagic]

@swapsmagic

Fix Resource Group State Info

Does this need to be backported to 0.288?

I don't think it's necessary, give the feature was introduced recently and disable by default. So unless someone started using it there is no need to backport it.