Skip to content

Latest commit

 

History

History
225 lines (170 loc) · 19.8 KB

release-4.0.9.md

File metadata and controls

225 lines (170 loc) · 19.8 KB
Raw
title summary
TiDB 4.0.9 Release Notes
TiDB 4.0.9 was released on December 21, 2020. The release includes compatibility changes, new features, improvements, bug fixes, and updates to TiKV, TiDB Dashboard, PD, TiFlash, and various tools. Notable changes include the deprecation of the `enable-streaming` configuration item in TiDB, support for storing the latest data of the storage engine on multiple disks in TiFlash, and various bug fixes in TiDB and TiKV.

TiDB 4.0.9 Release Notes

Release date: December 21, 2020

TiDB version: 4.0.9

Compatibility Changes

  • TiDB

    • Deprecate the enable-streaming configuration item #21055

  • TiKV

    • Reduce I/O and mutex contention when encryption at rest is enabled. The change is backwardly incompatible. If users need to downgrade the cluster to a version earlier than v4.0.9, security.encryption.enable-file-dictionary-log must be disabled and TiKV must be restarted before the downgrade. #9195

New Features

  • TiFlash

    • Support storing the latest data of the storage engine on multiple disks (experimental)

  • TiDB Dashboard

    • Support displaying and sorting by all fields in the SQL Statements page #749
    • Support zooming and panning the topology graph #772
    • Support displaying the disk usage information in the SQL Statements and Slow Queries pages #777
    • Support exporting list data in the SQL Statements and Slow Queries pages #778
    • Support customizing the Prometheus address #808
    • Add a page for cluster statistics #815
    • Add more time-related fields in the Slow Queries details #810

Improvements

  • TiDB

    • Avoid the (index) merge join in a heuristical way when converting equal conditions to other conditions #21146
    • Differentiate the types of user variables #21107
    • Support setting the GOGC variable in the configuration file #20922
    • Make the dumped binary time (Timestamp and Datetime) more compatible with MySQL #21135
    • Provide an error message for statements that use the LOCK IN SHARE MODE syntax #21005
    • Avoid outputting unnecessary warnings or errors when folding constants in shortcut-able expressions #21040
    • Raise an error when preparing the LOAD DATA statement #21199
    • Ignore the attribute of the integer zero-fill size when changing the integer column types #20986
    • Add the executor-related runtime information of DML statements in the result of EXPLAIN ANALYZE #21066
    • Disallow multiple updates on the primary key in a singe SQL statements #21113
    • Add a monitoring metric for the connection idle time #21301
    • Temporarily enable the slow log when the runtime/trace tool is running #20578

  • TiKV

    • Add the tag to trace the source of the split command #8936
    • Support dynamically changing the pessimistic-txn.pipelined configuration #9100
    • Reduce the impact on performance when running Backup & Restore and TiDB Lightning #9098
    • Add monitoring metrics for the ingesting SST errors #9096
    • Prevent the leader from being hibernated when some peers still need to replicate logs #9093
    • Increase the success rate of the pipelined pessimistic locking #9086
    • Change the default value of apply-max-batch-size and store-max-batch-size to 1024 #9020
    • Add the max-background-flushes configuration item #8947
    • Disable force-consistency-checks by default to improve performance #9029
    • Offload the queries on the Region size from pd heartbeat worker to split check worker #9185

  • PD

    • Check the TiKV cluster version when a TiKV stores become Tombstone, which prevents users from enabling incompatible features during the process of downgrade or upgrade #3213
    • Disallow the TiKV store of a lower version to change from Tombstone back to Up #3206

  • TiDB Dashboard

    • Keep expanding when "Expand" is clicked for SQL statements #775
    • Open detail pages in new windows for SQL Statements and Slow Queries #816
    • Improve descriptions for time-related fields in Slow Queries details #817
    • Display detailed error messages #794

  • TiFlash

    • Reduce the latency of replica reads
    • Refine TiFlash's error messages
    • Limit the memory usage of cache data when the data volume is huge
    • Add a monitoring metric for the number of coprocessor tasks being handled

  • Tools

    • Backup & Restore (BR)

      • Disallow the ambiguous --checksum false argument in the command line, which does not correctly disable checksum. Only --checksum=false is accepted. #588
      • Support changing the PD configuration temporarily so that PD can recover the original configuration after BR accidentally exists #596
      • Support analyzing tables after restore #622
      • Retry for the read index not ready and proposal in merging mode errors #626

    • TiCDC

      • Add an alert for enabling TiKV's Hibernate Region feature #1120
      • Reduce memory usage in the schema storage #1127
      • Add the feature of unified sorter, which accelerates replication when the data size of the incremental scan is large (experimental) #1122
      • Support configuring the maximum message size and the maximum message batch in the TiCDC Open Protocol message (only for Kafka sink) #1079

    • Dumpling

      • Retry dumping data on failed chunks #182
      • Support configuring both the -F and -r arguments at the same time #177
      • Exclude system databases in --filter by default #194
      • Support the --transactional-consistency parameter and support rebuilding MySQL connections during retry #199
      • Support using the -c,--compress parameter to specify the compression algorithm used by Dumpling. An empty string means no compression. #202

    • TiDB Lightning

      • Filter out all system schemas by default #459
      • Support setting a default value for the auto-random primary key for the Local-backend or Importer-backend #457
      • Use range properties to make the range split more precise in Local-backend #422
      • Support a human-readable format (such as "2.5 GiB") in tikv-importer.region-split-size, mydumper.read-block-size, mydumper.batch-size, and mydumper.max-region-size #471

    • TiDB Binlog

      • Exit the Drainer process with the non-zero code if the upstream PD is down or if applying DDL or DML statements to the downstream fails #1012

Bug Fixes

  • TiDB

    • Fix the issue of incorrect results when using a prefix index with the OR condition #21287
    • Fix a bug that might cause panic when automatic retry is enabled #21285
    • Fix a bug that occurs when checking partition definition according to column type #21273
    • Fix a bug that the value type of the partition expression is not consistent with the partition column type #21136
    • Fix a bug that the hash-type partition does not check whether the partition name is unique #21257
    • Fix the wrong results returned after inserting a value of the non-INT type into the hash partitioned table #21238
    • Fix the unexpected error when using index join in the INSERT statement in some cases #21249
    • Fix the issue that the BigInt unsigned column value in the CASE WHEN operator is incorrectly converted to the BigInt signed value #21236
    • Fix a bug that index hash join and index merge join do not consider collation #21219
    • Fix a bug that the partitioned table does not consider collation in the CREATE TABLE and SELECT syntax #21181
    • Fix the issue that the query result of slow_query might miss some rows #21211
    • Fix the issue that DELETE might not delete data correctly when the database name is not in a pure lower representation #21206
    • Fix a bug that causes schema change after DML operations #21050
    • Fix the bug that the coalesced column cannot be queried when using join #21021
    • Fix the wrong results of some semi-join queries #21019
    • Fix the issue that the table lock does not take effect on the UPDATE statement #21002
    • Fix the issue of stack overflow that occurs when building the recursive view #21001
    • Fix the unexpected result returned when performing index merge join operations on outer join #20954
    • Fix the issue that sometimes a transaction that has an undetermined result might be treated as failed #20925
    • Fix the issue that EXPLAIN FOR CONNECTION cannot show the last query plan #21315
    • Fix the issue that when Index Merge is used in a transaction with the Read Committed isolation level, the result might be incorrect #21253
    • Fix the auto-ID allocation failure caused by the transaction retry after the write conflict #21079
    • Fix the issue that JSON data cannot be correctly imported to TiDB using LOAD DATA #21074
    • Fix the issue that the default value of newly added Enum-type columns is incorrect #20998
    • Fix the issue that the adddate function inserts invalid characters #21176
    • Fix the issue that the wrong PointGet plan generated in some situations causes wrong results #21244
    • Ignore the conversion of daylight saving time in the ADD_DATE function to be compatible with MySQL #20888
    • Fix a bug that prevents inserting strings with trailing spaces that exceed varchar or char's length constraint #21282
    • Fix a bug that does not converting the integer from [1, 69] to [2001, 2069] or from [70, 99] to [1970, 1999] when comparing int with year #21283
    • Fix the panic caused by the overflowing result of the sum() function when calculating the Double type field #21272
    • Fix a bug that DELETE fails to add lock on the unique key #20705
    • Fix a bug that snapshot reads hits the lock cache #21539
    • Fix an issue of potential memory leak after reading a lot of data in a long-lived transaction #21129
    • Fix the issue that omitting the table alias in a subquery will have a syntax error returned #20367
    • Fix the issue that when the argument of the IN function in a query is the time type, the query might return an incorrect result #21290

  • TiKV

    • Fix the issue that Coprocessor might return wrong results when there are more than 255 columns #9131
    • Fix the issue that Region Merge might cause data loss during network partition #9108
    • Fix the issue that the ANALYZE statement might cause panic when using the latin1 character set #9082
    • Fix the wrong results returned when converting the numeric type to the time type #9031
    • Fix a bug that TiDB Lightning fails to ingest SST files to TiKV with the Importer-backend or Local-backend when Transparent Data Encryption (TDE) is enabled #8995
    • Fix the invalid advertise-status-addr value (0.0.0.0) #9036
    • Fix the issue that an error is returned indicating that a key exists when this key is locked and deleted in a committed transaction #8930
    • Fix the issue that the RocksDB cache mapping error causes data corruption #9029
    • Fix a bug that Follower Read might return stale data after the leader is transferred #9240
    • Fix the issue that stale old values might be read in the pessimistic lock #9282
    • Fix a bug that replica read might get stale data after the leader transfer #9240
    • Fix the issue of TiKV crash that occurs when receiving SIGPROF after profiling #9229

  • PD

    • Fix the issue that the leader roles specified using placement rules do not take effect in some cases #3208
    • Fix the issue that the trace-region-flow value is unexpectedly set to false #3120
    • Fix a bug that the service safepoint with infinite Time To Live (TTL) does not work #3143

  • TiDB Dashboard

    • Fix a display issue of time in the Chinese language #755
    • Fix a bug that the browser compatibility notice does not work #776
    • Fix the issue that the transaction start_ts is incorrectly displayed in some scenarios #793
    • Fix the issue that some SQL texts are incorrectly formatted #805

  • TiFlash

    • Fix the issue that INFORMATION_SCHEMA.CLUSTER_HARDWARE might contain the information of disks that are not in use
    • Fix the issue that the estimate on memory usage of Delta Cache is smaller than the actual usage
    • Fix the memory leak caused by thread information statistics

  • Tools

    • Backup & Restore (BR)

      • Fix the failure caused by special characters in S3 secret access keys #617

    • TiCDC

      • Fix the issue that multiple owners might exist when the owner campaign key is deleted #1104
      • Fix a bug that TiCDC might fail to continue replicating data when a TiKV node crashes or recovers from a crash. This bug only exists in v4.0.8. #1198
      • Fix the issue that the metadata is repeatedly flushed to etcd before a table is initialized #1191
      • Fix an issue of replication interruption caused by early GC or the latency of updating TableInfo when the schema storage caches TiDB tables #1114
      • Fix the issue that the schema storage costs too much memory when DDL operations are frequent #1127
      • Fix the goroutine leak when a changefeed is paused or stopped #1075
      • Increase the maximum retry timeout to 600 seconds in Kafka producer to prevent replication interruption caused by the service or network jitter in the downstream Kafka #1118
      • Fix a bug that the Kafka batch size does not take effect #1112
      • Fix a bug that some tables' row change might be lost when the network between TiCDC and PD has jitter and when there are paused changefeeds being resumed at the same time #1213
      • Fix a bug that the TiCDC process might exit when the network between TiCDC and PD is not stable #1218
      • Use a singleton PD client in TiCDC and fix a bug that TiCDC closes PD client by accident which causes replication block #1217
      • Fix a bug that the TiCDC owner might consume too much memory in the etcd watch client #1224

    • Dumpling

      • Fix the issue that Dumpling might get blocked when its connection to the MySQL database server is closed #190

    • TiDB Lightning

      • Fix the issue that keys are encoded using the wrong field information #437
      • Fix the issue that GC life time TTL does not take effect #448
      • Fix the issue that causes panic when manually stops the running TiDB Lightning in the Local-backend mode #484