Skip to content

Latest commit

 

History

History
1058 lines (773 loc) · 54.4 KB

RELEASE-NOTES.md

File metadata and controls

1058 lines (773 loc) · 54.4 KB
Raw

5.1.0

New feature

  1. Support SQL hint
  2. New DistSQL syntax: SHOW AUTHORITY RULE
  3. New DistSQL syntax: SHOW TRANSACTION RULE
  4. New DistSQL syntax: ALTER TRANSACTION RULE
  5. New DistSQL syntax: SHOW SQL_PARSER RULE
  6. New DistSQL syntax: ALTER SQL_PARSER RULE
  7. New DistSQL syntax: ALTER DEFAULT SHARDING STRATEGY
  8. New DistSQL syntax: DROP DEFAULT SHARDING STRATEGY
  9. New DistSQL syntax: CREATE DEFAULT SINGLE TABLE RULE
  10. New DistSQL syntax: SHOW SINGLE TABLES
  11. New DistSQL syntax: SHOW SINGLE TABLE RULES
  12. New DistSQL syntax: SHOW SHARDING TABLE NODES
  13. New DistSQL syntax: CREATE/ALTER/DROP SHARDING KEY GENERATOR
  14. New DistSQL syntax: SHOW SHARDING KEY GENERATORS
  15. New DistSQL syntax: REFRESH TABLE METADATA
  16. New DistSQL syntax: PARSE SQL, Output the abstract syntax tree obtained by parsing SQL
  17. New DistSQL syntax: SHOW UNUSED SHARDING ALGORITHMS
  18. New DistSQL syntax: SHOW UNUSED SHARDING KEY GENERATORS
  19. New DistSQL syntax: CREATE/DROP SHARDING SCALING RULE
  20. New DistSQL syntax: ENABLE/DISABLE SHARDING SCALING RULE
  21. New DistSQL syntax: SHOW SHARDING SCALING RULES
  22. New DistSQL syntax: SHOW INSTANCE MODE
  23. New DistSQL syntax: COUNT SCHEMA RULES
  24. Scaling: Add rateLimiter configuration and QPS TPS implementation
  25. Scaling: Add DATA_MATCH data consistency check
  26. Scaling: Add batchSize configuration to avoid possible OOME
  27. Scaling: Add streamChannel configuration and MEMORY implementation
  28. Scaling: Support MySQL BINARY data type
  29. Scaling: Support MySQL YEAR data type
  30. Scaling: Support PostgreSQL BIT data type
  31. Scaling: Support PostgreSQL MONEY data type
  32. Database discovery adds support for JDBC Spring Boot
  33. Database discovery adds support for JDBC Spring Namespace
  34. Database discovery adds support for openGauss
  35. Shadow DB adds support for logical data source transfer
  36. Add data type validator for column matching shadow algorithm
  37. Add support for xa start/end/prepare/commit/recover in encrypt case with only one data source

API Change

  1. Redesign the database discovery related DistSQL syntax
  2. In DistSQL, the keyword GENERATED_KEY is adjusted to KEY_GENERATE_STRATEGY
  3. Native authority provider is marked as deprecated and will be removed in a future version
  4. Scaling: Move scaling configuration from server.yaml to config-sharding.yaml
  5. Scaling: Rename clusterAutoSwitchAlgorithm SPI to completionDetector and refactor method parameter
  6. Scaling: Data consistency check API method rename and return type change
  7. Database discovery module API refactoring
  8. Readwrite-splitting supports static and dynamic configuration
  9. Shadow DB remove the enable configuration
  10. Shadow algorithm type modified

Enhancement

  1. Improve load multi single table performance
  2. Remove automatically added order by primary key clause
  3. Optimize binding table route logic without sharding column in join condition
  4. Support update sharding key when the sharding routing result keep the same
  5. Optimize rewrite engine performance
  6. Support select union/union all ... statements by federation engine
  7. Support insert on duplicate key update sharding column when route context keep same
  8. Use union all to merge sql route units for simple select to improve performance
  9. Supports autocommit in ShardingSphere-Proxy
  10. ShardingSphere openGauss Proxy supports sha256 authentication method
  11. Remove property java.net.preferIPv4Stack=true from Proxy startup script
  12. Remove the verification of null rules for JDBC
  13. Optimize performance of executing openGauss batch bind
  14. Disable Netty resource leak detector by default
  15. Supports describe prepared statement in PostgreSQL / openGauss Proxy
  16. Optimize performance of executing PostgreSQL batched inserts
  17. Add instance_id to the result of SHOW INSTANCE LIST
  18. Support to use instance_id to perform operations when enable/disable a proxy instance
  19. Support auto creative algorithm when CREATE SHARDING TABLE RULE, reducing the steps of creating rule
  20. Support specifying an existing KeyGenerator when CREATE SHARDING TABLE RULE
  21. DROP DATABASE supports IF EXISTS option
  22. DATANODES in SHARDING TABLE RULE supports enumerated inline expressions
  23. CREATE/ALTER SHARDING TABLE RULE supports complex sharding algorithm
  24. SHOW SHARDING TABLE NODES supports non-inline scenarios (range, time, etc.)
  25. When there is only one read data source in the readwrite-splitting rule, it is not allowed to be disabled
  26. Scaling: Add basic support of chunked streaming data consistency check
  27. Shadow algorithm decision logic optimization to improve performance

Refactor

  1. Refactor federation engine scan table logic
  2. Avoid duplicated TCL SQL parsing when executing prepared statement in Proxy
  3. Scaling: Add pipeline modules to redesign scaling
  4. Scaling: Refactor several job configuration structure
  5. Scaling: Precalculate tasks splitting and persist in job configuration
  6. Scaling: Add basic support of pipeline-core code reuse for encryption job
  7. Scaling: Add basic support of scaling job and encryption job combined running
  8. Scaling: Add input and output configuration, including workerThread and rateLimiter
  9. Scaling: Move blockQueueSize into streamChannel
  10. Scaling: Change jobId type from integer to text
  11. Optimize JDBC to load only the specified schema
  12. Optimize meta data structure of the registry center
  13. Rename Note shadow algorithm to HINT shadow algorithm

Bug Fix

  1. Support parsing function
  2. Fix alter table drop constrain
  3. Fix optimize table route
  4. Support Route resource group
  5. Support parsing binlog
  6. Support postgreSql/openGauss '&' and '|' operator
  7. Support parsing openGauss insert on duplicate key
  8. Support parse postgreSql/openGauss union
  9. Support query which table has column contains keyword
  10. Fix missing parameter in function
  11. Fix sub query table with no alias
  12. Fix utc timestamp function
  13. Fix alter encrypt column
  14. Support alter column with position encrypt column
  15. Fix delete with schema for postgresql
  16. Fix wrong route result caused by oracle parser ambiguity
  17. Fix projection count error when use sharding and encrypt
  18. Fix npe when using shadow and readwrite_splitting
  19. Fix wrong metadata when actual table is case insensitive
  20. Fix encrypt rewrite exception when execute multiple table join query
  21. Fix encrypt rewrite wrong result with table level queryWithCipherColumn
  22. Fix parsing chinese
  23. Fix encrypt exists sub query
  24. Fix full route caused by the MySQL BINARY keyword in the sharding condition
  25. Fix getResultSet method empty result exception when using JDBCMemoryQueryResult processing statement
  26. Fix incorrect shard table validation logic when creating store function/procedure
  27. Fix null charset exception occurs when connecting Proxy with some PostgreSQL client
  28. Fix executing commit in prepared statement cause transaction status incorrect in MySQL Proxy
  29. Fix client connected to Proxy may stuck if error occurred in PostgreSQL with non English locale
  30. Fix file not found when path of configurations contains blank character
  31. Fix transaction status may be incorrect cause by early flush
  32. Fix the unsigned datatype problem when query with PrepareStatement
  33. Fix protocol violation in implementations of prepared statement in MySQL Proxy
  34. Fix caching too many connections in openGauss batch bind
  35. Fix the problem of missing data in SHOW READWRITE_SPLITTING RULES when db-discovery and readwrite-splitting are used together
  36. Fix the problem of missing data in SHOW READWRITE_SPLITTING READ RESOURCES when db-discovery and readwrite-splitting are used together
  37. Fix the NPE when the CREATE SHARDING TABLE RULE statement does not specify the sub-database and sub-table strategy
  38. Fix NPE when PREVIEW SQL by schema.table
  39. Fix DISABLE statement could disable readwrite-splitting write data source in some cases
  40. Fix DIABLE INSTANCE could disable the current instance in some cases
  41. Fix the issue that user may query the unauthorized logic schema when the provider is SCHEMA_PRIVILEGES_PERMITTED
  42. Fix NPE when authority provider is not configured
  43. Scaling: Fix DB connection leak on XA initialization which triggered by data consistency check
  44. Scaling: Fix PostgreSQL replication stream exception on multiple data sources
  45. Scaling: Fix migrating updated record exception on PostgreSQL incremental phase
  46. Scaling: Fix MySQL 5.5 check BINLOG_ROW_IMAGE option failure
  47. Scaling: Fix PostgreSQL xml data type consistency check
  48. Fix database discovery failed to modify cron configuration
  49. Fix single read data source use weight loadbalance algorithm error
  50. Fix create redundant data souce without memory mode
  51. Fix column value matching shadow algorithm data type conversion exception

Change Log

  1. MILESTONE

5.0.0

New feature

  1. Support parsing SQL comment
  2. New DistSQL syntax: shadow rule management
  3. New DistSQL syntax: scaling job management
  4. New DistSQL syntax: disable proxy instance
  5. New DistSQL syntax: disable readwrite-splitting read data source
  6. New DistSQL syntax: DROP SHARDING ALGORITHM
  7. New DistSQL syntax: ALTER RESOURCE
  8. New DistSQL syntax: CREATE SHARDING ALGORITHM
  9. New DistSQL syntax: CREATE DEFAULT SHARDING [TABLE | DATABASE] STRATEGY
  10. New DistSQL syntax: SHOW ALL VARIABLE
  11. New DistSQL syntax：SHOW VARIABLE variableName;
  12. Support 3 modes, including Memory, Standalone and Cluster mode
  13. Proxy supports for openGauss
  14. Scaling: Add basic support for openGauss
  15. Scaling: Add incremental task completion detect algorithm SPI interface
  16. Scaling: Add data consistency check algorithm SPI interface
  17. Scaling: Basic support of create table on target automatically for MySQL and openGauss
  18. Scaling: Support auto switch cluster configuration when job finished
  19. Scaling: Add more DistSQL support such as data consistency check, etc

API Change

  1. Add schema name configuration for ShardingSphere-JDBC
  2. Add default sharding column configuration
  3. Change the default authority provider from NATIVE to ALL_PRIVILEGES_PERMITTED
  4. SCTL syntax adjustment, merged with DistSQL RAL syntax
  5. SHOW RESOURCES DistSQL is adjusted to SHOW SCHEMA RESOURCES
  6. Remove shadow logical field, support shadow algorithm

Enhancement

  1. Support parsing MySQL union/union all statement
  2. Support PostgreSQL ABORT statement
  3. Refactor and improve metadata loading process
  4. Support PostgreSQL CREATE INDEX statement to generate index automatically when no index is specified
  5. Support SQL statement execution with logical schema
  6. Support binding tables configuration with different sharding columns
  7. Optimize kernel performance
  8. Proxy supports queries to part of information_schema tables to optimize client connection experience
  9. DistSQL supports using quotation marks to use keywords as parameter names
  10. The password in the ADD RESOURCE statement supports special characters
  11. ADD RESOURCE supports custom JDBC parameters and connection pool properties
  12. DROP RESOURCE supports optional parameter ignore single tables, used to ignore single table rule restrictions
  13. Support the use of DistSQL to create sharding table rule based on readwrite-splitting rules
  14. SHOW DATABASES statement supports like syntax
  15. CREATE SHARDING TABLE RULE supports the use of inline expressions to specify resources
  16. CREATE SHARDING TABLE RULE supports configuration using dataNodes
  17. CREATE SHARDING TABLE RULE supports reuse of existing algorithms
  18. SET VARIABLE, support to modify proxy configuration
  19. PostgreSQL's protocol enhancements (Such as supports Portal, unspecified type)
  20. Using Netty executor to optimize Proxy performance in specified scenarios
  21. Make memory strictly fetch size configurable in Proxy
  22. Scaling: Improve support for PostgreSQL
  23. Scaling: Support concurrent data consistency check of source and target side

Refactor

  1. Refactor the SingleTable feature to support Encrypt multiple data sources
  2. Adjust the persistent data structure of the registry center state node
  3. Remove the SQL rewrite of DML for Shadow
  4. Support the SQL routing of DDL for Shadow
  5. Scaling: Refactor default implementation of incremental task completion detect algorithm
  6. Scaling: Refactor default implementation of data consistency check algorithm
  7. Scaling: Remove HTTP API and binary distribution

Bug Fix

  1. Fix sharding interval algorithm
  2. Fix SHOW INDEX FROM TABLE FROM TABLE statement rewrite exception
  3. Fix Encrypt multi tables join query rewrite exception
  4. Fix subquery index out of range exception
  5. Fix wrong result of Oracle paging query
  6. Fix the rewrite exception when KeyGenerateStrategy is not configured in the Sharding feature
  7. Fix federation executor engine exception caused by Oracle dialect case
  8. Fix Sharding and Encrypt integration usage rewrite exception
  9. Fix Oracle metadata loading exception
  10. Fix the issue that SHOW RESOURCES statement cannot display custom attributes
  11. Fix the issue that SQL execution exception is not thrown
  12. Fix Etcd can not send node added event
  13. Fix PostgreSQL rows contains null value may be missing in query result
  14. Fix PostgreSQL metadata columns are out-of-order
  15. Fix client character set may be incorrect in Proxy

Change Log

  1. MILESTONE

5.0.0-beta

New feature

  1. New DistSQL to load and show all the ShardingSphere configuration rules
  2. Support join SQL from different database instances
  3. Support multiple backend database types for new feature of database gateway
  4. Support creating and updating the authentication online
  5. Add new automated agent module

API Change

  1. QueryReplica configuration item is replaced by read-write-splitting
  2. Authentication configuration item of ShardingProxy is replaced by AUTHORITY
  3. Optimize the datasource configuration for ShardingJDBC with dataSourceClassName
  4. New API for automated sharding table configuration to provide standard and automated sharding tables
  5. Remove configuration item acceptor-size from ShardingProxy
  6. Create a built-in sharding algorithm SPI which allows users to directly configure the class name as in 4.x

Enhancement

  1. Improve metadata loading process distinctly
  2. Greatly enhance the SQL parsing for Oracle, SQLServer and PostgreSQL
  3. Support loading privileges from MySQL/PostgreSQL/SQLServer/Oracle
  4. Support DDL statement for encryption feature
  5. Support rewrite owner table name of projection if using sharding and encrypt together
  6. When using SELECT * from encrypt SQL, add quote char for rewritten column to avoid conflict with keyword
  7. Support PostgreSQL JSON/JSONB/pattern matching operator parse
  8. Support MySQL/PostgreSQL CREATE/ALTER/DROP TABLESPACE statement
  9. Support PostgreSQL PREPARE, EXECUTE, DEALLOCATE statement
  10. Support PostgreSQL EXPLAIN statement
  11. Support PostgreSQL START/END TRANSACTION statement
  12. Support PostgreSQL ALTER/DROP INDEX statement
  13. Support CREATE TABLESPACE of PostgreSQL
  14. Support MySQL CREATE LOADABLE FUNCTION statement
  15. Support MySQL/PostgreSQL ALTER TABLE RENAME statement
  16. Support PostgreSQL Close command

Refactor

  1. New schema structure in registry center
  2. Remove Nacos and Apollo config center support
  3. ShardingScaling leverages elasticJob as its job distribution
  4. Refactor the metadata content and its update process

Bug Fix

  1. Bug fix for cannot use SELECT * wildcard when readwrite-splitting only
  2. Fix instance error when the custom sharding algorithm does not match the configuration type.
  3. Fix NoSuchTableException when execute drop table if exists statement
  4. Fix wrong table rewrite in UPDATE ... SET ... statement
  5. Fix wrong table rewrite in CREATE/ALTER TABLE statement foreign key reference table
  6. Fix projection owner check exception when exist subquery temporary table
  7. Fix Oracle/SQL92 SELECT ... WHERE ... LIKE statement class cast exception
  8. Fix MySQL SELECT EXISTS ... FROM ... statement parse error
  9. Fix wrong result of SHOW INDEX statement
  10. Fix SELECT ... GROUP BY ... statement rewrite and merge result error
  11. Fix CREATE TABLE statement rewrite error for encrypt
  12. Fix exception occur in PostgreSQL Proxy when reading text format parameter values
  13. Enhance the support of array object for PostgreSQL Proxy
  14. Fix the bug of Datetype casting for ShardingProxy
  15. PostgreSQL Proxy supports using numeric type
  16. Fix PostgreSQL Proxy transaction command complete packet's tag incorrect
  17. Fix PostgreSQL Proxy may return packet which is not expected by client

Change Log

  1. MILESTONE

5.0.0-alpha

Build & Dependencies

  1. Upgrade the minimum supported version of JDK to Java8
  2. Update Zookeeper to version 3.6.x and curator to version 5.1.0
  3. Update Google Guava to version 29.0-jre

New Feature

  1. The pluggable architecture is available and support function extension through the SPI
  2. Independent SQL parsing engine is available to get SQL parsed AST for the different database dialects
  3. New RDL(Rule Definition Language) feature for ShardingSphere Proxy supports creating sharding rules and sharding tables
  4. ShardingSphere-Scaling supports resuming data migration from break-point
  5. ShardingSphere-Scaling supports using ShardingSphere JDBC to migrate data to new cluster
  6. ShardingSphere shadow database is available

API Changes

  1. New sharding/encryption/shadow/queryReplica API
  2. New sharding algorithm and strategy API
  3. New API for ShardingSphere Scaling to create task
  4. Remove DefaultDataSourceName setting item
  5. The separator of the parameter configuration item is changed from the dot ‘.’ to minus sign ‘-’
  6. Change parameter allow.range.query.with.inline.sharding from global parameter to sharding algorithm parameter

Refactor

  1. Refactor the architecture of SQL parsing module base on the database dialects
  2. Use SPI mechanism to reconstruct online metadata change processing
  3. Rename Orchestration module to Governance module
  4. Rename MasterSlave module to QueryReplica module
  5. Refactor the metadata structure in the governance registration center
  6. Refactor GovernmentShardingSphereDataSource
  7. ShardingSphere UI merges configuration center and registration center display layout

Enhancement

  1. The enhancement for MySQL and PostgreSQL SQL syntax definition and parsing process
  2. The enhancement for sub-queries in different database dialects
  3. Support MySQL view operations for non-sharding tables
  4. Support MySQL stored function and procedure operations for non-sharding tables
  5. Support SQLServer Top syntax
  6. Optimize the metadata loading to improve the startup speed
  7. Optimize batch insert performance
  8. Supports the use of Oracle RAC JDBC URL
  9. XA transaction manager adds support for Oracle database
  10. ShardingSphere Proxy supports the use of p6sy driver
  11. Add help information to the ShardingSphere Proxy startup script

Bug Fixes

  1. Fix alias rewriting error when processing OrderBy condition
  2. Fix SQL rewriting error when MYSQL Insert statement contains expression
  3. Fix parameter calculation error in update on duplicate SQL
  4. Fix generatedKeys gets wrong when batch inserting
  5. Fix the abnormal issue of multi-table verification in DML statement update operation
  6. Fix the NPE problem caused by executing SQL when the table does not exist
  7. Fix the exception when using the Show table command for an unconfigured table
  8. Fix metadata loading error when Oracle database has multiple qualified users
  9. Fix the issue that replica node cannot be enabled online
  10. Fix the problem that ShardingSphere-JDBC does not support PostgreSQL array type
  11. Fix the problem that ShardingSphere-Proxy does not respond when querying long blob data

Change Logs

  1. MILESTONE

4.1.1

Enhancement

  1. Add Sharding-Scaling & Sharding-UI dockerfile
  2. update MySQL & PostgreSQL version for proxy

Bug Fixes

  1. Fix parser syntax rule of SUBSTRING and CONVERT
  2. Fix parser ColumnSegment ClassCastException
  3. Fix TableMetaData load error when use Sharding-JDBC with oracle
  4. Fix getSchema NPE when use Sharding-JDBC with oracle
  5. Fix Sharding-JDBC parse sql NPE in PostgreSQL
  6. Fix Sharding-Proxy receive error response for PostgreSQL JDBC client
  7. Fix Sharding-Proxy response number of update is 0 for PostgreSQL JDBC client
  8. Fix Sharding-Proxy receive null for PostgreSQL column meta data
  9. Fix Sharding-Scaling NPE in MySQL incremental task

Change Logs

  1. MILESTONE

4.1.0

New Features

  1. Support scaling for ShardingSphere (alpha version)
  2. Move etcd registry center from experiment repo to apache repo
  3. Upgrade to Java 8

Enhancement

  1. Optimization for Sharing Parser with ANTLR Visitor improving the parsing performance of long SQL by 100%~1000%
  2. Use multiple threads to load metadata for different data sources
  3. Support allow.range.query.with.inline.sharding option
  4. The docker of ShardingSphere-Proxy supports loading external lib packages
  5. Support integration with Spring using @ShardingSphereTransactionType
  6. Enhance ShardingDataSource to compatible with Seata in micro-service distribution transaction

Refactor

  1. Remove leaf key generator

Bug Fixes

  1. Fix an exception caused by using a CHAR/VARCHAR type column as an order by item
  2. Refine DataTypeName syntax rules of all database dialects
  3. Fix an exception caused by executing BEGIN using prepared statement of MySQL C API
  4. Fix the problem that ALTER TABLE fails to execute when the field type of the table contains Integer/Double/BigDecimal
  5. Fix the problem of the stop index dislocation of segment with alias
  6. Fix the problem that overwriting SQL SELECT * FROM tbl WHERE encrypt_col = ? AND (sharding_key = ? OR sharding_key = ?) when using sharding + encrypt would throw StringIndexOutOfBoundsException
  7. Fix the problem of incorrect decoding after AES encoding when using ShardingSphere-Proxy in Spring Boot
  8. Fix a long-time problem of adding schema dynamically in ShardingSphere-UI

Change Logs

  1. MILESTONE

4.0.1

Bug Fixes

  1. Using guava cache to fix parsing deadlock.
  2. Oracle insert SQL could not work in encrypt mode.
  3. Proxy for PostgreSQL decode parameters error in all types except String.
  4. COM_STM_EXECUTE of proxy for MySQL could not support sysbench.
  5. None sharding strategy could not config in spring-boot.
  6. Plain column could not get from resultSet in encrypt mode.
  7. WasNull field was wrong in GroupByStreamMergeResult.
  8. Metadata.getColumns could not work in JDBC.
  9. IN operator contains space and \n \t \r could not supported by parser.

Enhancement

  1. Optimize antlr performance using two-stage parsing strategy.
  2. Add class filter constructor to restrict the illegal class from YAML.

Change Logs

  1. MILESTONE

4.0.0

API Changes

  1. Change package and maven groupId form io.shardingsphere to org.apache.shardingsphere.
  2. Adjust ShardingSphere-JDBC configuration API.
  3. Adjust persist structure for registry center.

New Features

  1. SQL92 Syntax available.
  2. ShardingSphere-Proxy for PostgreSQL protocol available.
  3. SQL 100% compatible if route to single data node.
  4. Less-than(<), greater-than(>) and Less-than-equal(<=), greater-than-equal(>=) for sharding key operator available.
  5. DISTINCT SQL syntax available.
  6. Broadcast table available.
  7. LEAF key generator available.
  8. XA Transaction available, Atomikos, Narayana and Bitronix integrated.
  9. BASE Transaction available, Seata integrated.
  10. Data encrypt available.
  11. Skywalking plugin available.
  12. ShardingSphere-UI available, an orchestration management platform.

Enhancement

  1. MariaDB supported.
  2. Improve the compatibility of SQL parsing.
  3. SELECT FOR UPDATE route to primary data source only.
  4. Hint in ShardingSphere-Proxy available.
  5. Make configuration of orchestration consistent between ShardingSphere-JDBC and ShardingSphere-Proxy.
  6. Renew modified data sources only, not renew all the data sources.
  7. Vibrate configurable for Snowflake key generator.

Bug Fixes

  1. Improve the compatibility of JDBC Driver URL.
  2. Delete statement with alias available.
  3. Check and disable updating sharding column.
  4. Fix wrong type of TINYINT and SMALLINT as INTEGER.

Change Logs

  1. MILESTONE #3
  2. MILESTONE #4
  3. MILESTONE #5
  4. MILESTONE #6
  5. MILESTONE #7
  6. MILESTONE #8
  7. MILESTONE #9

4.0.0.RC3

New Features

  1. ShardingSphere-UI, an orchestration management platform for ShardingSphere comes online.
  2. Not only SQLs from MySQL, PostgreSQL, SQLServer, Oracle, but any SQL92 Syntax can be parsed correctly and used in ShardingSphere.

Enhancement

  1. Support using less-than character(<) and greater-than character(>) for sharding data.
  2. When primary and replica dataSources exist, support executing SELECT FOR UPDATE on primary data source.
  3. Support hint in ShardingSphere-Proxy.
  4. Finish parsing DAL syntax for MySQL.
  5. Make configuration of orchestration compatible between ShardingSphere-JDBC and ShardingSphere-Proxy.

Bug Fixes

  1. Through Bug fix, the feature of encryption becomes much stable and applicable.
  2. Support delete statement with alias.
  3. Check and disable updating sharding column.
  4. Fix wrong type of TINYINT and SMALLINT as INTEGER.

Refactor

  1. Rename optimized module to preprocessor module.
  2. Decouple rewrite core module and sharding/encrypt features.

Change Logs

  1. MILESTONE

4.0.0.RC2

API Changes

  1. Optimize and delete API and configuration item of sharding logic index.
  2. Update the API of encryption to support the encrypted and plain data coexistence.

New Features

  1. Integration of Seata for distributed transaction.
  2. User can do data encryption by using ShardingProxy.
  3. User can use Leaf-segment generator to get distributed ID.
  4. Support Skywalking plugin for application performance management.

Enhancement

  1. Renew modified dataSources, not all the datasources to improve performance for configuration orchestration.
  2. Improve the compatibility of SQL parsing.

Refactor

  1. Remove DatabaseType enum, use dynamic SPI to load data source type.
  2. The parse engine upgrade from the 2nd generation to 3rd.
  3. The Refactoring of SQL rewriting module.

Change Logs

  1. MILESTONE

4.0.0.RC1

Merge all change logs of version 3.1.0.M1, 3.1.0, 3.1.0.1 and 4.0.0.M1. First apache release.

API Changes

  1. Adjust persist structure for orchestration's registry center.
  2. Adjust ShardingSphere-JDBC configuration API.
  3. Change package and maven groupId form io.shardingsphere to org.apache.shardingsphere.
  4. Adjust spring-boot-starter.

New Features

  1. XA Transaction available.
  2. Data encrypt available.
  3. Use PostgreSQL protocol access ShardingSphere-Proxy available.
  4. DISTINCT SQL syntax available.
  5. Broadcast table.
  6. All SQL 100% compatible if route to single data node (MySQL Only).

Change Logs

  1. MILESTONE #3
  2. MILESTONE #4
  3. MILESTONE #5
  4. MILESTONE #6

3.0.0

Milestones

  1. ShardingSphere-Proxy launch. Support the use of ShardingSphere in the form of database to support for MySQL CLI and GUI client

New Features

Core

  1. ISSUE #290 Support batch INSERT
  2. ISSUE #501 Support OR
  3. ISSUE #980 Support DCL
  4. ISSUE #1111 Support MySQL DAL

ShardingSphere-Proxy

  1. ISSUE #902 Support XA transaction
  2. ISSUE #916 Support authorization
  3. ISSUE #936 Support registry center
  4. ISSUE #1046 Support multiple logic databases

Enhancements

Core

  1. ISSUE #373 Support order by ?
  2. ISSUE #610 Route unicast for DQL without table
  3. ISSUE #701 Caching parsed results to improve performance
  4. ISSUE #773 Support sharding and autoincrement key of INSERT without column names
  5. ISSUE #935 Use YAML instead of JSON to store configurations in registry center
  6. ISSUE #1004 Properties can configure for sharding and replica query independent
  7. ISSUE #1205 Execute engine enhancement

ShardingSphere-JDBC

  1. ISSUE #652 Support Spring Boot Starter 2.X
  2. ISSUE #702 Support $->{..} for inline expression
  3. ISSUE #719 Support inject key generator objects by spring namespace
  4. ISSUE #720 Support inject sharding algorithm objects by spring namespace

Sharding-Opentracing

  1. ISSUE #1172 Opentracing enhancement

API changes

  1. ISSUE #1153 Adjust the maven artifactId for orchestration module
  2. ISSUE #1203 Adjust Spring namespace xsd for sharding and replica query
  3. ISSUE #1289 Adjust hint API
  4. ISSUE #1302 Refine package structure
  5. ISSUE #1305 Deprecated and remove sharding-jdbc-transaction-parent module
  6. ISSUE #1382 Remove type configuration in orchestration module

Bug Fixes

Core

  1. ISSUE #569 Failed to parse SQL for Oracle when ROWNUM is not at end
  2. ISSUE #628 Support data type jsonb for PostgreSQL
  3. ISSUE #646 When aliases in SELECT ITEMS correspond to the real column names of GROUP BY or ORDER BY, there is no need to generate derived columns
  4. ISSUE #806 NOT IN parse exception
  5. ISSUE #827 Endless loop for bad SQL like SELECT * FROM table WHERE id IN ()
  6. ISSUE #919 Inline expression with groovy may cause memory leak
  7. ISSUE #993 Fail to parsing PostgreSQL due to the quotation
  8. ISSUE #1015 Support SQL like SELECT id, COUNT(*) FROM table GROUP BY 1,2
  9. ISSUE #1120 Derived columns of GROUP BY / ORDER BY appear in query result
  10. ISSUE #1186 Dead lock may occur on MEMORY_STRICTLY mode when get connection on concurrency environment
  11. ISSUE #1265 RoundRobinReplicaLoadBalanceAlgorithm throw an ArrayIndexOutOfBoundsException when AtomicInteger overflow

ShardingSphere-JDBC

  1. ISSUE #372 Reuse PreparedStatement cause cache of route result do not clean
  2. ISSUE #629 Support transaction isolation on JDBC
  3. ISSUE #735 Unexpected replica datasource routing result when using Round-robin load-balance algorithm in Mybatis
  4. ISSUE #1011 Can't resolve placeholder in Spring Boot YAML configuration

2.0.3

New Features

Core

  1. ISSUE #600 Support TCL

Bug Fixes

Core

  1. ISSUE #540 Support SQL that alias is the keyword
  2. ISSUE #577 Support new line for YAML configuration

ShardingSphere-JDBC

  1. ISSUE #522 Replica database does not need to execute the DDL for replica query

2.0.2

Enhancements

Core

  1. ISSUE #475 Support CREATE INDEX
  2. ISSUE #525 Support DROP INDEX

Bug Fixes

Core

  1. ISSUE #521 ShardingProperties is invalid in YAML configuration
  2. ISSUE #529 Table name capitalization cannot be queried
  3. ISSUE #541 IS NOT NULL parse error
  4. ISSUE #557 When GROUP BY and ORDER BY aliases are inconsistent, stream merging should be used
  5. ISSUE #559 Support parsing numbers beginning with minus and decimal (e.g. -.12)
  6. ISSUE #567 Add escape char for derived columns or alias when using MySQL

ShardingSphere-JDBC

  1. ISSUE #520 Exception is no longer DuplicateKeyException when the unique key conflict

2.0.1

Enhancements

Core

  1. ISSUE #490 Using rownum greater than or equal to or less than or equal to the result of paging is incorrect in Oracle
  2. ISSUE #496 Logical table names in sharding configuration can be case sensitive
  3. ISSUE #497 Close registry center gracefully

Bug Fixes

ShardingSphere-JDBC

  1. ISSUE #489 Uses RuntimeBeanReference to prevent the creation of InnerBean on spring namespace
  2. ISSUE #491 Can't close connection by ResultSet.getStatement().getConnection().close()

2.0.0

Milestones

  1. API adjust. Brand new groupId and artifactId for Maven, package name and spring namespace name. Simplify and enhance API configuration, inline expression fully configuration support
  2. Support spring-boot-starter of ShardingSphere-JDBC
  3. Dynamic configuration. ZooKeeper and etcd can be used as registry to dynamically modify data sources and sharding configurations
  4. Database orchestration. Fusing database access procedures to access databases and disable access to replica databases
  5. ConfigMap support. Predefined metadata can be obtained in the sharding and replica query strategy
  6. Tracking system support. You can view the invocation chain of ShardingSphere-JDBC through sky-walking and other Opentracing based APM systems

Enhancements

Core

  1. ISSUE #386 Support SQL that does not contain table names, such as SELECT 1

ShardingSphere-JDBC

  1. ISSUE #407 Support Hyphen properties for sharding-jdbc-spring-boot-starter
  2. ISSUE #424 Providing SQL overall execution events

Bug Fixes

Core

  1. ISSUE #387 Prevent errors from keywords process when '`' exists in function + column name
  2. ISSUE #419 When SQL is rewritten, it does not determine whether alias is a keyword without the escape character, which results in SQL exception
  3. ISSUE #464 SQL if the varchar type is not closed due to the absence of matching single quotes, and the next varchar in SQL is the wrong SQL of Chinese characters, it will lead to higher use of CPU

ShardingSphere-JDBC

  1. ISSUE #394 Can't only close statement
  2. ISSUE #398 Use Hint routing to shield case sensitivity
  3. ISSUE #404 Sharding-jdbc's spring-boot-starter does not support HikariDataSource
  4. ISSUE #436 Replica query, when the RoundRobin algorithm is configured from the database and MyBatis is used, it can only be routed to the same replica database
  5. ISSUE #452 Sharding of DDL statements to more than one table causes a connection leak
  6. ISSUE #472 Before Connection executes createStatement, it calls getMetaData first and then setAutoCommit can not take effective connection to the database that was created later

1.5.4.1

Bug Fixes

  1. ISSUE #382 The query cannot be completed without sharding rule

1.5.4

Bug Fixes

  1. ISSUE #356 In the Where condition of SQL, the REGEXP operator is compatible with non sharding columns
  2. ISSUE #362 Replica query using PreparedStatement does not invoke the setParameter method to cause errors
  3. ISSUE #370 Error in calling getGeneratedKeys using native self increment primary key
  4. ISSUE #375 Data can not be obtained after paging second pages route to a single node
  5. ISSUE #379 When Mybatis is used to call Connection.getMetaData (), the connection is not close correct

1.5.3

Enhancements

  1. ISSUE #98 Replica query load balancing strategy support configuration
  2. ISSUE #196 Replica query and sharding configuration independence

Bug Fixes

  1. ISSUE #349 Incorrect function of ResultSet.wasNull causes null numeric type in DB to zero
  2. ISSUE #351 Tables that are included in the default data source but not in TableRule configuration are not properly executed
  3. ISSUE #353 In the Where condition of SQL, it is compatible with non sharding columns !=, !> and !< operator
  4. ISSUE #354 In the Where condition of SQL, NOT operators are compatible with non-sharding columns

1.5.2

Milestones

  1. The test engine of quality assurance, each SQL can run 60 test cases of different dimensions

Enhancements

  1. ISSUE #335 Support the GROUP BY + custom function SQL
  2. ISSUE #341 Support ORDER BY xxx NULLS FIRST | LAST statement of Oracle

Bug Fixes

  1. ISSUE #334 Parsing ORDER BY with functions will resolve the following ASC and DESC to the name attribute of OrderItem
  2. ISSUE #335 JOIN parsing is incorrect using the full name association of the table
  3. ISSUE #346 Parsing table name error of DDL statement DROP TABLE IF EXISTS USER

1.5.1

New Features

  1. ISSUE #314 Support DDL type SQL

Changes

  1. ISSUE #327 Close sql.show configuration by default

Bug Fixes

  1. ISSUE #308 Invalid return of database native GeneratedKey
  2. ISSUE #309 ORDER BY and GROUP BY in sub-queries are not included in the analytic context
  3. ISSUE #313 Support <> operator
  4. ISSUE #317 Parameter of LIMIT can not be type of Long
  5. ISSUE #320 SQL rewriting error of GROUP BY + LIMIT
  6. ISSUE #323 Parsing ORDER BY + Aggregation Expression error

1.5.0

Milestones

  1. The new SQL parsing module removes the dependence on Druid. We only need to parse the sharding context, and adopt a "semi understanding" concept for SQL to further improve performance and compatibility, and reduce code complexity
  2. The new SQL rewrite module adds an optimized rewrite module
  3. The new SQL merge module is rebuilt into 3 merging engines: streaming, memory and decorator

New Features

  1. Support Oracle, SQLServer and PostgreSQL
  2. Non functional sub-query support

Enhancements

  1. ISSUE #256 Configurable display sharding execute to SQL log
  2. ISSUE #291 Processing SQL use stream mode that contains only GroupBy

Changes

  1. Simplify the distributed self increasing sequence. Each table is supported by simplifying a multiple self increasing sequence to a single table supporting only a single distributed self increasing sequence, and no longer supporting workerID settings through environment variables
  2. Remove support for OR

Bug Fixes

  1. ISSUE #239 LIMIT routes to multiple query result sets. If there is only one result set that is not empty, the result of paging is incorrect
  2. ISSUE #263 Sharding and logical table configuration can be case insensitive
  3. ISSUE #292 When the memory mode handles GROUP BY statement, if there is paging information, it needs to be rewritten
  4. ISSUE #295 LIMIT 0 does not filter the result set according to paging restrictions

1.4.2

Enhancements

  1. ISSUE #219 Thread performance optimization
  2. ISSUE #215 Aggregated result set of stream sort StreamingOrderByReducerResultSet performance optimization
  3. ISSUE #161 When the result sets are merged, heap sort can be used to improve performance

Bug Fixes

  1. ISSUE #212 More meaningful hints for missing data source rules
  2. ISSUE #214 table_name.column_name in (?,?) in WHERE can't parser expression
  3. ISSUE #180 Batch execution of Update return inaccuracy
  4. ISSUE #225 The last character of automatic generation of Id is not 0

1.4.1

Enhancements

  1. ISSUE #191 Generating KeyGenerator of workerId based on IP of host
  2. ISSUE #192 Get workerId's KeyGenerator based on HOSTNAME's digital suffix
  3. ISSUE #210 Routing to single database and single table to remove supplementary SQL statement fragments

Bug Fixes

  1. ISSUE #194 Some component exceptions in the close method of Connection, Statement, ResultSet and other interfaces cause the close method of another component to be not invoked
  2. ISSUE #199 Sharding and reuse PreparedStatement object cause route error
  3. ISSUE #201 Event transmission missing before batch operation execution
  4. ISSUE #203 Merge events sent by the batch operation
  5. ISSUE #209 Parallel execution of multiple limit queries leads to IndexOutOfBoundsException

1.4.0

Enhancements

Automatic generation key implementation, including

  1. ISSUE #162 Implementation of distributed primary key algorithm
  2. ISSUE #163 Acquisition of a self increasing sequence of JDBC interfaces
  3. ISSUE #171 Sharding-jdbc-core coordination automatic generation sequence transformation
  4. ISSUE #172 The configuration of YAML and Spring supports the self increasing sequence

Bug Fixes

  1. ISSUE #176 The wasNull flag of AbstractMemoryResultSet is not reset in time

1.3.3

Enhancements

  1. ISSUE #59 PreparedStatement can call the correct underlying set method according to the parameter type when setting parameters

Bug Fixes

  1. ISSUE #149 When INSERT IGNORE INTO, if the data is duplicated, the value returned to -1 when ignored, and it should be returned to 0
  2. ISSUE #118 In the same thread, DQL is executed first, then DML is executed, and DML operation is executed from the replica database
  3. ISSUE #122 In cases where connections are not available (such as network interruption), transactions should be interrupted rather than retry
  4. ISSUE #152 PreparedStatement's cache causes an array out of bound
  5. ISSUE #150 With the latest SQLServer jdbc driver compatibility problem, Product Name should be changed from SQLServer to Microsoft SQL Server
  6. ISSUE #166 Druid data source stat filter multi-thread error reporting should be added to database connection level synchronization

1.3.2

Enhancements

  1. ISSUE #79 Optimizes limit for only one target table, does not modify limit offset

Bug Fixes

  1. ISSUE #36 ShardingPreparedStatement cannot set parameters repeatedly
  2. ISSUE #114 When ShardingPreparedStatement performs batch tasks, it repeatedly analyzes SQL and leads to OOM
  3. ISSUE #33 According to the MySQL document, queries similar to limit 100 and -1 format are not supported
  4. ISSUE #124 The return value of com.dangdang.ddframe.rdb.sharding.jdbc.adapter.AbstractStatementAdapter.getUpdateCount does not conform to the JDBC specification
  5. ISSUE #141 Multi-thread executor parameter setting failure

1.3.1

Enhancements

  1. ISSUE #91 Open support for Statement.getGeneratedKeys can return the original database self increase primary key
  2. ISSUE #92 Query DQL statement event sending

Bug Fixes

  1. ISSUE #89 Use replica query with sharding hint leads to conflict
  2. ISSUE #95 Write operations in the same thread read from the primary database changed to the same thread and within the same connection

1.3.0

New Features

  1. ISSUE #85 New feature for replica query

Enhancements

  1. ISSUE #82 TableRule can import the dataSourceName attribute to specify the data source corresponding to the TableRule
  2. ISSUE #88 Release restrictions on other databases, support standard SQL, do not support personalized paging statements

Bug Fixes

  1. ISSUE #81 Associative table query uses OR query condition to resolve the result exceptions

1.2.1

Refactor

  1. ISSUE #60 API adjust, remove ShardingDataSource, use factory instead
  2. ISSUE #76 ShardingRule and TableRule change to Builder pattern
  3. ISSUE #77 ShardingRule and TableRule change to Builder pattern

Enhancements

  1. ISSUE #61 Add the logical table name to the ShardingValue class
  2. ISSUE #66 Statement on the JDBC tier supports get/set MaxFieldSize, MaxRows and QueryTimeout
  3. ISSUE #72 Batch inserts supporting select union all
  4. ISSUE #78 Simplifying sharding only configuration, without configuring logical table and real table correspondence
  5. ISSUE #80 Simplifying the configuration that does not sharding, specifying the default data source, do not need configure TableRule

Bug Fixes

  1. ISSUE #63 No table name or table alias is added to the ORDER BY and GROUP BY derivation columns
  2. ISSUE #65 Performance enhancement for parsing condition context
  3. ISSUE #67 The soft transaction log cannot be deleted when routed to multiple tables
  4. ISSUE #71 Routing single sharding key by OFFSET of LIMIT error
  5. ISSUE #75 MemoryTransactionLogStorage retry times update concurrency problem

1.2.0

New Features

  1. ISSUE #53 The relationship between the real table and the logical table is not configured, and the real table is dynamically calculated by the sharding algorithm
  2. ISSUE #58 Soft transaction: the initial version of the best effort type

Refactor

  1. ISSUE #49 Adjust the property configuration
  2. ISSUE #51 Refactor of Hint interface

Bug Fixes

  1. ISSUE #43 The yaml file contains Chinese, and the operating system mode is not UTF-8 encoding, resulting in yaml can not be parsed
  2. ISSUE #48 Yaml file is not closed after reading
  3. ISSUE #57 At the analytic level, we can identify subqueries to ensure that the behavior of supplementary columns can be accurately positioned

1.1.0

New Features

  1. ISSUE #40 Support YAML configuration
  2. ISSUE #41 Support Spring namespace configuration
  3. ISSUE #42 Support inline expression configuration

Bug Fixes

  1. ISSUE #25 The problem of repeated results will appear under the OR expression

1.0.1

Enhancements

  1. ISSUE #39 Support the use of Hint method to register the key value to SQL routing

Bug Fixes

  1. ISSUE #11 The count function returns incorrectly without aliases
  2. ISSUE #13 The Insert statement does not write column names or write column names but column names do not contain sharding fields, occur broadcast route
  3. ISSUE #16 For now a new connection pool is executed every time SQL is executed. Instead, each ShardingDataSource object should be changed to share a pool
  4. ISSUE #18 When query Count, getObject() throws Exception: Unsupported data type: Object
  5. ISSUE #19 In SUM and AVG functions, merger is not executed if aliases are not added, and null pointer exception fired if aliases are added
  6. ISSUE #38 The compatibility between JPA and ShardingSphere-JDBC. JPA automatically add the column aliases of SELECT, resulting in ORDER BY obtaining ResultSet data only by aliases rather than column names

1.0.0

  1. Initial version