Skip to content

Latest commit

 

History

History
305 lines (167 loc) · 17.6 KB

CHANGELOG.md

File metadata and controls

305 lines (167 loc) · 17.6 KB

Sonic Changelog

1.4.9 (2024-06-16)

Changes

  • Update Rust code style to conform to new rustc requirements (preventing builds on rustc 1.79.0 and further) [@jaseemabid, #321].

1.4.8 (2023-12-14)

Changes

  • Pull out the arm64 platform from the Docker image, since it does not build in acceptable time via GitHub Actions due to using QEMU emulation (will wait that GitHub Actions provides a native arm64 runner) [@valeriansaliou].

1.4.7 (2023-12-14)

Bug Fixes

  • Fixed non-working arm64 builds due to hardcoded x86_64-unknown-linux-gnu Rust target in the Dockerfile [@valeriansaliou].

1.4.6 (2023-12-14)

New Features

  • The Docker image is now also available for the arm64 platform, in addition to amd64 [@PovilasID, #310].

1.4.5 (2023-12-11)

Bug Fixes

  • Fixed an issue where system clock can move back to the past on a virtualized system, resulting in client threads entering a crash loop due to mutex poisoning [@valeriansaliou].

1.4.4 (2023-12-08)

Bug Fixes

  • Fixed rocksdb not building due to a rust-bindgen version which was not compatible with clang version 16 [@anthonyroussel, #316].

Changes

  • Dependencies have been bumped to latest versions (namely: rocksdb, toml, regex-syntax, hashbrown, lindera-core, lindera-dictionary, lindera-tokenizer) [@valeriansaliou].

1.4.3 (2023-09-04)

Changes

  • Publish .deb packages for Debian 12 on x86_64 architecture [@valeriansaliou].

1.4.2 (2023-09-04)

Changes

  • Produce glibc builds from GitHub Actions whenever a new Sonic version gets released [@valeriansaliou].
  • Pull out tokenizer-japanese from the default features, as it x10 the final binary size [@valeriansaliou].

1.4.1 (2023-08-12)

New Features

  • Added support for Japanese word segmentation in tokenizer (note that as this adds quite some size overhead to the final binary size, the feature tokenizer-japanese can be disabled when building Sonic) [@nmkj-io, #311].

1.4.0 (2022-10-20)

Bug Fixes

Changes

  • Replaced Docker source image from Debian Slim to lighter Google distroless image [@0x0x1, #282].

New Features

  • Added an index enumeration LIST command to Sonic Channel [@trkohler, #293].

1.3.5 (2022-07-10)

Bug Fixes

  • Rolled back rocksdb version, as the latest version does not link properly in --release mode [@valeriansaliou].

1.3.4 (2022-07-10)

Changes

  • Dependencies have been bumped to latest versions (namely: rocksdb, clap, regex) [@valeriansaliou].

1.3.3 (2022-07-07)

Changes

  • Dependencies have been bumped to latest versions (namely: hashbrown, whatlang, regex) [@valeriansaliou].
  • Moved the release pipeline to GitHub Actions [@valeriansaliou].

New Features

1.3.2 (2021-11-09)

Bug Fixes

Changes

  • Code has been formatted according to clippy recommendations. This does not change the way Sonic behaves [@pleshevskiy, #233].

New Features

  • Added support for Chinese word segmentation in tokenizer (note that as this adds quite some size overhead to the final binary size, the feature tokenizer-chinese can be disabled when building Sonic) [@vincascm, #209].

1.3.1 (2021-11-02)

Changes

  • Apple Silicon is now supported [@valeriansaliou].
  • Added Norwegian stopwords [@mikalv, #236].
  • Added Catalan stopwords [@coopanio, #227].
  • Dependencies have been bumped to latest versions (namely: rocksdb, fst-levenshtein, fst-regex, hashbrown, whatlang, byteorder, rand) [@valeriansaliou].

Deprecations

1.3.0 (2020-06-27)

Changes

1.2.4 (2020-06-25)

Bug Fixes

  • Fixed multiple deadlocks, which where not noticed in practice by running Sonic at scale, but that are still theoretically possible [@BurtonQin, #213, #211].

Changes

New Features

  • Added a release script, with cross-compilation capabilities (currently for the x86_64 architecture, dynamically linked against GNU libraries) [@valeriansaliou, 961bab9].

1.2.3 (2019-10-14)

Changes

  • RocksDB compression algorithm has been changed from LZ4 to Zstandard, for a slightly better compression ratio, and much better read/write performance; this will be used for new SST files only [@valeriansaliou, cd4cdfb].
  • Dependencies have been bumped to latest versions (namely: rocksdb) [@valeriansaliou, cd4cdfb].

1.2.2 (2019-07-12)

Bug Fixes

  • Fixed a regression on optional configuration values not working anymore, due to an issue in the environment variable reading system introduced in v1.2.1 [@valeriansaliou, #155].

Changes

  • Optimized some aspects of FST consolidation and pending operations management [@valeriansaliou, #156].

1.2.1 (2019-07-08)

Changes

  • FST graph consolidation is now able to ignore new words when the graph is over configured limits, which are set with the new store.fst.graph.max_size and store.fst.graph.max_words configuration variables [@valeriansaliou, 53db9c1].
  • An integration testing infrastructure has been added to the Sonic automated test suite [@vilunov, #154].
  • Configuration values can now be sourced from environment variables, using the ${env.VARIABLE} syntax in config.cfg [@perzanko, #148].
  • Dependencies have been bumped to latest versions (namely: rand, radix and hashbrown) [@valeriansaliou, c1b1f54].

1.2.0 (2019-05-03)

Bug Fixes

  • Fixed a rare deadlock occurring when 3 concurrent operations get executed on different threads for the same collection, in the following timely order: PUSH then FLUSHB then PUSH [@valeriansaliou, d96546b].

Changes

  • Reworked the KV store manager to perform periodic memory flushes to disk, thus reducing startup time [@valeriansaliou, 6713488].
  • Stop accepting Sonic Channel commands when shutting down Sonic [@valeriansaliou, #131].

New Features

  • Introduced a server statistics INFO command to Sonic Channel [@valeriansaliou, #70].
  • Added the ability to disable the lexer for a command with the command modifier LANG(none) [@valeriansaliou, #108].
  • Added a backup and restore system for both KV and FST stores, which can be triggered over Sonic Channel with TRIGGER backup and TRIGGER restore [@valeriansaliou, #5].
  • Added the ability to disable KV store WAL (Write-Ahead Log) with the write_ahead_log option, which helps limit write wear on heavily loaded SSD-backed servers [@valeriansaliou, #130].

1.1.9 (2019-03-29)

Bug Fixes

  • RocksDB has been bumped to v5.18.3, which fixes a dead-lock occurring in RocksDB at scale when a compaction task is ran under heavy disk writes (ie. disk flushes). This dead-lock was causing Sonic to stop responding to any command issued for the frozen collection. This dead-lock was due to a bug in RocksDB internals (not originating from Sonic itself) [@baptistejamin, 19c4a10].

Changes

  • Reworked the FLUSHB command internals, which now use the atomic delete_range() operation provided by RocksDB v5.18 [@valeriansaliou, 660f8b7].

New Features

  • Added the LANG(<locale>) command modifier for QUERY and PUSH, that lets a Sonic Channel client force a text locale (instead of letting the lexer system guess the text language) [@valeriansaliou, #75].
  • The FST word lookup system, used by the SUGGEST command, now support all scripts via a restricted Unicode range forward scan [@valeriansaliou, #64].

1.1.8 (2019-03-27)

Bug Fixes

  • A store acquire lock has been added to prevent 2 concurrent threads from opening the same collection at the same time [@valeriansaliou, 2628077].

1.1.7 (2019-03-27)

Bug Fixes

  • A superfluous mutex was removed from KV and FST store managers, in an attempt to solve a rare dead-lock occurring on high-traffic Sonic setups in the KV store [@valeriansaliou, 60566d2].

1.1.6 (2019-03-27)

Changes

  • Reverted changes made in v1.1.5 regarding the open files rlimit, as this can be set from outside Sonic [@valeriansaliou, f6400c6].
  • Added Chinese Traditional stopwords [@dsewnr, #87].

Bug Fixes

  • Improved the way database locking is handled when calling a pool janitor; this prevents potential dead-locks under high load [@valeriansaliou, fa78372].

1.1.5 (2019-03-27)

New Features

  • Added the server.limit_open_files configuration variable to allow configuring rlimit [@valeriansaliou].

1.1.4 (2019-03-27)

Changes

New Features

  • Automatically adjust rlimit for the process to the hard limit allowed by the system (allows opening more FSTs in parallel) [@valeriansaliou].

1.1.3 (2019-03-25)

Changes

  • Limit the size of words that can hit against the FST graph, as the FST gets slower for long words [@valeriansaliou, #81].

Bug Fixes

  • Rework Sonic Channel buffer management using a VecDeque (Sonic should now work better in harsh network environments) [@valeriansaliou, 1c2b9c8].

1.1.2 (2019-03-24)

Changes

  • FST graph consolidation locking strategy has been improved even further, based on issues with the previous rework we have noticed at scale in production (now, consolidation locking is done at a lower-priority relative to actual queries and pushes to the index) [@valeriansaliou, #68].

1.1.1 (2019-03-24)

Changes

  • FST graph consolidation locking strategy has been reworked as to allow queries to be executed lock-free when the FST consolidate task takes a lot of time (previously, queries were being deferred due to an ongoing FST consolidate task) [@valeriansaliou, #68].
  • Removed special license clause introduced in v1.0.2, Sonic is full MPL 2.0 now. [@valeriansaliou]

1.1.0 (2019-03-21)

Breaking Changes

  • Change how buckets are stored in a KV-based collection (nest them in the same RocksDB database; this is much more efficient on setups with a large number of buckets - v1.1.0 is incompatible with the v1.0.0 KV database format) [@valeriansaliou].

Changes

1.0.2 (2019-03-20)

Changes

  • Re-license from MPL 2.0 to SOSSL 1.0 (Sonic has a special license clause) [@valeriansaliou].

1.0.1 (2019-03-19)

Changes

  • Added automated benchmarks (can be ran via cargo bench --features benchmark) [@valeriansaliou].
  • Reduced the time to query the search index by 50% via optimizations (in multiple methods, eg. the lexer) [@valeriansaliou].

1.0.0 (2019-03-18)

New Features