Merge remote-tracking branch 'origin/develop' into accountpermission

API-CHANGELOG.md

@@ -83,11 +83,13 @@ The [commandline](https://xrpl.org/docs/references/http-websocket-apis/api-conve
 
 The `network_id` field was added in the `server_info` response in version 1.5.0 (2019), but it is not returned in [reporting mode](https://xrpl.org/rippled-server-modes.html#reporting-mode). However, use of reporting mode is now discouraged, in favor of using [Clio](https://github.com/XRPLF/clio) instead.
 
-## XRP Ledger server version 2.2.0
+## XRP Ledger server version 2.4.0
 
-The following is a non-breaking addition to the API.
+### Addition in 2.4
 
-- The `feature` method now has a non-admin mode for users. (It was previously only available to admin connections.) The method returns an updated list of amendments, including their names and other information. ([#4781](https://github.com/XRPLF/rippled/pull/4781))
+- `ledger_entry`: `state` is added an alias for `ripple_state`.
+
+## XRP Ledger server version 2.3.0
 
 ### Breaking change in 2.3
 
@@ -105,6 +107,12 @@ The following additions are non-breaking (because they are purely additive).
 - In `Payment` transactions, `DeliverMax` has been added. This is a replacement for the `Amount` field, which should not be used. Typically, the `delivered_amount` (in transaction metadata) should be used. To ease the transition, `DeliverMax` is present regardless of API version, since adding a field is non-breaking.
 - API version 2 has been moved from beta to supported, meaning that it is generally available (regardless of the `beta_rpc_api` setting).
 
+## XRP Ledger server version 2.2.0
+
+The following is a non-breaking addition to the API.
+
+- The `feature` method now has a non-admin mode for users. (It was previously only available to admin connections.) The method returns an updated list of amendments, including their names and other information. ([#4781](https://github.com/XRPLF/rippled/pull/4781))
+
 ## XRP Ledger server version 1.12.0
 
 [Version 1.12.0](https://github.com/XRPLF/rippled/releases/tag/1.12.0) was released on Sep 6, 2023. The following additions are non-breaking (because they are purely additive).

CMakeLists.txt

@@ -73,6 +73,7 @@ set(SECP256K1_INSTALL TRUE)
 add_subdirectory(external/secp256k1)
 add_library(secp256k1::secp256k1 ALIAS secp256k1)
 add_subdirectory(external/ed25519-donna)
+add_subdirectory(external/antithesis-sdk)
 find_package(gRPC REQUIRED)
 find_package(lz4 REQUIRED)
 # Target names with :: are not allowed in a generator expression.

CONTRIBUTING.md

@@ -343,6 +343,68 @@ pip3 install pre-commit
 pre-commit install
 ```
 
+## Contracts and instrumentation
+
+We are using [Antithesis](https://antithesis.com/) for continuous fuzzing,
+and keep a copy of [Antithesis C++ SDK](https://github.com/antithesishq/antithesis-sdk-cpp/)
+in `external/antithesis-sdk`. One of the aims of fuzzing is to identify bugs
+by finding external conditions which cause contracts violations inside `rippled`.
+The contracts are expressed as `ASSERT` or `UNREACHABLE` (defined in
+`include/xrpl/beast/utility/instrumentation.h`), which are effectively (outside
+of Antithesis) wrappers for `assert(...)` with added name. The purpose of name
+is to provide contracts with stable identity which does not rely on line numbers.
+
+When `rippled` is built with the Antithesis instrumentation enabled
+(using `voidstar` CMake option) and ran on the Antithesis platform, the
+contracts become
+[test properties](https://antithesis.com/docs/using_antithesis/properties.html);
+otherwise they are just like a regular `assert`.
+To learn more about Antithesis, see
+[How Antithesis Works](https://antithesis.com/docs/introduction/how_antithesis_works.html)
+and [C++ SDK](https://antithesis.com/docs/using_antithesis/sdk/cpp/overview.html#)
+
+We continue to use the old style `assert` or `assert(false)` in certain
+locations, where the reporting of contract violations on the Antithesis
+platform is either not possible or not useful.
+
+For this reason:
+* The locations where `assert` or `assert(false)` contracts should continue to be used:
+  * `constexpr` functions
+  * unit tests i.e. files under `src/test`
+  * unit tests-related modules (files under `beast/test` and `beast/unit_test`)
+* Outside of the listed locations, do not use `assert`; use `ASSERT` instead,
+  giving it unique name, with the short description of the contract.
+* Outside of the listed locations, do not use `assert(false)`; use
+  `UNREACHABLE` instead, giving it unique name, with the description of the
+  condition being violated
+* The contract name should start with a full name (including scope) of the
+  function, optionally a named lambda, followed by a colon ` : ` and a brief
+  (typically at most five words) description. `UNREACHABLE` contracts
+  can use slightly longer descriptions. If there are multiple overloads of the
+  function, use common sense to balance both brevity and unambiguity of the
+  function name. NOTE: the purpose of name is to provide stable means of
+  unique identification of every contract; for this reason try to avoid elements
+  which can change in some obvious refactors or when reinforcing the condition.
+* Contract description typically (except for `UNREACHABLE`) should describe the
+  _expected_ condition, as in "I assert that _expected_ is true".
+* Contract description for `UNREACHABLE` should describe the _unexpected_
+  situation which caused the line to have been reached.
+* Example good name for an
+  `UNREACHABLE` macro `"Json::operator==(Value, Value) : invalid type"`; example
+  good name for an `ASSERT` macro `"Json::Value::asCString : valid type"`.
+* Example **bad** name
+  `"RFC1751::insert(char* s, int x, int start, int length) : length is greater than or equal zero"`
+  (missing namespace, unnecessary full function signature, description too verbose).
+  Good name: `"ripple::RFC1751::insert : minimum length"`.
+* In **few** well-justified cases a non-standard name can be used, in which case a
+  comment should be placed to explain the rationale (example in `contract.cpp`)
+* Do **not** rename a contract without a good reason (e.g. the name no longer
+  reflects the location or the condition being checked)
+* Do not use `std::unreachable`
+* Do not put contracts where they can be violated by an external condition
+  (e.g. timing, data payload before mandatory validation etc.) as this creates
+  bogus bug reports (and causes crashes of Debug builds)
+
 ## Unit Tests
 To execute all unit tests:
 

RELEASENOTES.md

@@ -6,6 +6,79 @@ This document contains the release notes for `rippled`, the reference server imp
 
 Have new ideas? Need help with setting up your node? [Please open an issue here](https://github.com/xrplf/rippled/issues/new/choose).
 
+# Version 2.3.0
+
+Version 2.3.0 of `rippled`, the reference server implementation of the XRP Ledger protocol, is now available. This release includes 8 new amendments, including Multi-Purpose Tokens, Credentials, Clawback support for AMMs, and the ability to make offers as part of minting NFTs. Additionally, this release includes important fixes for stability, so server operators are encouraged to upgrade as soon as possible.
+
+
+## Action Required
+
+If you run an XRP Ledger server, upgrade to version 2.3.0 as soon as possible to ensure service continuity.
+
+Additionally, new amendments are now open for voting according to the XRP Ledger's [amendment process](https://xrpl.org/amendments.html), which enables protocol changes following two weeks of >80% support from trusted validators. The exact time that protocol changes take effect depends on the voting decisions of the decentralized network.
+
+## Full Changelog
+
+### Amendments
+
+The following amendments are open for voting with this release:
+
+- **XLS-70 Credentials** - Users can issue Credentials on the ledger and use Credentials to pre-approve incoming payments when using Deposit Authorization instead of individually approving payers. ([#5103](https://github.com/XRPLF/rippled/pull/5103))
+    - related fix: #5189 (https://github.com/XRPLF/rippled/pull/5189)
+- **XLS-33 Multi-Purpose Tokens** - A new type of fungible token optimized for institutional DeFi including stablecoins. ([#5143](https://github.com/XRPLF/rippled/pull/5143))
+- **XLS-37 AMM Clawback** - Allows clawback-enabled tokens to be used in AMMs, with appropriate guardrails. ([#5142](https://github.com/XRPLF/rippled/pull/5142))
+- **XLS-52 NFTokenMintOffer** - Allows creating an NFT sell offer as part of minting a new NFT. ([#4845](https://github.com/XRPLF/rippled/pull/4845))
+- **fixAMMv1_2** - Fixes two bugs in Automated Market Maker (AMM) transaction processing. ([#5176](https://github.com/XRPLF/rippled/pull/5176))
+- **fixNFTokenPageLinks** - Fixes a bug that can cause NFT directories to have missing links, and introduces a transaction to repair corrupted ledger state. ([#4945](https://github.com/XRPLF/rippled/pull/4945))
+- **fixEnforceNFTokenTrustline** - Fixes two bugs in the interaction between NFT offers and trust lines. ([#4946](https://github.com/XRPLF/rippled/pull/4946))
+- **fixInnerObjTemplate2** - Standardizes the way inner objects are enforced across all transaction and ledger data. ([#5047](https://github.com/XRPLF/rippled/pull/5047))
+
+The following amendment is partially implemented but not open for voting:
+
+- **InvariantsV1_1** - Adds new invariants to ensure transactions process as intended, starting with an invariant to ensure that ledger entries owned by an account are deleted when the account is deleted. ([#4663](https://github.com/XRPLF/rippled/pull/4663))
+
+### New Features
+
+- Allow configuration of SQLite database page size. ([#5135](https://github.com/XRPLF/rippled/pull/5135), [#5140](https://github.com/XRPLF/rippled/pull/5140))
+- In the `libxrpl` C++ library, provide a list of known amendments. ([#5026](https://github.com/XRPLF/rippled/pull/5026)) 
+
+### Deprecations
+
+- History Shards are removed. ([#5066](https://github.com/XRPLF/rippled/pull/5066))
+- Reporting mode is removed. ([#5092](https://github.com/XRPLF/rippled/pull/5092))
+
+For users wanting to store more ledger history, it is recommended to run a Clio server instead.
+
+### Bug fixes
+
+- Fix a crash in debug builds when amm_info request contains an invalid AMM account ID. ([#5188](https://github.com/XRPLF/rippled/pull/5188))
+- Fix a crash caused by a race condition in peer-to-peer code. ([#5071](https://github.com/XRPLF/rippled/pull/5071))
+- Fix a crash in certain situations
+- Fix several bugs in the book_changes API method. ([#5096](https://github.com/XRPLF/rippled/pull/5096))
+- Fix bug triggered by providing an invalid marker to the account_nfts API method. ([#5045](https://github.com/XRPLF/rippled/pull/5045))
+- Accept lower-case hexadecimal in compact transaction identifier (CTID) parameters in API methods. ([#5049](https://github.com/XRPLF/rippled/pull/5049))
+- Disallow filtering by types that an account can't own in the account_objects API method. ([#5056](https://github.com/XRPLF/rippled/pull/5056))
+- Fix error code returned by the feature API method when providing an invalid parameter. ([#5063](https://github.com/XRPLF/rippled/pull/5063))
+- (API v3) Fix error code returned by amm_info when providing invalid parameters. ([#4924](https://github.com/XRPLF/rippled/pull/4924))
+
+### Other Improvements
+
+- Adds a new default hub, hubs.xrpkuwait.com, to the config file and bootstrapping code. ([#5169](https://github.com/XRPLF/rippled/pull/5169))
+- Improve error message when commandline interface fails with `rpcInternal` because there was no response from the server. ([#4959](https://github.com/XRPLF/rippled/pull/4959))
+- Add tools for debugging specific transactions via replay. ([#5027](https://github.com/XRPLF/rippled/pull/5027), [#5087](https://github.com/XRPLF/rippled/pull/5087))
+- Major reorganization of source code files. ([#4997](https://github.com/XRPLF/rippled/pull/4997))
+- Add new unit tests. ([#4886](https://github.com/XRPLF/rippled/pull/4886))
+- Various improvements to build tools and contributor documentation. ([#5001](https://github.com/XRPLF/rippled/pull/5001), [#5028](https://github.com/XRPLF/rippled/pull/5028), [#5052](https://github.com/XRPLF/rippled/pull/5052), [#5091](https://github.com/XRPLF/rippled/pull/5091), [#5084](https://github.com/XRPLF/rippled/pull/5084), [#5120](https://github.com/XRPLF/rippled/pull/5120), [#5010](https://github.com/XRPLF/rippled/pull/5010). [#5055](https://github.com/XRPLF/rippled/pull/5055), [#5067](https://github.com/XRPLF/rippled/pull/5067), [#5061](https://github.com/XRPLF/rippled/pull/5061), [#5072](https://github.com/XRPLF/rippled/pull/5072), [#5044](https://github.com/XRPLF/rippled/pull/5044) )
+- Various code cleanup and refactoring. ([#4509](https://github.com/XRPLF/rippled/pull/4509), [#4521](https://github.com/XRPLF/rippled/pull/4521), [#4856](https://github.com/XRPLF/rippled/pull/4856), [#5190](https://github.com/XRPLF/rippled/pull/5190), [#5081](https://github.com/XRPLF/rippled/pull/5081), [#5053](https://github.com/XRPLF/rippled/pull/5053), [#5058](https://github.com/XRPLF/rippled/pull/5058), [#5122](https://github.com/XRPLF/rippled/pull/5122), [#5059](https://github.com/XRPLF/rippled/pull/5059), [#5041](https://github.com/XRPLF/rippled/pull/5041))
+
+
+Bug Bounties and Responsible Disclosures:
+
+We welcome reviews of the `rippled` code and urge researchers to responsibly disclose any issues they may find.
+
+To report a bug, please send a detailed report to: <bugs@xrpl.org>
+
+
 # Version 2.2.3
 
 Version 2.2.3 of `rippled`, the reference server implementation of the XRP Ledger protocol, is now available. This release fixes a problem that can cause full-history servers to run out of space in their SQLite databases, depending on configuration. There are no new amendments in this release.

cmake/RippledCompiler.cmake

@@ -120,7 +120,7 @@ else ()
   target_link_libraries (common
     INTERFACE
       -rdynamic
-      $<$<BOOL:${is_linux}>:-Wl,-z,relro,-z,now>
+      $<$<BOOL:${is_linux}>:-Wl,-z,relro,-z,now,--build-id>
       # link to static libc/c++ iff:
       #   * static option set and
       #   * NOT APPLE (AppleClang does not support static libc/c++) and
@@ -131,6 +131,17 @@ else ()
       >)
 endif ()
 
+# Antithesis instrumentation will only be built and deployed using machines running Linux.
+if (voidstar)
+  if (NOT CMAKE_BUILD_TYPE STREQUAL "Debug")
+    message(FATAL_ERROR "Antithesis instrumentation requires Debug build type, aborting...")
+  elseif (NOT is_linux)
+    message(FATAL_ERROR "Antithesis instrumentation requires Linux, aborting...")
+  elseif (NOT (is_clang AND CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 16.0))
+    message(FATAL_ERROR "Antithesis instrumentation requires Clang version 16 or later, aborting...")
+  endif ()
+endif ()
+
 if (use_mold)
   # use mold linker if available
   execute_process (

cmake/RippledCore.cmake

@@ -74,6 +74,7 @@ target_compile_definitions(xrpl.libxrpl
 target_compile_options(xrpl.libxrpl
   PUBLIC
     $<$<BOOL:${is_gcc}>:-Wno-maybe-uninitialized>
+    $<$<BOOL:${voidstar}>:-DENABLE_VOIDSTAR>
 )
 
 target_link_libraries(xrpl.libxrpl
@@ -89,6 +90,7 @@ target_link_libraries(xrpl.libxrpl
     secp256k1::secp256k1
     xrpl.libpb
     xxHash::xxhash
+    $<$<BOOL:${voidstar}>:antithesis-sdk-cpp>
 )
 
 if(xrpld)
@@ -129,6 +131,19 @@ if(xrpld)
     target_compile_definitions(rippled PRIVATE RIPPLED_RUNNING_IN_CI)
   endif ()
 
+  if(voidstar)
+    target_compile_options(rippled
+      PRIVATE
+        -fsanitize-coverage=trace-pc-guard
+    )
+    # rippled requires access to antithesis-sdk-cpp implementation file
+    # antithesis_instrumentation.h, which is not exported as INTERFACE
+    target_include_directories(rippled
+      PRIVATE
+        ${CMAKE_SOURCE_DIR}/external/antithesis-sdk
+    )
+  endif()
+
   # any files that don't play well with unity should be added here
   if(tests)
     set_source_files_properties(

cmake/RippledInstall.cmake

@@ -10,6 +10,7 @@ install (
     ripple_boost
     xrpl.libpb
     xrpl.libxrpl
+    antithesis-sdk-cpp
   EXPORT RippleExports
   LIBRARY DESTINATION lib
   ARCHIVE DESTINATION lib

cmake/RippledSettings.cmake

@@ -18,6 +18,9 @@ if(unity)
     set(CMAKE_UNITY_BUILD_BATCH_SIZE 15 CACHE STRING "")
   endif()
 endif()
+if(is_clang AND is_linux)
+  option(voidstar "Enable Antithesis instrumentation." OFF)
+endif()
 if(is_gcc OR is_clang)
   option(coverage "Generates coverage info." OFF)
   option(profile "Add profiling flags" OFF)

external/README.md

@@ -6,6 +6,7 @@ The Conan recipes include patches we have not yet pushed upstream.
 
 | Folder          | Upstream                                     | Description |
 |:----------------|:---------------------------------------------|:------------|
+| `antithesis-sdk`| [Project](https://github.com/antithesishq/antithesis-sdk-cpp/) | [Antithesis](https://antithesis.com/docs/using_antithesis/sdk/cpp/overview.html) SDK for C++ |
 | `ed25519-donna` | [Project](https://github.com/floodyberry/ed25519-donna) | [Ed25519](http://ed25519.cr.yp.to/) digital signatures |
 | `rocksdb`       | [Recipe](https://github.com/conan-io/conan-center-index/tree/master/recipes/rocksdb) | Fast key/value database. (Supports rotational disks better than NuDB.) |
 | `secp256k1`     | [Project](https://github.com/bitcoin-core/secp256k1)    | ECDSA digital signatures using the **secp256k1** curve |

external/antithesis-sdk/.clang-format

@@ -0,0 +1,3 @@
+---
+DisableFormat: true
+SortIncludes: false

external/antithesis-sdk/CMakeLists.txt

@@ -0,0 +1,17 @@
+cmake_minimum_required(VERSION 3.25)
+
+# Note, version set explicitly by rippled project
+project(antithesis-sdk-cpp VERSION 0.4.2 LANGUAGES CXX)
+
+add_library(antithesis-sdk-cpp INTERFACE antithesis_sdk.h)
+
+# Note, both sections below created by rippled project
+target_include_directories(antithesis-sdk-cpp INTERFACE
+  $<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>
+  $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}>
+)
+
+install(
+  FILES antithesis_sdk.h
+  DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}"
+)

external/antithesis-sdk/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Antithesis
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

external/antithesis-sdk/README.md

@@ -0,0 +1,8 @@
+# Antithesis C++ SDK
+
+This library provides methods for C++ programs to configure the [Antithesis](https://antithesis.com) platform. It contains three kinds of functionality:
+* Assertion macros that allow you to define test properties about your software or workload.
+* Randomness functions for requesting both structured and unstructured randomness from the Antithesis platform.
+* Lifecycle functions that inform the Antithesis environment that particular test phases or milestones have been reached.
+
+For general usage guidance see the [Antithesis C++ SDK Documentation](https://antithesis.com/docs/using_antithesis/sdk/cpp/overview.html)