Merge pull request #13 from Skydev0h/feature/optimisation

Wallet 5 revision 2 - features and optimisations

Contest.md

@@ -0,0 +1,26 @@
+## Contest note!
+
+<div align="center">
+<img alt="Contest logo" src="contest.png" height="280" width="280">
+</div>
+
+**Because of the extreme amount of optimizations, developer's discretion is advised!** *Evil laugh*
+
+The build system is the same as in the original Wallet V5, **no security features have been sacrificed**
+for performance improvements, that is - there are **practically no tradeoffs or compromises**.
+
+Message and storage layouts were **not changed**, although some rearragement might squeeze a little more gas,
+but that may break existing optimizations due to stack reordering.
+
+Also, **tests were improved** - a **Global Gas Counter** mechanism was added that accounts for gas in all transactions
+of all test suites and cases (except for negative and getter ones). This allows to keep an eye on other non-contest
+cases to track how bad is tradeoff when performing optimizations here and there.
+
+Another utility that was developed for contest is ***scalpel script***, that allows for a detailed, *really* detailed optimizations
+of the code by comparing lines of code function by function, printing out diffs, and providing detailed TVM files with
+stack comments and rewrites. This utility allowed to make some latter optimizations, since with each optimization
+next one becomes exponentionally harder to make. While result is not entirely precise and is needed to be verified
+by tests, this allows to instantly estimate whether there is some progress or not, since scalpel is executed immediately,
+while tests take approximately 10 seconds to execute.
+
+### Details of optimizations, their rationale and explanations, comparison of consumed gas both in test cases and not in test cases (global gas counter) are provided on a dedicated page: [Gas improvements](Improvements.rst).

README.md

@@ -10,7 +10,13 @@ Wallet V5 has 25% lower fees, can delegate payments for gas to third parties and
 -   `contracts` - source code of all the smart contracts of the project and their dependencies.
 -   `wrappers` - wrapper classes (implementing `Contract` from ton-core) for the contracts, including any [de]serialization primitives and compilation functions.
 -   `tests` - tests for the contracts.
--   `scripts` - scripts used by the project, mainly the deployment scripts.
+-   `scripts` - scripts used by the project, mainly the deployment scripts, additionally contains utilities for gas optimisation.
+-   `fift` - contains standard Fift v0.4.4 library including the assembler and disassembler for gas optimisation utilities.
+
+### Additional documentation
+
+-   [Gas improvements](Improvements.rst) - a log of improvements, detailed by primary code paths, global gas counters per commit.
+-   [Contest](Contest.md) - a note showing some information about interesting improvements during the optimisation contest.
 
 ## How to use
 

Specification.md

@@ -19,16 +19,18 @@ Thanks to [Andrew Gutarev](https://github.com/pyAndr3w) for the idea to set c5 r
 
 Thanks to [@subden](https://t.me/subden), [@botpult](https://t.me/botpult) and [@tvorogme](https://t.me/tvorogme) for ideas and discussion.
 
+Thanks to [Skydev](https://github.com/Skydev0h) for optimization and preparing the second revision of the contract.
+
 
 ## Features
 
 * 25% smaller computation fees.
 * Arbitrary amount of outgoing messages is supported via action list.
-* Wallet code can be upgraded transparently without breaking user's address in the future.
 * Wallet code can be extended by anyone in a decentralized and conflict-free way: multiple feature extensions can co-exist.
 * Extensions can perform the same operations as the signer: emit arbitrary messages on behalf of the owner, add and remove extensions.
 * Signed requests can be delivered via internal message to allow 3rd party pay for gas.
 * For consistency and ease of indexing, external messages also receive a 32-bit opcode.
+* To lay foundation for support of scenarios like 2FA or access recovery it is possible to disable signature authentication.
 
 ## Overview
 
@@ -39,8 +41,8 @@ Authentication:
 * by extension
 
 Operation types:
-* standard output actions
-* “set data” operation
+* standard output send message action
+* enable or disable public key (signature authentication)
 * install extension
 * remove extension
 
@@ -63,40 +65,12 @@ User may delegate this job to other apps via extensions.
 
 ### Extending the wallet
 
-**A. Use extensions**
-
 The best way to extend functionality of the wallet is to use the extensions mechanism that permit delegating access to the wallet to other contracts.
 
 From the perspective of the wallet, every extension can perform the same actions as the owner of a private key. Therefore limits and capabilities can be embedded in such an extension with a custom storage scheme.
 
 Extensions can co-exist simultaneously, so experimental capabilities can be deployed and tested independently from each other.
 
-**B. Code optimization**
-
-Backwards compatible code optimization **can be performed** with a single `set_code` action (`action_set_code#ad4de08e`) signed by the user. That is, hypothetical upgrade from `v5R1` to `v5R2` can be done in-place without forcing users to change their wallet address.
-
-If the optimized code requires changes to the data layout (e.g. reordering fields) the user can sign a request with two actions: `set_code` (in the standard action) and `set_data` (an extended action per this specification). Note that `set_data` action must make sure `seqno` is properly incremented after the upgrade as to prevent replays. Also, `set_data` must be performed right before the standard actions to not get overwritten by extension actions. The updated wallet **must** have the new subwallet ID to prevent accidental repeated migrations.
-
-User agents **should not** make `set_code` and `set_data` actions available via general-purpose API to prevent misuse and mistakes. Instead, they should be used as a part of migration logic for a specific wallet code.
-
-To restore the wallet by a seed phrase, user agent should use the original code and should expect the upgraded code to work in exactly the same way as previously.
-
-**C. Emergency upgrades**
-
-This is a variant of (B), so the same consideration apply. The difference is that functionality may be modified as to prevent user from suffering loss of funds. E.g. some previously possible actions or signed messages would lead to a failure.
-
-Just like with (B), user agents **should not** make `set_code` and `set_data` actions available via general-purpose API to prevent misuse and mistakes. Instead, they should be used as a part of migration logic for a specific wallet code.
-
-New users’ wallets **should not** be deployed with upgraded code. Instead, the improved wallet code should also be released as a new wallet version (e.g. v6, with a separate subwallet ID) and new wallets should be deployed with that code. This way `set_code` would be used as an emergency patch for existing wallets, while new wallets would be deployed directly with the major next version.
-
-
-**D. Substantial upgrades**
-
-We **do not recommend** performing substantial wallet upgrades in-place using `set_code`/`set_data` actions. Instead, user agents should have support for multiple accounts and easy switching between them.
-
-In-place migration requires maintaining backwards compatibility for all wallet features, which in turn could lead to increase in code size and higher gas and rent costs.
-
-
 ### Can the wallet outsource payment for gas fees?
 
 Yes! You can deliver signed messages via an internal message from a 3rd party wallet. Also, the message is handled exactly like an external one: after the basic checks the wallet takes care of the fees itself, so that 3rd party does not need to overpay for users who actually do have TONs.
@@ -123,6 +97,14 @@ You need to put two requests in your message body:
 
 Yes. We have considered constant-size schemes where the wallet only stores trusted extension code. However, extension authentication becomes combursome and expensive: plugin needs to transmit additional data and each request needs to recompute plugin’s address. We estimate that for the reasonably sized wallets (less than 100 plugins) authentication via the dictionary lookup would not exceed costs of indirect address authentication.
 
+### Why it can be useful to disallow authentication with signature?
+
+Ability to disallow authentication with signature enables two related use-cases:
+
+1. Two-factor authentication schemes: where control over wallet is fully delegated to an extension that checks two signatures: the user’s one and the signature from the auth service. Naturally, if the signature authentication in the wallet remains allowed, the second factor check is bypassed.
+
+2. Account recovery: delegating full control to another wallet in case of key compromise or loss. Wallet may contain larger amount of assets and its address could be tied to long-term contracts, therefore delegation to another controlling account is preferred to simply transferring the assets.
+
 ### What is library on masterchain?
 
 Library is a special code storage mechanism that allows to reduce storage cost for a new Wallet V5 contract instance. Wallet V5 contract code is stored into a masterchain library. 
@@ -143,7 +125,7 @@ wallet_id$_ global_id:int32 wc:int8 version:(## 8) subwallet_number:(## 32) = Wa
 - `global_id` is a TON chain identifier. TON Mainnet `global_id = -239` and TON Testnet `global_id = -3`.
 - `wc` is a Workchain. -1 for Masterchain and 0 for Basechain. 
 - `version`: current version of wallet v5 is `0`.
-- `subwallet_number` can be used to get multiplie wallet contracts binded to the single keypair.
+- `subwallet_number` can be used to get multiple wallet contracts bound to the single keypair.
 
 ## Packed address
 
@@ -166,48 +148,40 @@ Action types:
 ```tl-b
 // Standard actions from block.tlb:
 out_list_empty$_ = OutList 0;
-out_list$_ {n:#} prev:^(OutList n) action:OutAction
-  = OutList (n + 1);
-action_send_msg#0ec3c86d mode:(## 8) 
-  out_msg:^(MessageRelaxed Any) = OutAction;
-action_set_code#ad4de08e new_code:^Cell = OutAction;
-action_reserve_currency#36e6b809 mode:(## 8)
-  currency:CurrencyCollection = OutAction;
-libref_hash$0 lib_hash:bits256 = LibRef;
-libref_ref$1 library:^Cell = LibRef;
-action_change_library#26fa1dd4 mode:(## 7) { mode <= 2 }
-  libref:LibRef = OutAction;
+out_list$_ {n:#} prev:^(OutList n) action:OutAction = OutList (n + 1);
+action_send_msg#0ec3c86d mode:(## 8) out_msg:^(MessageRelaxed Any) = OutAction;
 
 // Extended actions in W5:
 action_list_basic$0 {n:#} actions:^(OutList n) = ActionList n 0;
 action_list_extended$1 {m:#} {n:#} action:ExtendedAction prev:^(ActionList n m) = ActionList n (m+1);
 
-action_set_data#1ff8ea0b data:^Cell = ExtendedAction;
 action_add_ext#1c40db9f addr:MsgAddressInt = ExtendedAction;
 action_delete_ext#5eaef4a4 addr:MsgAddressInt = ExtendedAction;
+action_set_signature_auth_allowed#20cbb95a allowed:(## 1) = ExtendedAction;
 ```
 
 Authentication modes:
 
 ```tl-b
-signed_request$_ 
-  signature:    bits512                   // 512
-  subwallet_id: uint32                    // 512+32
-  valid_until:  uint32                    // 512+32+32
-  msg_seqno:    uint32                    // 512+32+32+32 = 608
-  inner: InnerRequest = SignedRequest;
-
-internal_signed#7369676E signed:SignedRequest = InternalMsgBody;
-internal_extension#6578746E inner:InnerRequest = InternalMsgBody;
-external_signed#7369676E signed:SignedRequest = ExternalMsgBody;
+signed_request$_             // 32 (opcode from outer)
+  wallet_id:    WalletID     // 80
+  valid_until:  #            // 32
+  msg_seqno:    #            // 32
+  inner:        InnerRequest // 1 .. (1 + 32 + 256) + ^Cell
+  signature:    bits512      // 512
+= SignedRequest;             // Total: 688 .. 976 + ^Cell
+
+internal_signed#73696e74 signed:SignedRequest = InternalMsgBody;
+internal_extension#6578746e inner:InnerRequest = InternalMsgBody;
+external_signed#7369676e signed:SignedRequest = ExternalMsgBody;
 
 actions$_ {m:#} {n:#} actions:(ActionList n m) = InnerRequest;
 ```
 
 Contract state:
 ```tl-b
-wallet_id$_ global_id:int32 wc:int8 version:(## 8) subwallet_number:(## 32) = WalletID;
-contract_state$_ seqno:# wallet_id:WalletID public_key:(## 256) extensions_dict:(HashmapE 256 int8) = ContractState;
+wallet_id$_ global_id:# wc:int8 version:(## 8) subwallet_number:# = WalletID;
+contract_state$_ seqno:int33 wallet_id:WalletID public_key:(## 256) extensions_dict:(HashmapE 256 int8) = ContractState;
 ```
 
 ## Source code