Template generation

.gitmodules

@@ -1,3 +1,9 @@
 [submodule "tests/test_data/lib/forge-std"]
 	path = tests/test_data/lib/forge-std
 	url = https://github.com/foundry-rs/forge-std
+[submodule "tests/test_data/lib/properties"]
+	path = tests/test_data/lib/properties
+	url = https://github.com/crytic/properties
+[submodule "tests/test_data/lib/solmate"]
+	path = tests/test_data/lib/solmate
+	url = https://github.com/transmissions11/solmate

CONTRIBUTING.md

@@ -30,8 +30,12 @@ Below is a rough outline of fuzz-utils's design:
 
 ```text
 .
-├── fuzzers # Contains supported fuzzer classes that parse and generate the test files
-├── templates # String templates used for test generation
+├── generate # Classes related to the `generate` command
+|   └── fuzzers # Supported fuzzer classes
+├── parsing # Contains the main parser logic
+|   └── commands # Flags and execution logic per supported subparser
+├── template # Classes related to the `template` command
+├── templates # Common templates such as the default config and templates for test and harness generation
 ├── utils # Utility functions
 ├── main.py # Main entry point
 └── ...

Makefile

@@ -68,7 +68,10 @@ reformat:
 test tests: $(VENV)/pyvenv.cfg
 	. $(VENV_BIN)/activate && \
 		solc-select use 0.8.19 --always-install && \
-		pytest $(T) $(TEST_ARGS)
+		cd tests/test_data && \
+		forge install && \
+		cd ../.. && \
+		pytest --ignore tests/test_data/lib $(T) $(TEST_ARGS)
 
 .PHONY: package
 package: $(VENV)/pyvenv.cfg

README.md

@@ -1,16 +1,22 @@
 <img src="./logo.png" alt="Slither Static Analysis Framework Logo" width="500" />
 
-# Automated tool for generating Foundry unit tests from smart contract fuzzer failed properties
+# Automated utility tooling for smart contract fuzzers
 
-`fuzz-utils` is a Python tool that generates unit tests from [Echidna](https://github.com/crytic/echidna) and [Medusa](https://github.com/crytic/medusa/tree/master) failed properties, using the generated reproducer files. It uses [Slither](https://github.com/crytic/slither) for determining types and jinja2 for generating the test files using string templates.
+`fuzz-utils` is a set of Python tools that aim to improve the developer experience when using smart contract fuzzing.
+The tools include:
+- automatically generate unit tests from [Echidna](https://github.com/crytic/echidna) and [Medusa](https://github.com/crytic/medusa/tree/master) failed properties, using the generated reproducer files. 
+- automatically generate a Echidna/Medusa compatible fuzzing harness.
+
+`fuzz-utils` uses [Slither](https://github.com/crytic/slither) for determining types and `jinja2` for generating the test files using string templates.
 
 **Disclaimer**: Please note that `fuzz-utils` is **under development**. Currently, not all Solidity types are supported and some types (like `bytes*`, and `string`) might be improperly decoded from the corpora call sequences. We are investigating a better corpus format that will ease the creation of unit tests.
 
 ## Features
 `fuzz-utils` provides support for:
-- ✔️ Generating Foundry unit tests from the fuzzer corpus of single entry point fuzzing harnesses
-- ✔️ Medusa and Echidna corpora
-- ✔️ Solidity types: `bool`,`uint*`, `int*`, `address`, `struct`, `enum`, single-dimensional fixed-size arrays and dynamic arrays, multi-dimensional fixed-size arrays.
+- ✔️ Generating Foundry unit tests from the fuzzer corpus of single entry point fuzzing harnesses.
+- ✔️ Generating fuzzing harnesses, `Actor` contracts, and templated `attack` contracts to ease fuzzing setup.
+- ✔️ Supports Medusa and Echidna corpora
+- ✔️ Test generation supports Solidity types: `bool`,`uint*`, `int*`, `address`, `struct`, `enum`, single-dimensional fixed-size arrays and dynamic arrays, multi-dimensional fixed-size arrays.
 
 Multi-dimensional dynamic arrays, function pointers, and other more complex types are in the works, but are currently not supported.
 ## Installation and pre-requisites
@@ -23,24 +29,110 @@ pip install fuzz-utils
 
 These commands will install all the Python libraries and tools required to run `fuzz-utils`. However, it won't install Echidna or Medusa, so you will need to download and install the latest version yourself from its official releases ([Echidna](https://github.com/crytic/echidna/releases), [Medusa](https://github.com/crytic/medusa/releases)).
 
-## Example
+## Tools
+The available tool commands are:
+- [`init`](#initializing-a-configuration-file) - Initializes a configuration file
+- [`generate`](#generating-unit-tests) - generates unit tests from a corpus
+- [`template`](#generating-fuzzing-harnesses) - generates a fuzzing harness
+
+### Generating unit tests
+
+The `generate` command is used to generate Foundry unit tests from Echidna or Medusa corpus call sequences.
+
+**Command-line options:**
+- `compilation_path`: The path to the Solidity file or Foundry directory
+- `-cd`/`--corpus-dir` `path_to_corpus_dir`: The path to the corpus directory relative to the working directory.
+- `-c`/`--contract` `contract_name`: The name of the target contract.
+- `-td`/`--test-directory` `path_to_test_directory`: The path to the test directory relative to the working directory.
+- `-i`/`--inheritance-path` `relative_path_to_contract`: The relative path from the test directory to the contract (used for inheritance).
+- `-f`/`--fuzzer` `fuzzer_name`: The name of the fuzzer, currently supported: `echidna` and `medusa`
+- `--named-inputs`: Includes function input names when making calls
+- `--config`: Path to the fuzz-utils config JSON file
+- `--all-sequences`: Include all corpus sequences when generating unit tests.
+
+**Example**
 
 In order to generate a test file for the [BasicTypes.sol](tests/test_data/src/BasicTypes.sol) contract, based on the Echidna corpus reproducers for this contract ([corpus-basic](tests/test_data/echidna-corpora/corpus-basic/)), we need to `cd` into the `tests/test_data` directory which contains the Foundry project and run the command:
 ```bash
-fuzz-utils ./src/BasicTypes.sol --corpus-dir echidna-corpora/corpus-basic --contract "BasicTypes" --test-directory "./test/" --inheritance-path "../src/" --fuzzer echidna
+fuzz-utils generate ./src/BasicTypes.sol --corpus-dir echidna-corpora/corpus-basic --contract "BasicTypes" --test-directory "./test/" --inheritance-path "../src/" --fuzzer echidna
 ```
 
-Running this command should generate a `BasicTypes_Echidna_Test.sol` file in the [tests](/tests/test_data/test/) directory of the Foundry project.
+Running this command should generate a `BasicTypes_Echidna_Test.sol` file in the [test](/tests/test_data/test/) directory of the Foundry project.
 
-## Command-line options
+### Generating fuzzing harnesses
 
-Additional options are available for the script:
+The `template` command is used to generate a fuzzing harness. The harness can include multiple `Actor` contracts which are used as proxies for user actions, as well as `attack` contracts which can be selected from a set of premade contracts that perform certain common attack scenarios.
 
-- `-cd`/`--corpus-dir` `path_to_corpus_dir`: The path to the corpus directory relative to the working directory.
-- `-c`/`--contract` `contract_name`: The name of the contract.
-- `-td`/`--test-directory` `path_to_test_directory`: The path to the test directory relative to the working directory.
-- `-i`/`--inheritance-path` `relative_path_to_contract`: The relative path from the test directory to the contract (used for inheritance).
-- `-f`/`--fuzzer` `fuzzer_name`: The name of the fuzzer, currently supported: `echidna` and `medusa`
+**Command-line options:**
+- `compilation_path`: The path to the Solidity file or Foundry directory
+- `-n`/`--name` `name: str`: The name of the fuzzing harness.
+- `-c`/`--contracts` `target_contracts: list`: The name of the target contract.
+- `-o`/`--output-dir` `output_directory: str`: Output directory name. By default it is `fuzzing`
+- `--config`: Path to the `fuzz-utils` config JSON file
+
+**Example**
+
+In order to generate a fuzzing harness for the [BasicTypes.sol](tests/test_data/src/BasicTypes.sol) contract, we need to `cd` into the `tests/test_data/` directory which contains the Foundry project and run the command:
+```bash
+fuzz-utils template ./src/BasicType.sol --name "DefaultHarness" --contracts BasicTypes
+```
+
+Running this command should generate the directory structure in [tests/test_data/test/fuzzing](tests/test_data/test/fuzzing), which contains the fuzzing harness [DefaultHarness](tests/test_data/test/fuzzing/harnesses/DefaultHarness.sol) and the Actor contract [DefaultActor](tests/test_data/test/fuzzing/actors/ActorDefault.sol).
+
+## Utilities
+
+### Initializing a configuration file
+
+The `init` command can be used to initialize a default configuration file in the project root. 
+
+**Configuration file:**
+Using the configuration file allows for more granular control than just using the command-line options. Valid configuration options are listed below:
+```json
+{
+    "generate": {
+        "targetContract": "BasicTypes",              // The Echidna/Medusa fuzzing harness 
+        "compilationPath": "./src/BasicTypes",       // Path to the file or Foundry directory
+        "corpusDir": "echidna-corpora/corpus-basic", // Path to the corpus directory
+        "fuzzer": "echidna",                         // `echidna` | `medusa`
+        "testsDir": "./test/",                       // Path to the directory where the tests will be generated
+        "inheritancePath": "../src/",                // Relative path from the testing directory to the contracts
+        "namedInputs": false,                        // True | False, whether to include function input names when making calls
+        "allSequences": false,                       // True | False, whether to generate tests for the entire corpus (including non-failing sequences)
+    },
+    "template": {
+        "name": "DefaultHarness",                    // The name of the fuzzing harness that will be generated
+        "targets": ["BasicTypes"],                   // The contracts to be included in the fuzzing harness
+        "outputDir": "./test/fuzzing",               // The output directory where the files and directories will be saved
+        "compilationPath": ".",                      // The path to the Solidity file (if single target) or Foundry directory
+        "actors": [                                  // At least one actor is required. If the array is empty, the DefaultActor which wraps all of the functions from the target contracts will be generated
+            {
+                "name": "Default",                   // The name of the Actor contract, saved as `Actor{name}`
+                "targets": ["BasicTypes"],           // The list of contracts that the Actor can interact with
+                "number": 3,                         // The number of instances of this Actor that will be used in the harness
+                "filters": {                         // Used to filter functions so that only functions that fulfill certain criteria are included
+                    "strict": false,                 // If `true`, only functions that fulfill *all* the criteria will be included. If `false`, functions that fulfill *any* criteria will be included
+                    "onlyModifiers": [],             // List of modifiers to include
+                    "onlyPayable": false,            // If `true`, only `payable` functions will be included. If `false`, both payable and non-payable functions will be included
+                    "onlyExternalCalls": [],         // Only include functions that make a certain external call. E.g. [`transferFrom`]
+                },
+            }
+        ],
+        "attacks": [                                 // A list of premade attack contracts to include. 
+            {
+                "name": "Deposit",                   // The name of the attack contract. 
+                "targets": ["BasicTypes"],           // The list of contracts that the attack contract can interact with
+                "number": 1,                         // The number of instances of this attack contract that will be used in the harness
+                "filters": {                         // Used to filter functions so that only functions that fulfill certain criteria are included
+                    "strict": false,                 // If `true`, only functions that fulfill *all* the criteria will be included. If `false`, functions that fulfill *any* criteria will be included
+                    "onlyModifiers": [],             // List of modifiers to include
+                    "onlyPayable": false,            // If `true`, only `payable` functions will be included. If `false`, both payable and non-payable functions will be included
+                    "onlyExternalCalls": [],         // Only include functions that make a certain external call. E.g. [`transferFrom`]
+                },
+            }
+        ],
+    },
+}
+```
 
 ## Contributing
 For information about how to contribute to this project, check out the [CONTRIBUTING](CONTRIBUTING.md) guidelines.

fuzz_utils/generate/FoundryTest.py

@@ -0,0 +1,99 @@
+"""The FoundryTest class that handles generation of unit tests from call sequences"""
+import os
+import sys
+import json
+from typing import Any
+import jinja2
+
+from slither import Slither
+from slither.core.declarations.contract import Contract
+from fuzz_utils.utils.crytic_print import CryticPrint
+
+from fuzz_utils.generate.fuzzers.Medusa import Medusa
+from fuzz_utils.generate.fuzzers.Echidna import Echidna
+from fuzz_utils.templates.foundry_templates import templates
+
+
+class FoundryTest:  # pylint: disable=too-many-instance-attributes
+    """
+    Handles the generation of Foundry test files
+    """
+
+    def __init__(
+        self,
+        config: dict,
+        slither: Slither,
+        fuzzer: Echidna | Medusa,
+    ) -> None:
+        self.inheritance_path = config["inheritancePath"]
+        self.target_name = config["targetContract"]
+        self.corpus_path = config["corpusDir"]
+        self.test_dir = config["testsDir"]
+        self.all_sequences = config["allSequences"]
+        self.slither = slither
+        self.target = self.get_target_contract()
+        self.fuzzer = fuzzer
+
+    def get_target_contract(self) -> Contract:
+        """Gets the Slither Contract object for the specified contract file"""
+        contracts = self.slither.get_contract_from_name(self.target_name)
+        # Loop in case slither fetches multiple contracts for some reason (e.g., similar names?)
+        for contract in contracts:
+            if contract.name == self.target_name:
+                return contract
+
+        # TODO throw error if no contract found
+        sys.exit(-1)
+
+    def create_poc(self) -> str:
+        """Takes in a directory path to the echidna reproducers and generates a test file"""
+
+        file_list: list[dict[str, Any]] = []
+        tests_list = []
+        dir_list = []
+        if self.all_sequences:
+            dir_list = self.fuzzer.corpus_dirs
+        else:
+            dir_list = [self.fuzzer.reproducer_dir]
+
+        # 1. Iterate over each directory and reproducer file (open it)
+        for directory in dir_list:
+            for entry in os.listdir(directory):
+                full_path = os.path.join(directory, entry)
+
+                if os.path.isfile(full_path):
+                    try:
+                        with open(full_path, "r", encoding="utf-8") as file:
+                            file_list.append({"path": full_path, "content": json.load(file)})
+                    except Exception:  # pylint: disable=broad-except
+                        print(f"Fail on {full_path}")
+
+        # 2. Parse each reproducer file and add each test function to the functions list
+        for idx, file_obj in enumerate(file_list):
+            try:
+                tests_list.append(
+                    self.fuzzer.parse_reproducer(file_obj["path"], file_obj["content"], idx)
+                )
+            except Exception:  # pylint: disable=broad-except
+                print(f"Parsing fail on {file_obj['content']}: index: {idx}")
+
+        # 4. Generate the test file
+        template = jinja2.Template(templates["CONTRACT"])
+        write_path = f"{self.test_dir}{self.target_name}"
+        inheritance_path = f"{self.inheritance_path}{self.target_name}"
+
+        # 5. Save the test file
+        test_file_str = template.render(
+            file_path=f"{inheritance_path}.sol",
+            target_name=self.target_name,
+            amount=0,
+            tests=tests_list,
+            fuzzer=self.fuzzer.name,
+        )
+        with open(f"{write_path}_{self.fuzzer.name}_Test.t.sol", "w", encoding="utf-8") as outfile:
+            outfile.write(test_file_str)
+        CryticPrint().print_success(
+            f"Generated a test file in {write_path}_{self.fuzzer.name}_Test.t.sol"
+        )
+
+        return test_file_str

fuzz_utils/fuzzers/Echidna.py → fuzz_utils/generate/fuzzers/Echidna.py

@@ -18,6 +18,7 @@
 from fuzz_utils.utils.error_handler import handle_exit
 
 
+# pylint: disable=too-many-instance-attributes
 class Echidna:
     """
     Handles the generation of Foundry test files from Echidna reproducers
@@ -33,6 +34,7 @@ def __init__(
         self.reproducer_dir = f"{corpus_path}/reproducers"
         self.corpus_dirs = [f"{corpus_path}/coverage", self.reproducer_dir]
         self.named_inputs = named_inputs
+        self.declared_variables: set[tuple[str, str]] = set()
 
     def get_target_contract(self) -> Contract:
         """Finds and returns Slither Contract"""
@@ -51,17 +53,26 @@ def parse_reproducer(self, file_path: str, calls: Any, index: int) -> str:
         call_list = []
         end = len(calls) - 1
         function_name = ""
+        has_low_level_call: bool = False
+
+        # before each test case, we clear the declared variables, as those are locals
+        self.declared_variables = set()
+
         # 1. For each object in the list process the call object and add it to the call list
         for idx, call in enumerate(calls):
             call_str, fn_name = self._parse_call_object(call)
             call_list.append(call_str)
+            has_low_level_call = has_low_level_call or ("(success, " in call_str)
             if idx == end:
                 function_name = fn_name + "_" + str(index)
 
         # 2. Generate the test string and return it
         template = jinja2.Template(templates["TEST"])
         return template.render(
-            function_name=function_name, call_list=call_list, file_path=file_path
+            function_name=function_name,
+            call_list=call_list,
+            file_path=file_path,
+            has_low_level_call=has_low_level_call,
         )
 
     # pylint: disable=too-many-locals,too-many-branches
@@ -119,7 +130,6 @@ def _parse_call_object(self, call_dict: dict[Any, Any]) -> tuple[str, str]:
                 for idx, input_param in enumerate(slither_entry_point.parameters):
                     call_definition[idx] = input_param.name + ": " + call_definition[idx]
                 parameters_str = "{" + ", ".join(call_definition) + "}"
-                print(parameters_str)
             else:
                 parameters_str = ", ".join(call_definition)
 
@@ -307,5 +317,13 @@ def _get_memarr(
 
         input_type = input_parameter.type
         name = f"dyn{input_type}Arr_{index}"
-        declaration = f"{input_type}[] memory {name} = new {input_type}[]({length});\n"
+
+        # If the variable was already declared, just assign the new value
+        if (input_type, name) in self.declared_variables:
+            declaration = f"{name} = new {input_type}[]({length});\n"
+        else:
+            declaration = f"{input_type}[] memory {name} = new {input_type}[]({length});\n"
+
+        self.declared_variables.add((input_type, name))
+
         return name, declaration