Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests for the equivalence of one- and two-stage compilation via IR #12141

Merged
merged 2 commits into from
Nov 18, 2021
+98 −0
Merged

Tests for the equivalence of one- and two-stage compilation via IR #12141

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0d82fe9
cmdlineTests.sh: Test the equivalence of one and two-stage compilatio…
cameel Oct 14, 2021
e2e3934
Install diffutils on macOS to get diff with color support
cameel Oct 15, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .circleci/osx_install_dependencies.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ then
brew install cmake
brew install wget
brew install coreutils
brew install diffutils
./scripts/install_obsolete_jsoncpp_1_7_4.sh

# z3
Expand Down
12 changes: 12 additions & 0 deletions scripts/common.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,18 @@ function msg_on_error
fi
}

function diff_values
{
(( $# >= 2 )) || fail "diff_values requires at least 2 arguments."

local value1="$1"
local value2="$2"
shift
shift

diff --color=auto --unified=0 <(echo "$value1") <(echo "$value2") "$@"
}

function safe_kill
{
local PID=${1}
Expand Down
85 changes: 85 additions & 0 deletions test/cmdlineTests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -308,6 +308,61 @@ function test_solc_assembly_output
fi
}

function test_via_ir_equivalence()
{
(( $# <= 2 )) || fail "This function accepts at most two arguments."

if [[ $2 != --optimize ]] && [[ $2 != "" ]]
then
fail "The second argument must be --optimize if present."
fi

local solidity_code="$1"
local optimize_flag="$2"

local optimizer_flags=()
[[ $optimize_flag == "" ]] || optimizer_flags+=("$optimize_flag")

local ir_output
ir_output=$(
echo "$solidity_code" |
msg_on_error --no-stderr "$SOLC" - --ir-optimized --debug-info location "${optimizer_flags[@]}" |
sed '/^Optimized IR:$/d'
)

local asm_output_two_stage asm_output_via_ir
asm_output_two_stage=$(
echo "$ir_output" |
msg_on_error --no-stderr "$SOLC" - --strict-assembly --asm "${optimizer_flags[@]}" |
sed '/^======= <stdin>/d' |
sed '/^Text representation:$/d'
)
asm_output_via_ir=$(
echo "$solidity_code" |
msg_on_error --no-stderr "$SOLC" - --experimental-via-ir --asm --debug-info location "${optimizer_flags[@]}" |
sed '/^======= <stdin>/d' |
sed '/^EVM assembly:$/d'
)

diff_values "$asm_output_two_stage" "$asm_output_via_ir" --ignore-space-change --ignore-blank-lines

local bin_output_two_stage bin_output_via_ir
bin_output_two_stage=$(
echo "$ir_output" |
msg_on_error --no-stderr "$SOLC" - --strict-assembly --bin "${optimizer_flags[@]}" |
sed '/^======= <stdin>/d' |
sed '/^Binary representation:$/d'
)
bin_output_via_ir=$(
echo "$solidity_code" |
msg_on_error --no-stderr "$SOLC" - --experimental-via-ir --bin "${optimizer_flags[@]}" |
sed '/^======= <stdin>/d' |
sed '/^Binary:$/d'
)

diff_values "$bin_output_two_stage" "$bin_output_via_ir" --ignore-space-change --ignore-blank-lines
}

## RUN

echo "Checking that the bug list is up to date..."
Expand Down Expand Up @@ -533,6 +588,36 @@ printTask "Testing assemble, yul, strict-assembly and optimize..."
test_solc_assembly_output "{ let x := 0 }" "{ { } }" "--strict-assembly --optimize"
)

printTask "Testing the eqivalence of --experimental-via-ir and a two-stage compilation..."
(
printTask " - Smoke test"
test_via_ir_equivalence "contract C {}"

printTask " - Smoke test (optimized)"
test_via_ir_equivalence "contract C {}" --optimize

externalContracts=(
deposit_contract.sol
FixedFeeRegistrar.sol
_stringutils/stringutils.sol
Comment on lines +600 to +602
Copy link
Member Author

@cameel cameel Oct 14, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are more in semanticTests/externalContracts/ but these are the only ones that aren't either a multi-file test (can't compile them without splitting first) or a file containing multiple contracts (the --ir-optimized output needs splitting in that case).

The deposit contract takes a bit long to compile. All the compilations in this whole subshell take ~30 seconds total to run on my machine and that contract is a big part of that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually more worried about there not being multi-file or multi-contract tests.

Copy link
Member Author

@cameel cameel Nov 9, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's doable for multi-file but I'd have to restructure the functions a bit and pass some extra paths as arguments. It would be easiest with a map and that's a real pain in Bash so I'd probably just do it in Python. Should I do that? We probably have enough single-file contracts that I didn't think it was worth spending time on.

For multi-contract it's because in Yul mode we only support compiling one file at a time so I'd have to manually split the IR output, assemble and then combine again in the right order before diffing. If we really want that, I'd rather just add a multi-file mode in the CLI than cobble it up in Bash.

)
requiresOptimizer=(
deposit_contract.sol
FixedFeeRegistrar.sol
)

for contractFile in "${externalContracts[@]}"
do
if ! [[ "${requiresOptimizer[*]}" =~ $contractFile ]]
then
printTask " - ${contractFile}"
test_via_ir_equivalence "$(cat "${REPO_ROOT}/test/libsolidity/semanticTests/externalContracts/${contractFile}")"
fi

printTask " - ${contractFile} (optimized)"
test_via_ir_equivalence "$(cat "${REPO_ROOT}/test/libsolidity/semanticTests/externalContracts/${contractFile}")" --optimize
done
)

printTask "Testing standard input..."
SOLTMPDIR=$(mktemp -d)
Expand Down