Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VM, Common: implement eip-3540 (EOF1) #1719

Merged
merged 34 commits into from
Mar 15, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
66a1054
Add EIP json
acolytec3 Feb 11, 2022
a162fec
Partial changes to enable EIp3540 and start code checks
acolytec3 Feb 14, 2022
e214e1b
Finish code validation checks and API tests
acolytec3 Feb 14, 2022
b9a9a29
Move eof params to common
acolytec3 Feb 15, 2022
eb3e982
Code execution context updates
acolytec3 Feb 15, 2022
3cdb742
Add exception for invalid EOF format
acolytec3 Feb 16, 2022
222e496
Various fixes
acolytec3 Feb 16, 2022
f7ab311
Gate push handler changes behind EIP
acolytec3 Feb 16, 2022
080a912
Remove ethereum/tests tests
acolytec3 Feb 16, 2022
583fa21
Clarify eof helper variable names
acolytec3 Feb 16, 2022
3b5b208
more naming clarifications
acolytec3 Feb 16, 2022
3c560e8
rename bytecode to container
acolytec3 Feb 16, 2022
d77e3e0
check that section sizes are greater than 0
acolytec3 Feb 16, 2022
a650bd0
VM, Common: Add EIP-3670 (EOF - Code Validation) (#1743)
acolytec3 Feb 28, 2022
6f92ceb
Fix typos, add tests, update error EOF handler
acolytec3 Feb 28, 2022
fc5ead8
EIP3540 tests
acolytec3 Feb 28, 2022
f034ba1
Lint fixes
acolytec3 Feb 28, 2022
7815b71
Fix tests
acolytec3 Mar 1, 2022
c16877a
Lint/uncomment tests
acolytec3 Mar 1, 2022
57a3f3a
More adjustments to EOF1 logic
acolytec3 Mar 3, 2022
ce2f314
compartmentalize tests
acolytec3 Mar 3, 2022
8b10866
Add checks for newly deployed contract code
acolytec3 Mar 4, 2022
ef45dea
Merge branch 'master' into implement-eip3540
acolytec3 Mar 4, 2022
88afc65
Fix state test runner for specified EIPs
acolytec3 Mar 7, 2022
38a3141
Merge branch 'master' into implement-eip3540
jochem-brouwer Mar 11, 2022
f27bc01
vm: add eip3540 tests invalid eof initcode
jochem-brouwer Mar 11, 2022
605e576
vm: lint
jochem-brouwer Mar 11, 2022
1114476
vm/tests: cleanup 3540 tests
jochem-brouwer Mar 11, 2022
3fb6bab
Merge branch 'master' into implement-eip3540
holgerd77 Mar 14, 2022
80933d2
Address feedback
acolytec3 Mar 14, 2022
9d028f0
lint
acolytec3 Mar 14, 2022
6282e89
Merge branch 'master' into implement-eip3540
holgerd77 Mar 15, 2022
9fea5bb
Merge branch 'master' into implement-eip3540
acolytec3 Mar 15, 2022
8f2d17b
Merge branch 'master' into implement-eip3540
holgerd77 Mar 15, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions packages/common/src/eips/3540.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
"name": "EIP-3540",
"number": 3540,
"comment": "EVM Object Format (EOF) v1",
"url": "https://eips.ethereum.org/EIPS/eip-3540",
"status": "Review",
"minimumHardfork": "london",
"requiredEIPs": [
3541
],
"gasConfig": {},
"gasPrices": {},
"vm": {},
"pow": {}
}
15 changes: 15 additions & 0 deletions packages/common/src/eips/3670.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
"name": "EIP-3670",
"number": 3670,
"comment": "EOF - Code Validation",
"url": "https://eips.ethereum.org/EIPS/eip-3670",
"status": "Review",
"minimumHardfork": "london",
"requiredEIPs": [
3540
],
"gasConfig": {},
"gasPrices": {},
"vm": {},
"pow": {}
}
2 changes: 2 additions & 0 deletions packages/common/src/eips/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,11 @@ export const EIPs: eipsType = {
2930: require('./2930.json'),
3198: require('./3198.json'),
3529: require('./3529.json'),
3540: require('./3540.json'),
3541: require('./3541.json'),
3554: require('./3554.json'),
3607: require('./3607.json'),
3670: require('./3670.json'),
3675: require('./3675.json'),
3855: require('./3855.json'),
3860: require('./3860.json'),
Expand Down
48 changes: 44 additions & 4 deletions packages/vm/src/evm/evm.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ import TxContext from './txContext'
import Message from './message'
import EEI from './eei'
// eslint-disable-next-line
import { short } from './opcodes/util'
import { eof1CodeAnalysis, eof1ValidOpcodes, short } from './opcodes/util'
import { Log } from './types'
import { default as Interpreter, InterpreterOpts, RunState } from './interpreter'

Expand Down Expand Up @@ -106,6 +106,14 @@ export function INVALID_BYTECODE_RESULT(gasLimit: BN): ExecResult {
}
}

export function INVALID_EOF_RESULT(gasLimit: BN): ExecResult {
return {
returnValue: Buffer.alloc(0),
gasUsed: gasLimit,
exceptionError: new VmError(ERROR.INVALID_EOF_FORMAT),
}
}

export function VmErrorResult(error: VmError, gasUsed: BN): ExecResult {
return {
returnValue: Buffer.alloc(0),
Expand Down Expand Up @@ -387,8 +395,8 @@ export default class EVM {
if (this._vm.DEBUG) {
debug(`Start bytecode processing...`)
}
let result = await this.runInterpreter(message)

let result = await this.runInterpreter(message)
// fee for size of the return value
let totalGas = result.gasUsed
let returnFee = new BN(0)
Expand Down Expand Up @@ -422,7 +430,36 @@ export default class EVM {
this._vm._common.isActivatedEIP(3541) &&
result.returnValue.slice(0, 1).equals(Buffer.from('EF', 'hex'))
) {
result = { ...result, ...INVALID_BYTECODE_RESULT(message.gasLimit) }
if (!this._vm._common.isActivatedEIP(3540)) {
result = { ...result, ...INVALID_BYTECODE_RESULT(message.gasLimit) }
}
// Begin EOF1 contract code checks
// EIP-3540 EOF1 header check
const eof1CodeAnalysisResults = eof1CodeAnalysis(result.returnValue)
if (!eof1CodeAnalysisResults?.code) {
result = {
...result,
...INVALID_EOF_RESULT(message.gasLimit),
}
} else if (this._vm._common.isActivatedEIP(3670)) {
// EIP-3670 EOF1 opcode check
const codeStart = eof1CodeAnalysisResults.data > 0 ? 10 : 7
// The start of the code section of an EOF1 compliant contract will either be
// index 7 (if no data section is present) or index 10 (if a data section is present)
// in the bytecode of the contract
if (
!eof1ValidOpcodes(
result.returnValue.slice(codeStart, codeStart + eof1CodeAnalysisResults.code)
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is from the EIP-3670 spec:

At contract creation time both initcode and code are iterated instruction-by-instruction

Might it be that you are doing validation only on the code and not the initcode? Or am I missing something?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is checking the code returned by the create contract to verify its valid . The initcode is checked when it is run through the runInterpreter function call.

) {
result = {
...result,
...INVALID_EOF_RESULT(message.gasLimit),
}
} else {
result.gasUsed = totalGas
}
}
} else {
result.gasUsed = totalGas
}
Expand Down Expand Up @@ -504,7 +541,10 @@ export default class EVM {
let result = eei._result
let gasUsed = message.gasLimit.sub(eei._gasLeft)
if (interpreterRes.exceptionError) {
if (interpreterRes.exceptionError.error !== ERROR.REVERT) {
if (
interpreterRes.exceptionError.error !== ERROR.REVERT &&
interpreterRes.exceptionError.error !== ERROR.INVALID_EOF_FORMAT
) {
gasUsed = message.gasLimit
}

Expand Down
51 changes: 47 additions & 4 deletions packages/vm/src/evm/interpreter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,13 @@ import { ERROR, VmError } from '../exceptions'
import Memory from './memory'
import Stack from './stack'
import EEI from './eei'
import { Opcode, handlers as opHandlers, OpHandler, AsyncOpHandler } from './opcodes'
import {
Opcode,
handlers as opHandlers,
OpHandler,
AsyncOpHandler,
eof1CodeAnalysis,
} from './opcodes'
import { dynamicGasHandlers } from './opcodes/gas'

export interface InterpreterOpts {
Expand Down Expand Up @@ -87,9 +93,46 @@ export default class Interpreter {
}

async run(code: Buffer, opts: InterpreterOpts = {}): Promise<InterpreterResult> {
this._runState.code = code
this._runState.programCounter = opts.pc ?? this._runState.programCounter
if (
!this._vm._common.isActivatedEIP(3540) ||
!code.slice(0, 1).equals(Buffer.from('ef', 'hex'))
) {
// EIP-3540 isn't active and first byte is not 0xEF - treat as legacy bytecode
this._runState.code = code
} else if (this._vm._common.isActivatedEIP(3540)) {
if (!code.slice(1, 2).equals(Buffer.from('00', 'hex'))) {
// Bytecode contains invalid EOF magic byte
return {
runState: this._runState,
exceptionError: new VmError(ERROR.INVALID_BYTECODE_RESULT),
}
}
if (!code.slice(2, 3).equals(Buffer.from('01', 'hex'))) {
// Bytecode contains invalid EOF version number
return {
runState: this._runState,
exceptionError: new VmError(ERROR.INVALID_EOF_FORMAT),
}
}
// Code is EOF1 format
const codeSections = eof1CodeAnalysis(code)
if (!codeSections) {
// Code is invalid EOF1 format if `codeSections` is falsy
return {
runState: this._runState,
exceptionError: new VmError(ERROR.INVALID_EOF_FORMAT),
acolytec3 marked this conversation as resolved.
Show resolved Hide resolved
}
}

if (codeSections.data) {
// Set code to EOF container code section which starts at byte position 10 if data section is present
this._runState.code = code.slice(10, 10 + codeSections!.code)
} else {
// Set code to EOF container code section which starts at byte position 7 if no data section is present
this._runState.code = code.slice(7, 7 + codeSections!.code)
}
}
this._runState.programCounter = opts.pc ?? this._runState.programCounter
// Check that the programCounter is in range
const pc = this._runState.programCounter
if (pc !== 0 && (pc < 0 || pc >= this._runState.code.length)) {
Expand All @@ -105,7 +148,7 @@ export default class Interpreter {
(opCode === 0x56 || opCode === 0x57 || opCode === 0x5e)
) {
// Only run the jump destination analysis if `code` actually contains a JUMP/JUMPI/JUMPSUB opcode
this._runState.validJumps = this._getValidJumpDests(code)
this._runState.validJumps = this._getValidJumpDests(this._runState.code)
this._runState.shouldDoJumpAnalysis = false
}
this._runState.opCode = opCode
Expand Down
8 changes: 7 additions & 1 deletion packages/vm/src/evm/opcodes/functions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -823,8 +823,14 @@ export const handlers: Map<number, OpHandler> = new Map([
0x60,
function (runState) {
const numToPush = runState.opCode - 0x5f
if (
runState.eei._common.isActivatedEIP(3540) &&
runState.programCounter + numToPush > runState.code.length
) {
trap(ERROR.OUT_OF_RANGE)
}
const loaded = new BN(
runState.eei.getCode().slice(runState.programCounter, runState.programCounter + numToPush)
runState.code.slice(runState.programCounter, runState.programCounter + numToPush)
)
runState.programCounter += numToPush
runState.stack.push(loaded)
Expand Down
91 changes: 91 additions & 0 deletions packages/vm/src/evm/opcodes/util.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import Common from '@ethereumjs/common'
import { BN, keccak256, setLengthRight, setLengthLeft } from 'ethereumjs-util'
import { handlers } from '.'
import { ERROR, VmError } from './../../exceptions'
import { RunState } from './../interpreter'

Expand Down Expand Up @@ -261,3 +262,93 @@ export function updateSstoreGas(
return new BN(common.param('gasPrices', 'sstoreSet'))
}
}

/**
*
* @param container A `Buffer` containing bytecode to be checked for EOF1 compliance
* @returns an object containing the size of the code section and data sections for a valid
* EOF1 container or else undefined if `container` is not valid EOF1 bytecode
*
* Note: See https://eips.ethereum.org/EIPS/eip-3540 for further details
*/
export const eof1CodeAnalysis = (container: Buffer) => {
const magic = 0x00
const version = 0x01
const secCode = 0x01
const secData = 0x02
const secTerminator = 0x00
let computedContainerSize = 0
const sectionSizes = {
code: 0,
data: 0,
}
if (container[1] !== magic || container[2] !== version)
// Bytecode does not contain EOF1 "magic" or version number in expected positions
return

if (
// EOF1 bytecode must be more than 7 bytes long for EOF1 header plus code section (but no data section)
container.length > 7 &&
// EOF1 code section indicator
container[3] === secCode &&
// EOF1 header terminator
container[6] === secTerminator
) {
sectionSizes.code = (container[4] << 8) | container[5]
// Calculate expected length of EOF1 container based on code section
computedContainerSize = 7 + sectionSizes.code
// EOF1 code section must be at least 1 byte long
if (sectionSizes.code < 1) return
} else if (
// EOF1 container must be more than 10 bytes long if data section is included
container.length > 10 &&
// EOF1 code section indicator
container[3] === secCode &&
// EOF1 data section indicator
container[6] === secData &&
// EOF1 header terminator
container[9] === secTerminator
) {
sectionSizes.code = (container[4] << 8) | container[5]
sectionSizes.data = (container[7] << 8) | container[8]
// Calculate expected length of EOF1 container based on code and data sections
computedContainerSize = 10 + sectionSizes.code + sectionSizes.data
// Code & Data sizes cannot be 0
if (sectionSizes.code < 1 || sectionSizes.data < 1) return
}
if (container.length !== computedContainerSize) {
// Computed container length based on section details does not match length of actual bytecode
return
}
return sectionSizes
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is checking on so many conditions in such a condensed form that it is extremely hard to review and generally follow. Can we have this dramatically [TM] more commented, so basically to a comment with a condition on more or less every line of code? Guess that would make things easier to grasp.

Also: one technique @ryanio often applies and I guess which would help here as well is to instead of checking for the positive condition (if (container[1] === magic && container[2] === version) {) and then nesting further down check for the negative (if (container[1] !== magic || container[2] !== version) {) and the directly do the return. This prevents further nesting and could be minimally applied on this example I made but I guess also on other code parts.


export const eof1ValidOpcodes = (code: Buffer) => {
// EIP-3670 - validate all opcodes
const opcodes = new Set(handlers.keys())
opcodes.add(0xfe) // Add INVALID opcode to set

let x = 0
while (x < code.length) {
const opcode = code[x]
x++
if (!opcodes.has(opcode)) {
// No invalid/undefined opcodes
return false
}
if (opcode >= 0x60 && opcode <= 0x7f) {
// Skip data block following push
x += opcode - 0x5f
if (x > code.length - 1) {
// Push blocks must not exceed end of code section
return false
}
}
}
const terminatingOpcodes = new Set([0x00, 0xf3, 0xfd, 0xfe, 0xff])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh wow, missed this 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, me too. But writing those new tests you suggested helped me catch it 🎆

// Per EIP-3670, the final opcode of a code section must be STOP, RETURN, REVERT, INVALID, or SELFDESTRUCT
if (!terminatingOpcodes.has(code[code.length - 1])) {
return false
}
return true
}
1 change: 1 addition & 0 deletions packages/vm/src/exceptions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ export enum ERROR {
INVALID_RETURNSUB = 'invalid RETURNSUB',
INVALID_JUMPSUB = 'invalid JUMPSUB',
INVALID_BYTECODE_RESULT = 'invalid bytecode deployed',
INVALID_EOF_FORMAT = 'invalid EOF format',
INITCODE_SIZE_VIOLATION = 'initcode exceeds max initcode size',

// BLS errors
Expand Down
2 changes: 1 addition & 1 deletion packages/vm/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -205,7 +205,7 @@ export default class VM extends AsyncEventEmitter {
if (opts.common) {
// Supported EIPs
const supportedEIPs = [
1559, 2315, 2537, 2565, 2718, 2929, 2930, 3198, 3529, 3541, 3607, 3855, 3860,
1559, 2315, 2537, 2565, 2718, 2929, 2930, 3198, 3529, 3540, 3541, 3607, 3670, 3855, 3860,
]
for (const eip of opts.common.eips()) {
if (!supportedEIPs.includes(eip)) {
Expand Down
Loading