Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support Arbitrum L2->L1 message using CLI self-relay option #3986

Merged
merged 140 commits into from
Aug 9, 2024
Merged
Changes from 1 commit
Commits
Show all changes
140 commits
Select commit Hold shift + click to select a range
2b3812d
Implement strawman mailbox polling self relay
yorhodes May 1, 2024
e87bbfc
Handle already delivered in self relay
yorhodes May 1, 2024
6379420
Implement get processed receipt
yorhodes May 1, 2024
ea220d0
Implement metadata builders
yorhodes May 2, 2024
3cdb9cd
Dedupe infra and SDK s3 utils
yorhodes May 2, 2024
cd5fab5
Add back trusted relayer signer check
yorhodes May 2, 2024
ea8dd8b
Unify client-s3 deps
yorhodes May 2, 2024
d5f140e
Fix lint
yorhodes May 2, 2024
ca1dd22
Add logs to relay logic
yorhodes May 2, 2024
33eba47
Test aggregation metadata builder
yorhodes May 3, 2024
a01bcf8
Fix tests
yorhodes May 3, 2024
e7d7c2b
Use solidity generated aggregation fixtures
yorhodes May 3, 2024
8f4a214
Test multisig encoder with fixtures
yorhodes May 3, 2024
f115d48
Fix lint
yorhodes May 3, 2024
b742d9e
Make fixtures use consistent
yorhodes May 3, 2024
dff6149
Improve validator signature fetching
yorhodes May 3, 2024
5503507
Add logs to multisig builder
yorhodes May 3, 2024
e2486b4
Import SignatureLike properly
yorhodes May 3, 2024
a394b5a
Add fixtures to pjson
yorhodes May 3, 2024
58af80a
Use assert in place of if throw
yorhodes May 3, 2024
d6eef0c
Add max recurse depth
yorhodes May 3, 2024
be43eec
Add deepFind obj util
yorhodes May 4, 2024
664c84f
Add s3 caching
yorhodes May 4, 2024
207214f
Use hook context in metadata builders
yorhodes May 4, 2024
3e83957
Add hook and ism config caching to core
yorhodes May 4, 2024
5adae48
Export merkle tree event type
yorhodes May 4, 2024
ba41373
Fix lint
yorhodes May 4, 2024
ffb8f37
Revert unnecessary change
yorhodes May 4, 2024
baedddc
Pass through dispatch tx from CLI
yorhodes May 4, 2024
be2407d
Add relay option for dispatch tx hash
yorhodes May 4, 2024
b26e8aa
Add tx receipt to polling relayer
yorhodes May 4, 2024
9e03c11
Fix metadata building bugs
yorhodes May 5, 2024
9551c34
Simplify core interface
yorhodes May 5, 2024
16c9f8d
Separate relayer from core
yorhodes May 5, 2024
d7ab146
Implement metadata builders
yorhodes May 2, 2024
40011d3
Test aggregation metadata builder
yorhodes May 3, 2024
7043320
Fix tests
yorhodes May 3, 2024
15082e5
Use solidity generated aggregation fixtures
yorhodes May 3, 2024
d022df3
Test multisig encoder with fixtures
yorhodes May 3, 2024
b03b55e
Fix lint
yorhodes May 3, 2024
fff1381
Make fixtures use consistent
yorhodes May 3, 2024
ed244ed
Add logs to multisig builder
yorhodes May 3, 2024
12023ba
Import SignatureLike properly
yorhodes May 3, 2024
78d462d
Add fixtures to pjson
yorhodes May 3, 2024
95611e8
Use assert in place of if throw
yorhodes May 3, 2024
3a99ce5
Restore core
yorhodes May 5, 2024
3478594
Restore builders to static
yorhodes May 5, 2024
7eca416
Restore s3 and utils
yorhodes May 5, 2024
3e37fe6
Add networked builders
yorhodes May 5, 2024
d65183e
Pull in aws changes
yorhodes May 5, 2024
023fb7b
Merge branch 'ts-metadata-builders-build' into self-relay-polling
yorhodes May 5, 2024
b73e8ee
Improve relayer API
yorhodes May 6, 2024
3c1f4a1
Add caching to infra relay script
yorhodes May 6, 2024
88d3457
Infer dispatch tx from message ID
yorhodes May 6, 2024
3b39262
Simplify message status command
yorhodes May 6, 2024
9ec2571
Improve status logs
yorhodes May 6, 2024
c4e53ef
Address more pr comments
yorhodes May 6, 2024
f01bbb5
Hook schema fixes
yorhodes May 8, 2024
0259fcc
Add zod schema for relayer cache
yorhodes May 8, 2024
5c544da
init
aroralanuk May 27, 2024
4fbcd1a
forge install: nitro-contracts
aroralanuk May 27, 2024
068ff5a
sendTxToL1
aroralanuk May 27, 2024
23fac90
forge install: foundry-arbitrum
aroralanuk May 27, 2024
7fc660f
rm foundry-arb
aroralanuk May 27, 2024
658aa64
add event
aroralanuk May 27, 2024
9d97aea
Fix infra build
yorhodes May 28, 2024
85db35f
Merge branch 'main' into self-relay-polling
yorhodes May 28, 2024
1b5ddad
Discard changes to solidity/test/isms/AggregationIsm.t.sol
yorhodes May 28, 2024
eceb2d8
temp deploy script
aroralanuk May 30, 2024
45b13d0
just contracts
aroralanuk May 30, 2024
1ce117a
Merge branch 'kunal/arb-l2-hook-contracts' into kunal/arb-l2-l1-hook
aroralanuk May 30, 2024
7c09ade
deploy hook script
aroralanuk Jun 1, 2024
0c12f2f
direct mailbox call
aroralanuk Jun 6, 2024
aa3f5e5
change to exeTx in verify
aroralanuk Jun 11, 2024
e23d2f7
lock on verifyMessageId
aroralanuk Jun 13, 2024
b5a3e96
add tests
aroralanuk Jun 14, 2024
070c182
revert temp changes
aroralanuk Jun 14, 2024
f7f8b20
merged with main
aroralanuk Jun 14, 2024
ba4812b
Merge branch 'self-relay-polling' into kunal/arb-l1-selfrelay
aroralanuk Jun 14, 2024
4d0585c
deploy new TR
aroralanuk Jun 18, 2024
8527c46
change recipient
aroralanuk Jun 18, 2024
8ade1e7
Merge branch 'main' into kunal/arb-l1-selfrelay
aroralanuk Jun 18, 2024
fa5ca46
commit
aroralanuk Jun 19, 2024
c61970a
customHook
aroralanuk Jun 24, 2024
b4ab0d9
add latest
aroralanuk Jun 26, 2024
59a7da4
Merge branch 'main' into kunal/arb-l2-l1-hook
aroralanuk Jun 26, 2024
0bb418c
merge
aroralanuk Jun 26, 2024
f33e775
Merge branch 'kunal/arb-l2-l1-hook' into kunal/arb-l1-selfrelay
aroralanuk Jun 26, 2024
56fb53c
changes from selrelay
aroralanuk Jun 26, 2024
89395a2
changes for async outbox call
aroralanuk Jun 27, 2024
b555b74
add gas estimate
aroralanuk Jun 27, 2024
e2da11f
revert
aroralanuk Jun 27, 2024
3f6a18f
Update CODE_OF_CONDUCT.md
aroralanuk Jun 27, 2024
734c330
forge-std
aroralanuk Jun 27, 2024
88e7eeb
Merge branch 'kunal/arb-l2-l1-hook' into kunal/arb-l1-selfrelay
aroralanuk Jun 27, 2024
6bee384
deploy new contracts
aroralanuk Jun 27, 2024
6b9e4c2
adjust to read verifyMessage first
aroralanuk Jun 27, 2024
65d73ff
address yorke's comments
aroralanuk Jun 27, 2024
b5a78f3
switch to slicing
aroralanuk Jun 28, 2024
db6cef9
Merge branch 'kunal/arb-l2-l1-hook' into kunal/arb-l1-selfrelay
aroralanuk Jun 28, 2024
efcc07f
isVerified
aroralanuk Jun 28, 2024
c1a581b
Merge branch 'main' into self-relay-polling
aroralanuk Jun 28, 2024
3eb8e01
fix sdk
aroralanuk Jun 28, 2024
16f6d2c
revert core changes
aroralanuk Jun 28, 2024
fcf5def
reorder
aroralanuk Jun 28, 2024
ba5c547
revert index
aroralanuk Jun 28, 2024
6eee086
rm buildMetadata from core
aroralanuk Jun 28, 2024
c4dfdc4
Merge branch 'self-relay-polling' into kunal/arb-l1-selfrelay
aroralanuk Jul 1, 2024
a1d9c26
merge changes
aroralanuk Jul 1, 2024
44bcc14
cleanup
aroralanuk Jul 1, 2024
03d3264
changeset
aroralanuk Jul 1, 2024
7028332
Merge branch 'main' into kunal/arb-l1-selfrelay
aroralanuk Jul 17, 2024
cd2ca89
merge changes
aroralanuk Jul 17, 2024
3b8a0d8
Enable metadata builder unit tests
yorhodes Jul 23, 2024
dd0db06
deploy arb
aroralanuk Jul 23, 2024
3cb7cb2
omit arb unit test
aroralanuk Jul 23, 2024
6593b6e
Merge branch 'enable-builder-tests' into kunal/arb-l1-selfrelay
aroralanuk Jul 23, 2024
456c9af
evmHookModule
aroralanuk Jul 24, 2024
43134ae
changeset
aroralanuk Jul 24, 2024
c86e7f7
Merge branch 'kunal/metadata-builder-hook-crud' into kunal/arb-l1-sel…
aroralanuk Jul 24, 2024
33bc79e
inter
aroralanuk Jul 25, 2024
0c6997c
arb test still broken
aroralanuk Jul 31, 2024
2abb27c
arb route debugging
aroralanuk Aug 1, 2024
22b293a
arb l2 to l1 bridge
aroralanuk Aug 2, 2024
599f56c
change
aroralanuk Aug 5, 2024
e2f1223
arb tests working OK
aroralanuk Aug 7, 2024
49e3998
rm arb test in deployment
aroralanuk Aug 7, 2024
b6a5336
yorke's comments
aroralanuk Aug 7, 2024
4abf6fe
changeset
aroralanuk Aug 7, 2024
8416528
Merge branch 'main' into kunal/arb-l1-selfrelay
aroralanuk Aug 7, 2024
ab216c9
revert
aroralanuk Aug 7, 2024
a86c479
yarn build works again
aroralanuk Aug 8, 2024
e21efdc
fixes
aroralanuk Aug 8, 2024
2db3200
inherit type from L2ToL1Event
aroralanuk Aug 8, 2024
5649b51
import decoding type from IOutbox interface
aroralanuk Aug 8, 2024
0dc08ee
adding config for arb
aroralanuk Aug 8, 2024
d9841c3
revert
aroralanuk Aug 8, 2024
071b708
omit arb from hook deployer test
aroralanuk Aug 8, 2024
642d64b
rm callvalue from encoding/decoding
aroralanuk Aug 8, 2024
415bb13
rm gasoverhead
aroralanuk Aug 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Test multisig encoder with fixtures
  • Loading branch information
yorhodes committed May 3, 2024
commit 8f4a214144b645623b8df2c0b85d5d73b14b0625
3 changes: 2 additions & 1 deletion solidity/test/isms/AggregationIsm.t.sol
Original file line number Diff line number Diff line change
@@ -58,10 +58,11 @@ contract AggregationIsmTest is Test {
offsets = bytes.concat(offsets, abi.encodePacked(offset));
start = end;
metametadata = abi.encodePacked(metametadata, metadata);
vm.serializeBytes(structured, i.toString(), metadata);
} else {
offsets = bytes.concat(offsets, abi.encodePacked(uint64(0)));
vm.serializeString(structured, i.toString(), "null");
}
vm.serializeBytes(structured, i.toString(), metadata);
}

string memory path = string(
118 changes: 89 additions & 29 deletions solidity/test/isms/MultisigIsm.t.sol
Original file line number Diff line number Diff line change
@@ -3,6 +3,8 @@ pragma solidity ^0.8.13;

import "forge-std/Test.sol";

import "@openzeppelin/contracts/utils/Strings.sol";

import {IMultisigIsm} from "../../contracts/interfaces/isms/IMultisigIsm.sol";
import {TestMailbox} from "../../contracts/test/TestMailbox.sol";
import {StaticMerkleRootMultisigIsmFactory, StaticMessageIdMultisigIsmFactory} from "../../contracts/isms/multisig/StaticMultisigIsm.sol";
@@ -20,6 +22,8 @@ import {ThresholdTestUtils} from "./IsmTestUtils.sol";
abstract contract AbstractMultisigIsmTest is Test {
using Message for bytes;
using TypeCasts for address;
using Strings for uint256;
using Strings for uint8;

uint32 constant ORIGIN = 11;
StaticThresholdAddressSetFactory factory;
@@ -30,32 +34,72 @@ abstract contract AbstractMultisigIsmTest is Test {

function metadataPrefix(
bytes memory message
) internal view virtual returns (bytes memory);
) internal virtual returns (bytes memory, string memory);

function getMetadata(
uint8 m,
uint8 n,
bytes32 seed,
bytes memory message
) internal returns (bytes memory) {
uint32 domain = mailbox.localDomain();
uint256[] memory keys = addValidators(m, n, seed);
uint256[] memory signers = ThresholdTestUtils.choose(m, keys, seed);
string memory structured = "structured";

bytes32 digest;
{
uint32 domain = mailbox.localDomain();
(bytes32 root, uint32 index) = merkleTreeHook.latestCheckpoint();
bytes32 messageId = message.id();
bytes32 merkleTreeAddress = address(merkleTreeHook)
.addressToBytes32();
digest = CheckpointLib.digest(
domain,
merkleTreeAddress,
root,
index,
messageId
);
}

(bytes32 root, uint32 index) = merkleTreeHook.latestCheckpoint();
bytes32 messageId = message.id();
bytes32 digest = CheckpointLib.digest(
domain,
address(merkleTreeHook).addressToBytes32(),
root,
index,
messageId
uint256[] memory signers = ThresholdTestUtils.choose(
m,
addValidators(m, n, seed),
seed
);
bytes memory metadata = metadataPrefix(message);

vm.serializeUint(structured, "type", uint256(ism.moduleType()));

(bytes memory metadata, string memory prefix) = metadataPrefix(message);
vm.serializeString(structured, "prefix", prefix);

string memory signatures = "signatures";
for (uint256 i = 0; i < m; i++) {
(uint8 v, bytes32 r, bytes32 s) = vm.sign(signers[i], digest);
metadata = abi.encodePacked(metadata, r, s, v);

string memory signature = "signature";
vm.serializeUint(signature, "v", uint256(v));
vm.serializeBytes32(signature, "r", r);
signature = vm.serializeBytes32(signature, "s", s);
vm.serializeString(signatures, i.toString(), signature);
}

vm.serializeString(
structured,
"signatures",
vm.serializeString(signatures, "dummy", "dummy") // idk
);

string memory path = string(
abi.encodePacked(
"./fixtures/multisig/",
m.toString(),
"-",
n.toString(),
".json"
)
);

vm.writeJson(vm.serializeBytes(structured, "encoded", metadata), path);
return metadata;
}

@@ -125,6 +169,7 @@ abstract contract AbstractMultisigIsmTest is Test {
contract MerkleRootMultisigIsmTest is AbstractMultisigIsmTest {
using TypeCasts for address;
using Message for bytes;
using Strings for uint256;

function setUp() public {
mailbox = new TestMailbox(ORIGIN);
@@ -138,16 +183,31 @@ contract MerkleRootMultisigIsmTest is AbstractMultisigIsmTest {
// TODO: test merkleIndex != signedIndex
function metadataPrefix(
bytes memory message
) internal view override returns (bytes memory) {
) internal override returns (bytes memory metadata, string memory prefix) {
uint32 checkpointIndex = uint32(merkleTreeHook.count() - 1);
return
abi.encodePacked(
address(merkleTreeHook).addressToBytes32(),
checkpointIndex,
message.id(),
merkleTreeHook.proof(),
checkpointIndex
);
bytes32[32] memory proof = merkleTreeHook.proof();
bytes32 messageId = message.id();
bytes32 merkleTreeAddress = address(merkleTreeHook).addressToBytes32();

prefix = "prefix";
vm.serializeUint(prefix, "index", uint256(checkpointIndex));
vm.serializeUint(prefix, "signedIndex", uint256(checkpointIndex));
vm.serializeBytes32(prefix, "merkleTree", merkleTreeAddress);
string memory proofString = "proof";
for (uint256 i = 0; i < 32; i++) {
vm.serializeBytes32(proofString, i.toString(), proof[i]);
}
proofString = vm.serializeString(proofString, "dummy", "dummy");
vm.serializeString(prefix, "proof", proofString);
prefix = vm.serializeBytes32(prefix, "id", messageId);

metadata = abi.encodePacked(
merkleTreeAddress,
checkpointIndex,
messageId,
proof,
checkpointIndex
);
}
}

@@ -166,13 +226,13 @@ contract MessageIdMultisigIsmTest is AbstractMultisigIsmTest {

function metadataPrefix(
bytes memory
) internal view override returns (bytes memory) {
) internal override returns (bytes memory metadata, string memory prefix) {
(bytes32 root, uint32 index) = merkleTreeHook.latestCheckpoint();
return
abi.encodePacked(
address(merkleTreeHook).addressToBytes32(),
root,
index
);
bytes32 merkleTreeAddress = address(merkleTreeHook).addressToBytes32();
prefix = "prefix";
vm.serializeBytes32(prefix, "root", root);
vm.serializeUint(prefix, "signedIndex", uint256(index));
prefix = vm.serializeBytes32(prefix, "merkleTree", merkleTreeAddress);
metadata = abi.encodePacked(merkleTreeAddress, root, index);
}
}
11 changes: 7 additions & 4 deletions typescript/sdk/src/ism/metadata/aggregation.test.ts
Original file line number Diff line number Diff line change
@@ -14,10 +14,13 @@ const path = '../../solidity/fixtures/aggregation';
const files = readdirSync(path);
const fixtures: Fixture[] = files
.map((f) => JSON.parse(readFileSync(`${path}/${f}`, 'utf8')))
.map((contents) => ({
...contents,
submoduleMetadata: Object.values(contents),
}));
.map((contents) => {
const { encoded, ...values } = contents;
return {
encoded,
submoduleMetadata: Object.values(values),
};
});

describe('AggregationMetadataBuilder', () => {
fixtures.forEach((fixture, i) => {
31 changes: 18 additions & 13 deletions typescript/sdk/src/ism/metadata/aggregation.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { WithAddress, ensure0x, strip0x } from '@hyperlane-xyz/utils';
import { WithAddress, fromHexString, toHexString } from '@hyperlane-xyz/utils';

import { DispatchedMessage } from '../../core/types.js';
import { DerivedIsmConfigWithAddress } from '../read.js';
@@ -7,7 +7,7 @@ import { AggregationIsmConfig } from '../types.js';
import { BaseMetadataBuilder, MetadataBuilder } from './builder.js';

export interface AggregationIsmMetadata {
submoduleMetadata: string[];
submoduleMetadata: Array<string | null>;
}

const RANGE_SIZE = 4;
@@ -41,38 +41,43 @@ export class AggregationIsmMetadataBuilder

let encoded = Buffer.alloc(rangeSize, 0);
metadata.submoduleMetadata.forEach((meta, index) => {
if (strip0x(meta).length === 0) {
if (meta === null) {
return;
}

const start = encoded.length;
encoded = Buffer.concat([encoded, Buffer.from(meta, 'hex')]);
encoded = Buffer.concat([encoded, fromHexString(meta)]);
const end = encoded.length;

const rangeStart = this.rangeIndex(index);
encoded.writeUint32BE(start, rangeStart);
encoded.writeUint32BE(end, rangeStart + RANGE_SIZE);
});

return ensure0x(encoded.toString('hex'));
return toHexString(encoded);
}

static metadataRange(metadata: string, index: number): string {
static metadataRange(
metadata: string,
index: number,
): { start: number; end: number; encoded: string } {
const rangeStart = this.rangeIndex(index);
const encoded = Buffer.from(metadata, 'hex');
const encoded = fromHexString(metadata);
const start = encoded.readUint32BE(rangeStart);
const end = encoded.readUint32BE(rangeStart + RANGE_SIZE);
return ensure0x(encoded.subarray(start, end).toString('hex'));
}

static hasMetadata(metadata: string, index: number): boolean {
return this.metadataRange(metadata, index).length > 0;
return {
start,
end,
encoded: toHexString(encoded.subarray(start, end)),
};
}

static decode(metadata: string, count: number): AggregationIsmMetadata {
const submoduleMetadata = [];
for (let i = 0; i < count; i++) {
submoduleMetadata.push(this.metadataRange(metadata, i));
const range = this.metadataRange(metadata, i);
const submeta = range.start > 0 ? range.encoded : null;
submoduleMetadata.push(submeta);
}
return { submoduleMetadata };
}
71 changes: 71 additions & 0 deletions typescript/sdk/src/ism/metadata/multisig.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
import { expect } from 'chai';
import { readFileSync, readdirSync } from 'fs';

import { SignatureLike } from '@hyperlane-xyz/utils';

import { ModuleType } from '../types.js';

import { MultisigMetadata, MultisigMetadataBuilder } from './multisig.js';

type Fixture = {
decoded: MultisigMetadata;
encoded: string;
};

const path = '../../solidity/fixtures/multisig';
const files = readdirSync(path);
const fixtures: Fixture[] = files
.map((f) => JSON.parse(readFileSync(`${path}/${f}`, 'utf8')))
.map((contents) => {
const type = contents.type as MultisigMetadata['type'];

const { dummy, ...signatureValues } = contents.signatures;
const signatures = Object.values<SignatureLike>(signatureValues);

let decoded: MultisigMetadata;
if (type === ModuleType.MERKLE_ROOT_MULTISIG) {
const { dummy, ...branchValues } = contents.prefix.proof;
const branch = Object.values<string>(branchValues);
decoded = {
type,
proof: {
branch,
leaf: contents.prefix.id,
index: contents.prefix.signedIndex,
},
checkpoint: {
root: '',
index: contents.prefix.index,
merkle_tree_hook_address: contents.prefix.merkleTree,
},
signatures,
};
} else {
decoded = {
type,
checkpoint: {
root: contents.prefix.root,
index: contents.prefix.signedIndex,
merkle_tree_hook_address: contents.prefix.merkleTree,
},
signatures,
};
}
return { decoded, encoded: contents.encoded };
});

describe('MultisigMetadataBuilder', () => {
fixtures.forEach((fixture, i) => {
it(`should encode fixture ${i}`, () => {
expect(MultisigMetadataBuilder.encode(fixture.decoded)).to.equal(
fixture.encoded,
);
});

it(`should decode fixture ${i}`, () => {
expect(
MultisigMetadataBuilder.decode(fixture.encoded, fixture.decoded.type),
).to.deep.equal(fixture.decoded);
});
});
});
Loading
Loading