-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for tuples #1235
Add support for tuples #1235
Conversation
I'm adding things bit by bit here. The most recent chunk of work done was to rewrite the Eventually, I'll give things another once over to make sure I didn't miss anything. Known issues:
|
web3/_utils/abi.py
Outdated
# Component is tuple (possibly tuple list)... | ||
|
||
array_dims = abi_type_str[TUPLE_PREFIX_LEN:] | ||
if array_dims: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we tighten this up at all? Match against a regex or something? This jumps out as a good way to end up with weird errors jumping out in the case that the ABI type string is malformed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good idea. I'll add that.
web3/_utils/abi.py
Outdated
new_comp = copy.copy(comp) | ||
new_comp['type'] = TUPLE_PREFIX | ||
|
||
# Assume `arg` is iterable |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same here, easy to do a quick runtime check that isinstance(arg, collections.abc.Iterable)
or maybe tighter as not isinstance(arg, (bytes, str)) and isinstance(arg, collections.abc.Sequence)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a runtime type check using eth_utils.is_list_like
.
web3/_utils/abi.py
Outdated
|
||
if isinstance(arg, abc.Mapping): | ||
# Arg is mapping. Convert to properly ordered sequence. | ||
arg = tuple(arg[c['name']] for c in sub_comps) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's going to take me a long time to not have a knee-jerk reflex that python dictionaries aren't ordered (since they now are)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep, I know they're ordered. I still think that most computer scientists assume that mappings provide no guarantees about any order that results from the internal representation of the data structure. Users could also provide a dictionary with all the right keys but the wrong order.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you meant sub_comps
, I believe that should always be a list
instance or an itertools.repeat
iterator.
f4fdafc
to
b35c254
Compare
c7c6d19
to
85c1837
Compare
8e76bd8
to
808fa92
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Most comments are about style: For example, I find single-letter and abbreviated variables a chore to read through, even in single-line comprehensions. There are quite a few, and I only commented on a couple of them.
The only substantive piece is that we should probably add tests for nested arrays, and some other deep-nested scenarios, like a double-nested tuple in a double-nested array.
""" | ||
CONTRACT_TUPLE_CODE = '0x608060405234801561001057600080fd5b506108ca806100206000396000f300608060405260043610610041576000357c0100000000000000000000000000000000000000000000000000000000900463ffffffff1680638e1ae3c714610046575b600080fd5b34801561005257600080fd5b5061006d60048036036100689190810190610403565b610083565b60405161007a9190610696565b60405180910390f35b61008b610093565b819050919050565b6060604051908101604052806000815260200160608152602001606081525090565b60006100c18235610850565b905092915050565b600082601f83011215156100dc57600080fd5b81356100ef6100ea826106e5565b6106b8565b9150818183526020840193506020810190508385602084028201111561011457600080fd5b60005b83811015610144578161012a88826100b5565b845260208401935060208301925050600181019050610117565b5050505092915050565b600082601f830112151561016157600080fd5b600261017461016f8261070d565b6106b8565b9150818385602084028201111561018a57600080fd5b60005b838110156101ba57816101a088826102bf565b84526020840193506020830192505060018101905061018d565b5050505092915050565b600082601f83011215156101d757600080fd5b81356101ea6101e58261072f565b6106b8565b9150818183526020840193506020810190508360005b8381101561023057813586016102168882610377565b845260208401935060208301925050600181019050610200565b5050505092915050565b600082601f830112151561024d57600080fd5b813561026061025b82610757565b6106b8565b9150818183526020840193506020810190508385602084028201111561028557600080fd5b60005b838110156102b5578161029b88826103ef565b845260208401935060208301925050600181019050610288565b5050505092915050565b60006102cb8235610870565b905092915050565b60006102df823561087c565b905092915050565b6000606082840312156102f957600080fd5b61030360606106b8565b90506000610313848285016103ef565b600083015250602082013567ffffffffffffffff81111561033357600080fd5b61033f8482850161023a565b602083015250604082013567ffffffffffffffff81111561035f57600080fd5b61036b848285016101c4565b60408301525092915050565b60006080828403121561038957600080fd5b61039360606106b8565b905060006103a3848285016102d3565b60008301525060206103b78482850161014e565b602083015250606082013567ffffffffffffffff8111156103d757600080fd5b6103e3848285016100c9565b60408301525092915050565b60006103fb8235610886565b905092915050565b60006020828403121561041557600080fd5b600082013567ffffffffffffffff81111561042f57600080fd5b61043b848285016102e7565b91505092915050565b61044d81610810565b82525050565b600061045e826107b0565b8084526020840193506104708361077f565b60005b828110156104a257610486868351610444565b61048f826107dc565b9150602086019550600181019050610473565b50849250505092915050565b6104b7816107bb565b6104c08261078c565b60005b828110156104f2576104d68583516105c2565b6104df826107e9565b91506020850194506001810190506104c3565b5050505050565b6000610504826107c6565b8084526020840193508360208202850161051d85610796565b60005b84811015610556578383038852610538838351610637565b9250610543826107f6565b9150602088019750600181019050610520565b508196508694505050505092915050565b6000610572826107d1565b808452602084019350610584836107a3565b60005b828110156105b65761059a868351610687565b6105a382610803565b9150602086019550600181019050610587565b50849250505092915050565b6105cb81610830565b82525050565b6105da8161083c565b82525050565b60006060830160008301516105f86000860182610687565b50602083015184820360208601526106108282610567565b9150506040830151848203604086015261062a82826104f9565b9150508091505092915050565b600060808301600083015161064f60008601826105d1565b50602083015161066260208601826104ae565b506040830151848203606086015261067a8282610453565b9150508091505092915050565b61069081610846565b82525050565b600060208201905081810360008301526106b081846105e0565b905092915050565b6000604051905081810181811067ffffffffffffffff821117156106db57600080fd5b8060405250919050565b600067ffffffffffffffff8211156106fc57600080fd5b602082029050602081019050919050565b600067ffffffffffffffff82111561072457600080fd5b602082029050919050565b600067ffffffffffffffff82111561074657600080fd5b602082029050602081019050919050565b600067ffffffffffffffff82111561076e57600080fd5b602082029050602081019050919050565b6000602082019050919050565b6000819050919050565b6000602082019050919050565b6000602082019050919050565b600081519050919050565b600060029050919050565b600081519050919050565b600081519050919050565b6000602082019050919050565b6000602082019050919050565b6000602082019050919050565b6000602082019050919050565b600073ffffffffffffffffffffffffffffffffffffffff82169050919050565b60008115159050919050565b6000819050919050565b6000819050919050565b600073ffffffffffffffffffffffffffffffffffffffff82169050919050565b60008115159050919050565b6000819050919050565b60008190509190505600a265627a7a72305820fb4abae14c4bcd2cd8a12f35862cb09a2e0f9004eb5fdbfdb76658a753570c8c6c6578706572696d656e74616cf50037' # noqa: E501 | ||
CONTRACT_TUPLE_RUNTIME = '0x608060405260043610610041576000357c0100000000000000000000000000000000000000000000000000000000900463ffffffff1680638e1ae3c714610046575b600080fd5b34801561005257600080fd5b5061006d60048036036100689190810190610403565b610083565b60405161007a9190610696565b60405180910390f35b61008b610093565b819050919050565b6060604051908101604052806000815260200160608152602001606081525090565b60006100c18235610850565b905092915050565b600082601f83011215156100dc57600080fd5b81356100ef6100ea826106e5565b6106b8565b9150818183526020840193506020810190508385602084028201111561011457600080fd5b60005b83811015610144578161012a88826100b5565b845260208401935060208301925050600181019050610117565b5050505092915050565b600082601f830112151561016157600080fd5b600261017461016f8261070d565b6106b8565b9150818385602084028201111561018a57600080fd5b60005b838110156101ba57816101a088826102bf565b84526020840193506020830192505060018101905061018d565b5050505092915050565b600082601f83011215156101d757600080fd5b81356101ea6101e58261072f565b6106b8565b9150818183526020840193506020810190508360005b8381101561023057813586016102168882610377565b845260208401935060208301925050600181019050610200565b5050505092915050565b600082601f830112151561024d57600080fd5b813561026061025b82610757565b6106b8565b9150818183526020840193506020810190508385602084028201111561028557600080fd5b60005b838110156102b5578161029b88826103ef565b845260208401935060208301925050600181019050610288565b5050505092915050565b60006102cb8235610870565b905092915050565b60006102df823561087c565b905092915050565b6000606082840312156102f957600080fd5b61030360606106b8565b90506000610313848285016103ef565b600083015250602082013567ffffffffffffffff81111561033357600080fd5b61033f8482850161023a565b602083015250604082013567ffffffffffffffff81111561035f57600080fd5b61036b848285016101c4565b60408301525092915050565b60006080828403121561038957600080fd5b61039360606106b8565b905060006103a3848285016102d3565b60008301525060206103b78482850161014e565b602083015250606082013567ffffffffffffffff8111156103d757600080fd5b6103e3848285016100c9565b60408301525092915050565b60006103fb8235610886565b905092915050565b60006020828403121561041557600080fd5b600082013567ffffffffffffffff81111561042f57600080fd5b61043b848285016102e7565b91505092915050565b61044d81610810565b82525050565b600061045e826107b0565b8084526020840193506104708361077f565b60005b828110156104a257610486868351610444565b61048f826107dc565b9150602086019550600181019050610473565b50849250505092915050565b6104b7816107bb565b6104c08261078c565b60005b828110156104f2576104d68583516105c2565b6104df826107e9565b91506020850194506001810190506104c3565b5050505050565b6000610504826107c6565b8084526020840193508360208202850161051d85610796565b60005b84811015610556578383038852610538838351610637565b9250610543826107f6565b9150602088019750600181019050610520565b508196508694505050505092915050565b6000610572826107d1565b808452602084019350610584836107a3565b60005b828110156105b65761059a868351610687565b6105a382610803565b9150602086019550600181019050610587565b50849250505092915050565b6105cb81610830565b82525050565b6105da8161083c565b82525050565b60006060830160008301516105f86000860182610687565b50602083015184820360208601526106108282610567565b9150506040830151848203604086015261062a82826104f9565b9150508091505092915050565b600060808301600083015161064f60008601826105d1565b50602083015161066260208601826104ae565b506040830151848203606086015261067a8282610453565b9150508091505092915050565b61069081610846565b82525050565b600060208201905081810360008301526106b081846105e0565b905092915050565b6000604051905081810181811067ffffffffffffffff821117156106db57600080fd5b8060405250919050565b600067ffffffffffffffff8211156106fc57600080fd5b602082029050602081019050919050565b600067ffffffffffffffff82111561072457600080fd5b602082029050919050565b600067ffffffffffffffff82111561074657600080fd5b602082029050602081019050919050565b600067ffffffffffffffff82111561076e57600080fd5b602082029050602081019050919050565b6000602082019050919050565b6000819050919050565b6000602082019050919050565b6000602082019050919050565b600081519050919050565b600060029050919050565b600081519050919050565b600081519050919050565b6000602082019050919050565b6000602082019050919050565b6000602082019050919050565b6000602082019050919050565b600073ffffffffffffffffffffffffffffffffffffffff82169050919050565b60008115159050919050565b6000819050919050565b6000819050919050565b600073ffffffffffffffffffffffffffffffffffffffff82169050919050565b60008115159050919050565b6000819050919050565b60008190509190505600a265627a7a72305820fb4abae14c4bcd2cd8a12f35862cb09a2e0f9004eb5fdbfdb76658a753570c8c6c6578706572696d656e74616cf50037' # noqa: E501 | ||
CONTRACT_TUPLE_ABI = json.loads('[{"constant":true,"inputs":[{"components":[{"name":"a","type":"uint256"},{"name":"b","type":"uint256[]"},{"components":[{"name":"x","type":"int256"},{"name":"y","type":"bool[2]"},{"name":"z","type":"address[]"}],"name":"c","type":"tuple[]"}],"name":"s","type":"tuple"}],"name":"method","outputs":[{"components":[{"name":"a","type":"uint256"},{"name":"b","type":"uint256[]"},{"components":[{"name":"x","type":"int256"},{"name":"y","type":"bool[2]"},{"name":"z","type":"address[]"}],"name":"c","type":"tuple[]"}],"name":"","type":"tuple"}],"payable":false,"stateMutability":"pure","type":"function"}]') # noqa: E501 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit opinion: it's easier to understand what's going on in an ABI with a native python list, like:
CONTRACT_TUPLE_ABI = [
{
"constant": True,
"inputs": [
{
"components":[
{"name":"a", "type":"uint256"},
{"name":"b", "type":"uint256[]"},
{
"components":[
{"name":"x", "type":"int256"},
{"name":"y", "type":"bool[2]"},
{"name":"z", "type":"address[]"}
],
"name":"c",
"type":"tuple[]",
}
],
"name":"s",
"type":"tuple",
}
],
"name":"method",
"outputs":[{
"components":[
{"name":"a", "type":"uint256"},
{"name":"b", "type":"uint256[]"},
{"components":[
{"name":"x", "type":"int256"},
...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was just following the style of existing fixture code with that one.
FALLBACK_FUNCTION = json.loads('[{"constant": false, "inputs": [], "name": "getData", "outputs": [{"name": "r", "type": "uint256"}], "payable": false, "stateMutability": "nonpayable", "type": "function"}, {"payable": false, "stateMutability": "nonpayable", "type": "fallback"}]') # noqa: E501 | ||
MULTIPLE_FUNCTIONS = json.loads(''' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ABI's supplied as strings are already parsed as json, so it's unnecessary to do a json.loads
. Although I still prefer native python over embedding json here. (Definitely not a PR blocker, though)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Was following the style of existing code.
web3/_utils/abi.py
Outdated
@@ -226,7 +234,10 @@ def check_if_arguments_can_be_encoded(function_abi, args, kwargs): | |||
if len(function_abi.get('inputs', [])) != len(arguments): | |||
return False | |||
|
|||
types = get_abi_input_types(function_abi) | |||
try: | |||
types, arguments = get_abi_inputs(function_abi, arguments) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit overwriting the local variable arguments
. I assume this is tuple
related, so maybe we need a different name to distinguish arguments
from typed_arguments
(or something). Since now I guess we don't want to use the old arguments
value for anything except a length check.
web3/_utils/abi.py
Outdated
|
||
if isinstance(args, abc.Mapping): | ||
# `args` is mapping, convert to properly ordered sequence | ||
args = tuple(args[i['name']] for i in inputs) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
args = tuple(args[i['name']] for i in inputs) | |
args = tuple(args[input['name']] for input in inputs) |
nit opinion: even in short comprehensions, I think it's easier to read with a full variable name. It prevents your eyes having to dart back and forth, so you can more or less read it straight through in one pass.
web3/_utils/abi.py
Outdated
|
||
def _convert_abi_input(comp, arg): | ||
""" | ||
Converts an argument ``arg`` corresponding to an ABI component ``comp`` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
component
here seems confusing because each element of a tuple is called component
. Maybe this is a arg_type
, or arg_spec
since it includes a name?
]), | ||
), | ||
), | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like this is already working, but might be nice to add a regression test for nested arrays of tuples. Seems like the kind of thing that could break in a rewrite/refactor.
web3/_utils/abi.py
Outdated
# If type is array, determine item type and annotate all | ||
# items in iterable with that type | ||
item_type_str = abi_type.item_type.to_type_str() | ||
data_value = [abi_sub_tree(item_type_str, i) for i in data_value] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe item
instead of i
? Not sure if that's what i
is short for, here.
web3/_utils/abi.py
Outdated
# items in iterable respectively with those types | ||
data_value = type(data_value)( | ||
abi_sub_tree(comp_type.to_type_str(), i) | ||
for comp_type, i in zip(abi_type.components, data_value) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe component
instead of i
here?
web3/_utils/contracts.py
Outdated
@@ -240,6 +241,8 @@ def get_function_info(fn_name, contract_abi=None, fn_abi=None, args=None, kwargs | |||
|
|||
fn_arguments = merge_args_and_kwargs(fn_abi, args, kwargs) | |||
|
|||
_, fn_arguments = get_abi_inputs(fn_abi, fn_arguments) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit overwrote fn_arguments
, but now it has a slightly different meaning. Maybe typed_arguments
or annotated_arguments
? (same as the one in web3/_utils/abi
I commented on)
@carver I sort of feel like avoiding single letter variables can be just as confusing. In a lot of the examples cited, the cognitive overhead of the short variable names seems pretty minimal. Also, giving things longer names often leads to a code section taking up two or three lines when it could have taken up one. I feel like the overhead of reading a statement that runs across multiple lines is as much if not more than the overhead of reading a statement that uses meaningful abbreviations. Additionally, the typical alternative of using a variable name that simply lacks pluralization is pretty confusing because then two variables only differ by one letter and it's easy to get typos. Anyway, I'll change it if we really want but I've often felt like I don't understand the aversion to that coding pattern. I wanted to explain some of my thoughts about that. |
@davesque can you confirm this is 👍 to merge (i.e. you don't have anything outstanding that you know needs to be addressed). @kclowes or maybe @njgheorghita . IIUC correctly, this gives us support for tuples, but only as raw tuples, and not any rich object representations. I think what comes next is some version of support for those.
It isn't clear to me exactly how these should be implemented but I think doing them in this order probably makes sense. The |
@pipermerriam Yes it's ready to merge. I was waiting for your review. It seemed like a fairly complex and important piece of work so I didn't want to merge after just one review. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Two somewhat nitpicky bits of feedback. Both might be good candidates to just quickly write up in a "Good First Issue" ticket as candy for a new contributor to fix-up.
web3/_utils/abi.py
Outdated
|
||
if isinstance(arg, abc.Mapping): | ||
# Arg is mapping. Align values according to abi order. | ||
arg = tuple(arg[abi['name']] for abi in sub_abis) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have a convention of avoiding overwriting local variables, preferring to instead use new variable names. Maybe we can call this aligned_arg
if is_mapping(arg):
aligned_arg = tuple(...)
else:
aligned_arg = arg
NOTE: the above statement has an implicit
else
clause which is also somewhat against our conventions. It might should look something like.
if is_mapping(arg):
...
elif is_list_like(arg):
...
else:
raise TypeError("Something something invalid")
This would simply be moving the TypeError
from below up to be part of this clause.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also noting this relates to #1267 as it appears we already support dictionaries for the input arguments for tuple types.
web3/_utils/abi.py
Outdated
# If type is array, determine item type and annotate all | ||
# items in iterable with that type | ||
item_type_str = abi_type.item_type.to_type_str() | ||
data_value = [ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same here with convention of not overwriting local variables which if fixed will require this if/elif
to get a final else
clause.
@carver @pipermerriam Thanks guys for the reviews. When the tests pass, I'll go ahead and merge this. |
What was wrong?
This is a rebuild of PR #1147 . Commits retain authorship from original PR where appropriate.
How was it fixed?
Adds support for tuples by modifying low-level utility functions used for ABI dict processing and encodability checks.
Cute Animal Picture