web3.utils.sha3 and web3.utils.soliditySha3 return null when an empty byte sequence is hashed

[smarx]

Based on the code, this appears to be done intentionally for some reason, but it seems like a very bad idea to return the wrong answer. There's also no mention of this non-standard behavior in the documentation.

From https://github.com/ethereum/web3.js/blob/1.0/packages/web3-utils/src/utils.js#L434,L435:

var SHA3_NULL_S = '0xc5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470';
...
    if(returnValue === SHA3_NULL_S) {
        return null;

Removing those lines would fix the problem.

[jefflau]

I've had this issue crop up in ENS as well.

[nivida]

I would propose to add an additional parameter called raw_null which would be of type boolean and does configure the function to return the real hash value of it instead of the mapped null value.

@cgewecke @smarx @jefflau do you agree with this solution?

[smarx]

I would much rather see it return the actual hash by default, but it's better than nothing.

I imagine the argument against fixing this is concern about backward compatibility... maybe it's possible to deprecate the old function and create a new one that has the fix by default?

[nivida]

I imagine the argument against fixing this is concern about backward compatibility... maybe it's possible to deprecate the old function and create a new one that has the fix by default?

Yes, this is correct. I think adding new functions with close to the same behavior will lead to confusion by most developers. It's better to just add a configuration property for now and to apply the breaking change later with a minor release.

[nivida]

@smarx I've just applied this change to the utils.sha3 method which does work but because of the dynamic parameter detection of the utils.soliditySha3 method do we actually have to create new functions.