Block.timestamp in UNIX nanoseconds outputBlockFormatter in 1.2.6 (quorum, RAFT)

[VladoDemcak]

We are using quorum with RAFT consensus. Once a block is mined the blockchain returns the block with timestamp in UNIX format with nanoseconds.

Example:
block.timestamp is "0x16015c8446c19c16" (decimal: "1585650267003657238", GMT: Tuesday, March 31, 2020 10:24:27.003 AM**)

#2791 has the fix for the problem and it has been merged into 1.0.

However in 1.x (as well as in the latest release 1.2.6) there is still utils.hexToNumber(block.timestamp); which fails for our setup.
https://github.com/ethereum/web3.js/blob/f9344d894866cbf64d67b49106b9b80562835af2/packages/web3-core-helpers/src/formatters.js#L295-L301

The #3070 (also referenced in #2791) has "Lower priority" issues - especially #1905 which is sadly still unchecked and still not merged in 1.2.6 (and 1.x).

My questions:

• Is there any chance to get the fix for block.timestamp in 1.2.6 in near future?
• Web3.js - 1.3.0 looks promising with lot of fixes including block.timestamp fix. Do you have any date when you will release 1.3.0?
• 2.0.0-alpha.1 has been released 8 months ago. Are there any plans for release of stable version 2.x?

(if needed I can prepare steps to reproduce but it looks like the issue is well know but with low priority)

[cgewecke]

Hi @VladoDemcak

There's recurring discussion (ex: #3224) about how to support this for 1.x ... the main problem is that it would be a breaking change to reformat the timestamp as a BN and most users would not derive any direct benefit from it.

Historically, the way platforms with special requirements like this have managed is by using Web3's extend feature. An example implementation can be seen at jpmorganchase/quorum.js here.

Do you think that would work for your case?

2.0.0-alpha.1 has been released 8 months ago. Are there any plans for release of stable version 2.x?

1.x is the project's current focus and will be receiving the lion's share of regular bug fixes and improvements for the near term.

[VladoDemcak]

@cgewecke

Thanks! We checked extend feature and looks interesting.

Is it somehow possible to override the newBlockHeaders default subscription?

[cgewecke]

@VladoDemcak Ah, it looks like jpmorganchase/quorum.js deals with that part here and here.

Out of curiosity, is their extension for Quorum/RAFT not suitable for your purposes?

[VladoDemcak]

@cgewecke
I saw the parts. And I still don't know if their extension is what I am looking for. The problem is that their extension creates namespace web3.quorum (I cannot use namespace) and other than that, there is only sendRawPrivateTransaction.

https://github.com/jpmorganchase/quorum.js/blob/d306778aa4cc66fed1c5f90845c5db14c0974822/lib/index.js#L347-L352

For example, simple thing like getBlock works great! Without extend it fails.

quorumjs.extend(web3);
 const block = await web3.eth.getBlock(1);
 console.log(`Block: ${JSON.stringify(block)}`);

But the sendSignedTransaction is what I need and what doesnt work.

I will try to explain my problem in more detail.

What we are trying to do is very simple sendSignedTransaction with WS as provider.
(The funny part is that RAFT+Quorum and sendSignedTransaction works with HTTP Provider (when we dont have listener "sendSignedTransaction(tx).on('confirmation', (confNum, rec) => {})" But we dont want to use HTTP because as documentation says "The HTTP provider is deprecated, as it won’t work for subscriptions.". )

So I have:

const web3 = new Web3("ws://localhost:23000"); 
const tx = ...
await web3.eth.sendSignedTransaction(tx);  <--- fails here

The tx is mined correctly and we can see the new block with the tx on ledger. But within "send step" the web3js is trying to do some confirmation or something (still investigating what is happening).

The startWatching function creates new subscription (not with extended RAFT/quorum subscription but with default one - I think this is our problem)
So I assume (not sure need to verify) it will create subscription newHeads

then it will go through:

and then fail on:

I understand the fix will break some functionality in web3 however we would need to find a way how to make "RAFT + quorum + WSProvider + sendSignedTransaction" work.
Thanks for your help.

[cgewecke]

The funny part is that RAFT+Quorum and sendSignedTransaction works with HTTP Provider (when we dont have listener "sendSignedTransaction(tx).on('confirmation', (confNum, rec) => {})" But we dont want to use HTTP because as documentation says "The HTTP provider is deprecated, as it won’t work for subscriptions.".

Wasn't aware of this deprecation notice in the docs, thanks for noting.

It's not correct. HTTP is the most robust/reliable connection atm and will continue to be fully supported. Confirmations polling over HTTP should work correctly as of Web3 1.2.2.

Will look further at what can be done for WS though...subscriptions seems less well supported by extend than it should be.

[VladoDemcak]

@cgewecke

It's not correct. HTTP is the most robust/reliable connection atm and will continue to be fully supported.

Ok! This might change our approach and design.

So let's assume we will use HTTP provider for sendSignedTransaction without on('confirmation'). This worked.

But unfortunately, we want to have an event listener - contract.events.<eventName>- at the same time.

I tried the event listener in test/e2e.contract.events (and also in my application) with "http" provider, but it is failing.
https://github.com/ethereum/web3.js/blob/67ea9d097737a1fb3c26cb2ae9a237245d600947/test/e2e.contract.events.js#L26

Are we able to have event listen for contract.events.<eventName> with HTTP provider? If not, what would you suggest? Is it ok to have two instances of web3 one for ws one for http? Because each provider will block something in our scenario.

[cgewecke]

@VladoDemcak With HTTP you can use getPastEvents. To reproduce the effect of subscription you'd need to call it within a block polling loop you write yourself, so it's a little more complicated.

Is it ok to have two instances of web3 one for ws one for http?

Yes, that should be fine. They're separate connections and you'd have separate contract instances for each as well.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment, otherwise this issue will be closed in 7 days