Test encoding optimiations

[Gsantomaggio]

No description provided.

[DanielePalaia]

HI @qweeze, we are trying to do some general optimization work on the project and we would like also to review the branch you originally posted here: #30

The code is running and it is saving us few seconds over millions of messges but It seems like there an issue importing _FIELD of the dataclasses module. Can you help us going through this? Thanks!

[qweeze]

Hi @DanielePalaia

I re-implemented this optimization with dataclasses in a safer manner here. Didn't benchmark it though, but it shouldn't be much slower than the version using _FIELD. Hope this helps!

[Gsantomaggio]

Thanks @qweeze
I tried the branch and:

(.venv) ➜  rstream git:(feat/encoding-optimizations) python docs/examples/basic_producers/producer_send_batch_binary.py
Sent 1000000 messages in 9.1153 seconds. 109706.0540 messages per second

Master:

(.venv) ➜  rstream git:(master) python docs/examples/basic_producers/producer_send_batch_binary.py
Sent 1000000 messages in 7.7188 seconds. 129553.2735 messages per second

So the current implementation seems to be faster.

[qweeze]

@Gsantomaggio I added some more fixes, and run the same benchmark. It showed ~20% speedup (6.8 vs 5.6 seconds). Can you try it again on updated branch?

[Gsantomaggio]

closed in favour of #100