Mitigate noise from py-ipfs-api due to its mis-handling of 0-length payloads per #485

[machawk1]

The issue of using py-ipfs-api to add_bytes() of zero-length payloads is still an outstanding issue per ipfs-shipyard/py-ipfs-http-client#137. This causes a lot of noise by the ipwb indexer when encountering these records, like when indexing samples/warcs/IAH-20080430204825-00000-blackbook.warc.gz.

This PR prevents these records from being attempted to be pushed into IPFS using py-ipfs-api. When ipfs-shipyard/py-ipfs-http-client#137 is fixed, we can change the logic but for now, this will short-circuit the attempt from ipwb.

[codecov]

Codecov Report

Merging #572 into master will decrease coverage by 0.07%.
The diff coverage is 22.22%.

@@            Coverage Diff             @@
##           master     #572      +/-   ##
==========================================
- Coverage   23.84%   23.76%   -0.08%     
==========================================
  Files           6        6              
  Lines        1191     1199       +8     
  Branches      179      180       +1     
==========================================
+ Hits          284      285       +1     
- Misses        887      893       +6     
- Partials       20       21       +1

Impacted Files	Coverage Δ
ipwb/indexer.py	52.58% <22.22%> (-1.32%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3979cdc...941855b. Read the comment docs.

[ibnesayeed]

We might want to add a special hash for now (e.g., 0000000000000000000...) to indicate zero bytes. And then check this well known hash in the replay to not attempt to fetch it. Current changes will ignore such records from being replayed as they are skipped from index.

[ibnesayeed]

See the comment under the Conversation tab

[machawk1]

@ibnesayeed Fabricating data here seems like the wrong approach. It would be better to know what a blank string equates to as an IPFS hash and include it here rather than the 0s.

[ibnesayeed]

I agree, but I do not know how to add an empty string/bytesarray to IFPS.

[machawk1]

Merging this for now. We can revisit it later but keeping the status quo makes for messy testing.