fix: dag|block put: do not allow blocks over BS limit

[lidel]

Update 2/23 (@schomatis): working on this.

---

Summary

TLDR: bigger blocks can't be exchanged over libp2p, so we want to make a loud error if that happens, but allow people to override it if they are ok with that.

We've introduced soft-limit in ipfs object API (#8414) but are missing the same behavior in modern ipfs dag|block put 🙈

Problem (go-ipfs 0.11)

With both dag|block put, I am able to create a block (eg. 10MB) that is too big to be exchanged over bitswap:

dag put

$ dd if=/dev/zero bs=10MB count=1 | ipfs dag put --input-codec=raw --store-codec=raw
1+0 records in
1+0 records out
10000000 bytes (10 MB, 9.5 MiB) copied, 0.0373717 s, 268 MB/s
bafkreihv4avkohth6qoxsar2ckgkgw5nq3hxwzswsz574ceewor4ims6v4

$ ipfs dag stat bafkreihv4avkohth6qoxsar2ckgkgw5nq3hxwzswsz574ceewor4ims6v4
Size: 10000000, NumBlocks: 1

block put

$ dd if=/dev/zero bs=10MB count=1 | ipfs block put
1+0 records in
1+0 records out
10000000 bytes (10 MB, 9.5 MiB) copied, 0.0331271 s, 302 MB/s
QmetSrEEUmD5z6wS5Eamn5KrizYsdkthqNioEoMepYFQte

$ ipfs block stat QmetSrEEUmD5z6wS5Eamn5KrizYsdkthqNioEoMepYFQte
Key: QmetSrEEUmD5z6wS5Eamn5KrizYsdkthqNioEoMepYFQte
Size: 10000000

Solution

• reuse softBlockLimit, allowBigBlock from fix(cli): object add-link: do not allow blocks over BS limit #8414 (extract them to be public consts?)
• add --allow-big-block behavior to ipfs dag put
• add --allow-big-block behavior to ipfs block put
• make dag and put return informative error, eg:

  produced block is over 1MB: big blocks can't be exchanged with other peers. consider using dag-pb for automatic chunking of bigger files, or pass --allow-big-block to override.

[schomatis]

Working on this.

[schomatis]

make dag and put return informative error, eg:
@lidel Will be pushing a PR with the first 3 items, leaving the most appropriate error wording up to you.

[BigLep]

Internal note: we want this for 0.13

[BigLep]

2022-03-11: we also want dag-import.

[finkbeca]

Let me see if I have the time to do it this weekend, will follow-up then about if I'll be able to get it done, or if I should pass the torch to someone else to finish :)