Implement cbor ipld nodes and a first pass at the 'dag' command

[whyrusleeping]

Alright, this is ready for some review now

License: MIT
Signed-off-by: Jeromy why@ipfs.io

[whyrusleeping]

@Kubuxu @lgierth @kevina @hsanjuan Could you guys give me a little review on this one?

Also worth taking a look at the cbor ipld package here: https://github.com/ipfs/go-ipld-cbor

[kevina]

@whyrusleeping I'm afraid I have not been following the IPLD discussion very closely. What exactly is a "cbor ipld node"? In particular what does "cbor" stand for or at least mean.

[kevina]

This could use some test cases. I would to implement CBOR support for the filestore before this gets merged. I will probably be able find more bugs that way. Filestore integration should be easy, but be a few days before I can get to it.

[kevina]

So "go-libp2p-routing" is no longer used?

[whyrusleeping]

Oh, i guess it still is. But its imported indirectly via the go-libp2p-kad-dht, meaning that me accidentally removing this didnt break anything.

[kevina]

@whyrusleeping okay so scratch my filestore notes, it doesn't look like there is enough implemented yet for this node type to make its way to the filestore. I discovered that CBOR = Concise Binary Object Representation (RFC 7049), but I am even more confused on how this all fits in. I ping you on IRC tomorrow. (Sorry for all the noise).

[hsanjuan]

@kevina @whyrusleeping actually I appreciate the noise because I have similar questions. Did you discuss on IRC already? It would be good if I can read it too :)

[whyrusleeping]

@hsanjuan What questions do you have? I can provide answers here as its a better place for others to find the info

[hsanjuan]

@whyrusleeping so CBOR is just another serializing format and this would be used to store IPLD nodes locally, where now Protobuf is used?

Since you asked for review, I'll tell you that you forgot to document most things in go-ipld-cbor (https://godoc.org/github.com/ipfs/go-ipld-cbor) :)

Shouldn't the JSON->CBOR functions be part of go-ipld-cbor too? Since that seems to offer CBOR->JSON already?

[Kubuxu]

@hsanjuan protobuf is only for what is used currently which is unixfs, in future we might upgrade unixfs to IPLD but so far it is not planned as far as I know.

[Kubuxu]

LGTM, the conversion functions are a bit not nice but it is to be expected.

[ghost]

LGTM, just one question: why two formats for each of cbor or and protobuf, and the possible inconsistency between "dag-pb" and "protobuf"?

[whyrusleeping]

@lgierth both dag-pb and protobuf are the same thing. Just different names for it. I've traditionally called it just protobuf, but @diasdavid insists on calling it dag-pb.

If you want we can just pick one, but i'm a fan of 'being liberal in what you accept'

[ghost]

If you want we can just pick one, but i'm a fan of 'being liberal in what you accept'

cool I was just wondering 👍

[whyrusleeping]

Well, if i merge this now, i will have successfully met the goal we set for ipld integration. I've got a few LGTMs and this command is marked as experimental. Lets do this.