WIP: Dat daemon on websockets

[soyuka]

Related issue: #7

So far:

• add an http layer (experimental)
• move protocol to it's own package (not published yet)
• add a dat-daemon-ws module to allow the use of Websocket (see Provide HTTP (or WS) API to support browser extensions #7)

To do:

• build a client that works with the Websocket daemon (see ws/cli.js for instructions)

[RangerMauve]

Are there docs for how to use this?

[soyuka]

https://github.com/soyuka/dat-daemon#client there you go :) Lmk if you want to start an extension because I may give it a try if not!

[RangerMauve]

Awesome!

I'll only have time to do anything substantial this weekend.

I think a good goal for an MVP is the following:

• Have the daemon support a built in HTTP gateway or something for reading from a dat
• Look at how ipfs-companion does redirection
• Redirect dat:// urls to the gateway

[soyuka]

Look at how ipfs-companion does redirection

There you go :D (I already digged a bit into their code).

Redirect dat:// urls to the gateway
Have the daemon support a built in HTTP gateway or something for reading from a dat

Yes this is the hard part. For now what I suggest is to implement a simple readdir/readfile api in the daemon:

{action: READDIR, key: 123, path: 'X' }
{action: READFILE, key: 123, path: 'X' }

I'm not sure how I'd set up the Answer part though so that it'd be easy to document...

[RangerMauve]

Yeah, looks like this is where the request should be made down the socket.

One thing to keep in mind is different file types. At the moment answer is a string message, maybe it should be base64 encoded data for files which will be decoded before being sent in the reply?
Also, it would probably be important to support streaming data in the future.

[RangerMauve]

Also, does your RPC protocol support multiple messages being processed at once?

[soyuka]

At the moment answer is a string message, maybe it should be base64 encoded data for files which will be decoded before being sent in the reply?

We don't need base64 imo blobs/buffers can work.

Also, does your RPC protocol support multiple messages being processed at once?

I'm not sure I got that, may you elaborate?

[soyuka]

I'm currently thinking of a proper implementation to introduce easy file write/read things. Other actions are fairly easy to implement as Q&A.

I've come up with some channel api:

Client opens a socket on ws://localhost:1234/read/38bd32e351630dcb179ee52752d0312ccc6fe95e3ec35c1c9ba7d4d7f15276fe/path
and receives a stream of data (using websocket-stream, works great).

if the url is of form ws://x:1234/write/{key}/{path} the server writes the content.

We could also imagine a live statistics endpoint: ws://x:1234/stats/{key} (network or files events).

proof of concept code:

  const c2 = Websocket(`ws://localhost:${configuration.port}/38bd32e351630dcb179ee52752d0312ccc6fe95e3ec35c1c9ba7d4d7f15276fe`)
  const w = new Writable({
    write (message, enc, cb) {
      console.log(message)
      cb()
    }
  })

  w.on('finish', function() {
    console.log('end')
    c2.socket.close()
  })

  c2.pipe(w)

[RangerMauve]

@soyuka Typically when I've seen people use websockets, they maintain one persistent connection and send multiple messages down the wire. That's why I said " support multiple messages being processed", a person might have one connection and will send several commands down the wire, and will need to tie the answers back to the commands that got set.

JSON-RPC handles that by attaching a message ID to each request/response.

I don't think I've seen people storing actual data in websocket URLs, either, but it should work just fine.