WebSockets #1305

jwoertink · 2020-11-11T22:19:20Z

Purpose

Fixes #554

Description

I wanted to get the discussion moving on how WebSockets might look in Lucky. The idea here would be that Lucky provides a very basic interface to using WebSockets from the backend, but it would be up to the developer to implement the client side portion on their own with whatever tools they know.

This implementation tries to stick as close to current actions as we can while maintaining all of the type-safety features that Lucky gives us. It shouldn't feel like a ton of setup to get using.

Imagine you just generated a fresh Lucky app with this in it (maybe the wizard asks if you need websockets?), here is code that would be in your generated app:

# src/app_server.cr
def middleware : Array(HTTP::Handler)
    [
      Lucky::ForceSSLHandler.new,
      Lucky::HttpMethodOverrideHandler.new,
      Lucky::LogHandler.new,
      Lucky::ErrorHandler.new(action: Errors::Show),
      Lucky::RemoteIpHandler.new,
      Lucky::WebSocketRouteHandler.new, # <-- this line added
      Lucky::RouteHandler.new,
      Lucky::StaticCompressionHandler.new("./public", file_ext: "gz", content_encoding: "gzip"),
      Lucky::StaticFileHandler.new("./public", fallthrough: false, directory_listing: false),
      Lucky::RouteNotFoundHandler.new,
    ] of HTTP::Handler
  end

Then you add your action:

# src/actions/chats/create.cr
class Chats::Create < Lucky::WebSocketAction
  SOCKETS = [] of HTTP::WebSocket

  ws "/chats" do
    SOCKETS << socket
    socket.on_message do |message|
      SOCKETS.each do |s|
        s.send(message)
      end
    end

    socket.on_close do
      cleanup
    end
  end

  private def cleanup
    # cleanup socket
  end
end

And your JS:

document.addEventListener("DOMContentLoaded", ()=> {
  const room = document.getElementById("room");

  if (room) {
    const socket = new WebSocket("ws://localhost:5000/chats");
    socket.onopen = () => { socket.send("PING"); };
    const form = document.getElementById("form");

    form.addEventListener("submit", (e)=> {
      e.preventDefault();

      const input = document.getElementById("msg");
      socket.send(input.value);
      input.value = ""
    });

    socket.onmessage = (message) => {
      const chat = document.createElement("p")
      chat.innerText = message.data
      room.appendChild(chat)
    };
  }
});

Note

This setup "works", but it's still early planning stages. It'll probably change drastically before being ready for a real review.

Checklist

- An issue already exists detailing the issue/or feature request that this PR fixes
- All specs are formatted with crystal tool format spec src
- Inline documentation has been added and/or updated
- Lucky builds on docker with ./script/setup
- All builds and specs pass on docker with ./script/test

jwoertink · 2020-11-11T22:20:42Z

src/lucky/routable.cr

+  macro setup_ws_call_method(&block)
+
+    abstract def on_message(message)
+    abstract def on_close


My thought was by having these abstract, the websocket can proxy to them, and forces you to set these up. Websockets have a few other methods though.... are any of them as necessary? Is there ever a time you'd use a websocket and not use these?

src/lucky/routable.cr

src/lucky/web_socket.cr

jwoertink · 2020-11-11T22:24:07Z

src/lucky/web_socket_action.cr

+  def initialize(@context : HTTP::Server::Context, @route_params : Hash(String, String))
+    @websocket = Lucky::WebSocket.new(self.class) do |ws|
+      ws.on_message { |message| on_message(message) }
+      ws.on_close { on_close }


If we don't make on_message and on_close abstract, then we can't guarantee they will exist which means we can't do this proxy. You'd just have to use them on your own. Not bad, but is this the better path?

src/lucky/web_socket_action.cr

src/lucky/web_socket_handler.cr

fernandes · 2020-11-11T22:47:37Z

accepted_formats [:plain_text, :json], default: :plain_text doesn't make sense for a WS action

why does Chats::Index inherits from WebSocketAction, the behavior is totally different

src/lucky/routable.cr

src/lucky/web_socket_handler.cr

jwoertink · 2020-11-12T15:52:02Z

@fernandes With websockets, would you ever need to send an Accept header to say the messages being passed back and forth are of any specific type? Or does that not matter? I'm wondering if there should be a default accepted_formats set? 🤔

Also, I inherited the action from a base class just because that's what you'd normally do in any other action. My hope here is that this can feel like the rest of the actions (i.e. BrowserAction, ApiAction). Then you could have all of your websocket actions inherit from the same parent class in case they all need to do something like ensure you're logged in before connecting or whatever.

confact · 2020-11-20T14:34:39Z

This comes in nice timing. I was actually planning to use @fernandes cable repo for this.

What I want in WebSocket support is:

on message handling
on connected handling
on disconnect/close handling
subscribe/channel handling

fernandes · 2020-11-21T23:15:05Z

@confact ohhh that's nice to hear! I'm working with @jwoertink in some stability/scalability issues, so I'd advice to not use in production atm, hope to fix this soon 🤞

anything you can ping me on lucky discord server 😉

jwoertink · 2021-05-24T16:00:55Z

I built a mini chat app using this setup over the weekend. Worked pretty well, but now I have a better idea of how I want to do this. It's going to require a TON more work than I originally wanted, but will be much better in the long run.

I'll try to get a PoC out to flesh out the details, but in the end, I think this may have to be a post 1.0 feature. For now, anyone coming to this looking for WebSocket support, use https://github.com/cable-cr/cable

grepsedawk · 2022-02-09T18:57:59Z

As part of this PR, I think you'll also need to add wss to the possibilities for https://github.com/luckyframework/lucky/pull/1662/files#diff-88f9201f12cf1493e98c2358834dbae535ac1be4d45fe1f01429dd526b280ecaR51

jwoertink · 2024-03-31T17:35:48Z

I have a better idea of how I want to do this
I'll try to get a PoC out to flesh out the details

I wish I would have done this 😭 I was going to come back to this, and I don't have any of that code I worked on now.... I guess I'll have to just re-imagine the whole thing 😮‍💨

jwoertink · 2024-03-31T17:39:59Z

src/lucky/routable.cr

@@ -58,6 +58,17 @@ module Lucky::Routable
    end
  {% end %}

+  # Define a route that responds to a WebSocket request
+  macro ws(path, &block)


Would need wss too

jwoertink · 2024-03-31T17:43:39Z

src/lucky/web_socket_action.cr

+    @handler = HTTP::WebSocketHandler.new do |ws|
+      @socket = ws
+      ws.on_ping { ws.pong("PONG") }
+      call
+    end


This would create a new handler for every websocket action you added. I'm not sure how many a person would add, but it seems like you should only have 1, and the handler would just route to each action based on where it connected. This looks like what Kemal seems to be doing too. A single instance that just adds each route and the handler.

rmarronnier · 2024-09-12T16:27:17Z

Thanks for having started work on this.

Right now, as you or others, I'm using https://github.com/cable-cr/cable in my lucky project.

It's not really hard to setup and the semantics (channel, connection) are familiar for those with a rails background.

The semantics are also really efficient (a single connection by user and several channels, each with its own callbacks).

Using the semantics of a lucky action (a route and a http verb to create or close a connection/channel) will lead to a lot of boilerplate (create.cr and close.cr files).

I don't think the DX of Lucky::WS::Action is going to beat Cable::Channel with all callbacks and logic in a single file.

But ! What I don't like about cable / action cable is that it relies on a serialized json communication to start / close channels.

The format of those json messages is specific to action cable and requires json serialization capability on the client side.

We could replace this complexity / overhead with a route based logic.

example (shamelessly stolen from cable) :

class ChatChannel < Lucky::WS::Channel
  path: "/ws/chatrooms/:id"

  def subscribed
    stream_from "chat_#{id}"
  end

  def receive(data : String)
    broadcast_message = {} of String => String
    broadcast_message["message"] = data
    broadcast_message["current_user_id"] = connection.current_user.id
    ChatChannel.broadcast_to("chat_#{id}", broadcast_message)
  end

  def unsubscribed
    ChatChannel.broadcast_to("chat_#{id}", "#{connection.current_user.name} left the chat")
  end

  # These lucky actions / routes are automatically generated
  # post "/ws/chatrooms/:id/subscribe"
  # delete "/ws/chatrooms/:id/unsubscribe"
 end

It's extremely similar to the Cable::Channel API (maybe Lucky::WS::Channel < Cable::Channel ?)

TLDR; We could just extend cable.cr to give it lucky specific semantics like route actions for example.

Or what didn't you like in cable.cr you wish you had in lucky ?

jwoertink · 2024-09-12T17:00:03Z

@rmarronnier I definitely need to come back to this at some point. With my direction, you wouldn't do separate actions for create and close. The create name was just poor naming, but all of your connection would happen in that single action. This is the same on how it works with Kemal.

So a better example would be more like

# src/actions/chat_messages/channel.cr
class ChatMessages::Channel < Lucky::WebSocketAction
  ws "/chat_messages/:id" do |socket|
    # handle things how you need
  end
end

The idea is more that it's lower level letting the developer decide if there should be a concept of channels, or how the subscription and message receiving should work. Then on the front-end it would be all up to you to handle the connections how you want.

With that said, I totally agree about Cable. I love how the backend is setup, but I'm not a fan of how the messages are handled with the data being a stringified json... but I get why it's like that because if you wanted to map that data, you have no way to know the structure of the hash, but what you do know is that it's always String => String, it just happens to be that the value could be parsed in to JSON 😬

I'm open to other ideas though. Since this PR has fallen behind so much, I'm down to rethink it from the ground up. I still want to get it in, but I don't want to just bring in Cable and force everyone to have to use Cable on the client side.

rmarronnier · 2024-09-15T12:31:13Z

Thanks @jwoertink for the feedback.
A more low-level / barebone solution is actually quite nice.
It'd make it easier to work with specific client scenarios (htmx websockets come to mind)
And, yeah, you're right having a dependency on cable.cr is overkill (might as well use cable.cr directly)

jwoertink added 2 commits November 11, 2020 14:07

initial files for websocket compatibility.

58c9559

make sure this is formatted

3244366