Event enabling wstrade API

[TamimiGitHub]

I'm starting this discussion to gauge interest in real-time development with the westrade-api (plus github discussions are pretty cool!).

Some people were interested in websockets implementation (#21) to enable processing a stream of data. So a cool use-case is to have a stream stock market data and depending on stock prices execute a WealthSimple trade/action using the westrade-api.

This sounds interesting, however, imposes a couple of challenges including:

1. Scale: a proof of concept could be easily implemented with one application that has a websocket connection to a stock market and using westrade-API the application can execute trade actions. This however imposes tight coupling between the operational data layer (source stock market data) and the analytical data layer (westrade-API)

2. Flow control: A stream of data reflecting stock price updates could impose a flow control issue. This refers to having a data source (Stock market) publishing data over a websocket connection at a faster rate than the processing application (which utilizes westrade-API) resulting in a bottleneck scenario

From an architectural perspective, one way to approach event-driven architectures (where an event is a stock price update that drives a side effect like authorizing a trade using westrade-API or feeding into a ML model) is through using a message broker as seen below

This shows a separate client application dedicated to retrieve a stream of data from a stock exchange market over a websocket connection and publishes the data to the message broker over a predefined topic. Applications connected to the broker could subscribe to the topic of choice and consume the data. Having the message broker as the middleware layer acts as a shock absorber with queuing functionalities and effectively routes the events to the respective applications. Taking this architectural approach avoids tight coupling between the source of the data (publisher) and the consuming applications (subscribers).

Anyways, this is just a thought people that could spark an interesting conversation. What use-cases do you all have with westrade-API?

[ahmedsakr]

Neat walkthrough, Tamimi! I think you hit the nail with the statement that wstrade-api will be the bottleneck in a websocket-streamed quote source. wstrade-api is completely REST API-call based and that's considerably slower than a stream that pumps live data every x ms. (On a side note, this has me thinking about an internal caching mechanism in wstrade-api to reduce the number of unnecessary API calls it has to do implicitly).

quotes.get (the API that orders.marketBuy and orders.marketSell use to get market quote) can be configured to receive data from a websocket with the quotes.use API. quotes.use does not take a provider that is websocket enabled. Frankly, if wstrade-api were to take a websocket-enabled API, I am not sure how to implement that for a general web-socket without having source-specific implementation (I'm all ears if you know how!!).

The workaround for getting data into quotes.use should be trivial.

/** websocket stuff **/
const socket = new WebSocket('...');
socket.addEventListener('message', function (event) {
    // parse and store the stock quote values
});

quotes.use('NYSE', {
  async get(ticker): {
    // fetch the quote stored from the socket above
  }
});

// now everytime you do a marketBuy or marketSell for a NYSE-listed stock, the custom provider we defined above
// will be invoked by wstrade-api

if you wish to do limit buys and sells, that makes the implementation even simpler as you don't even have to use the quotes module.

/** websocket stuff **/
const socket = new WebSocket('...');
socket.addEventListener('message', function (event) {
  // parse stock quote values
  
  // You can directly invoke the limitBuy and and limitSell APIs with the price since you have it
  if (aaplPrice <= 120) {
    orders.limitBuy(....);
  }
});

[TamimiGitHub]

That's a pretty neat abstraction you have there! Nicely done 👏 A potential scenario where a quote.use would be used is if we want to have a functionality other than orders (perhaps analytics?).

The challenge with implementing a "native" websocket enabled API within westrade-api is interoperability.

I am not sure how to implement that for a general web-socket without having source-specific implementation

Good point and I'm not sure either on the top of my head. There are indeed a couple of ideas that could be used! Which got me thinking as well on how to abstract the websocket layer (similar to your quote.use abstraction). I'd like to dabble more with the websocket example you provided, and potentially find an open data stream to a stock market perhaps...

To avoid the above mentioned bottleneck issue, I would suggest taking an architectural view and look into using a message broker (like Solace for example 😉 ) and have it handle the high volume of streaming. Then, in my opinion, you would have two options:

1. Implement a subscriber application that connects to the broker and handles the stream of stock updates. Then separately uses wstrade-api to make the orders actions OR
2. Implement the subscriber to the broker within the wstrade-api (in quote.use maybe?) and abstract the broker connection from the application

The advantages of the first approach is loose coupling. Advocating for a microservices and EDA architecture, this would be a preferred choice. However you would loose on the abstraction piece that is offered in the second approached...

This is interesting and its good discussions 😄

[YassineElbouchaibi]

Hey guys, I have actually been playing around with websockets lately and wanted to ask some questions. Do you just want access to realtime quotes via websockets or do you want access to WSimple's Websocket service ? If all you want is realtime quotes via websockets, I have actually been working on a package for that myself, maybe that could refactored merged into this api. The real time data actually comes from the globe and mail's api which I reverse-engineered. They provide real-time data for US and CA markets. I reverse-engineered their api because at the time when I started using WSimple, they were the only ones providing realtime quotes for TSX. Now there is Yahoo Finance as well I believe.. Anyway I leave my repo here if you want to have a look and believe it has potential to be augmented or refactored and merged into here. https://github.com/YassineElbouchaibi/tgam-finance

[TamimiGitHub]

Hey Yassine! As far as I know, WSimple's WS is 15 minutes delayed which is not really real-time. And you can guess how this is could be an issue with stock market data 😅 I think what could be a best fit for this is just s realtime quotes websocket which is exactly what you have. What I would be interested in trying out, is publishing this real-time data into a broker (as per the basic diagram I put together earlier) and then have another app consuming this data that uses the wstrade-api

[TamimiGitHub]

Also, I ran your tool locally; very neat! Thinking out loud, a potential approach to this could be publishing to a broker inside the addSymbolDataListener

financeApi.addSymbolDataListener((a) => {
  console.log("SymbolDataListener: ", a)
  ** publish_to_broker(a, topic) **
})

[ahmedsakr]

Nice package, Yassine. You can already integrate your package with wstrade-api through the quotes.use interface. I added that interface specifically for anyone using wstrade-api to have the autonomy of configuring 3rd-party sources of quotes.

So, it wouldn't be necessary to merge it into wstrade-api. It would be outside the scope of the wstrade-api project to integrate a specific reverse-engineered source for quotes when many people might not need it, or prefer to use another source.