Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to Parse any WS payload using JSON !! #204

Closed
gnkarn opened this issue Jun 30, 2017 · 8 comments
Closed

Unable to Parse any WS payload using JSON !! #204

gnkarn opened this issue Jun 30, 2017 · 8 comments

Comments

@gnkarn
Copy link

gnkarn commented Jun 30, 2017

im looking for an example help that combines web sockets event with the JSON library , i wasn't able to find any working example . ( using the socket.io client )

here is the problem
somehow the payload received by the library function on message , gets more info than the one sent with the original message , this alone wounds be a problem, if there were an easy way to get rid of it , or to access , to the "true" message inside the payload.

for example :
original payload :
"{"msgName":"time","type":3,"message":"18:40:27 GMT+0000 (UTC)"}"

received payload
42["time","{"msgName":"time","type":3,"message":"18:40:27 GMT+0000 (UTC)"}"]

the consequence is that the original Json is not son anymore, so we have to gather the substring of it .
something like

String text = ((char *)&payload[0]);
int pos = text.indexOf('{'); // position of { json object start
String json1 = text.substring(pos,text.length()-2);

and finally apply
JsonObject& root = jsonBuffer.parseObject(json1);

But, parsing fails , even when the string is correctly formatted ,
in fact if i manually define the json as
const char* json1 = "{"msgName":"time","type":3,"message":"22:40:49 GMT+0000 (UTC)"}";

it is parsed ok as expected .

in summary , how to get easy access to the underlaying string that is referenced on the payload ?

@Links2004
Copy link
Owner

Links2004 commented Jul 3, 2017

the library only abstracts the websocket part of the socket.io protocol, the numbers befor the json are part of socket.io.

https://github.com/socketio/socket.io-protocol#socketio-protocol

JSON basics:
#124 (comment)

// this is a dirty hack to search the events
while(*data != 0) {
   if(*data == '2') {
       data++;
       break;
   }
   data++;
}
if(*data != 0) {
    DynamicJsonBuffer jsonBuffer;
    JsonObject& root = jsonBuffer.parseObject((char *) data);
}

numbers:

https://github.com/socketio/socket.io-protocol#packet

@gnkarn
Copy link
Author

gnkarn commented Jul 7, 2017

thank you ,
Will try to use just plain websockets ,
make a first attempt and it works , ok with short messages , but with longer payloads ( a matrix array , of 20 x 24 elements , each with 8 bit RGB values , by sending the complete frame at once ( as a text string) , it works ok in the browser with JS, but for the ESP it seams the message is too long .

So , i tested , by sending each column , one at a time , in a 10 frames per second frequency , but it seams too much for the ESP (?)

I think i should go with binary instead , of text . Will search if i found some guidance here , if not will open other "help Request" if you don't mind - For " How to work with binary Data frames"

@Links2004
Copy link
Owner

you can try this branch:
https://github.com/Links2004/arduinoWebSockets/tree/write_big_data
if it still not sends it, the message is simple to much for the ESP, to be send at once.

sure, each issue is a good help for others too, and shows my what need to be improved on the library.

@gnkarn
Copy link
Author

gnkarn commented Jul 7, 2017

before trying other branch, is there a way , to make the server, and client , not to buffer messages ?, just keep the most recent one ? ( similar than an udp type of connection , i guess) .
As in my case , ( led screen ) that is preferred in order to avoid communication buffer buildup and increase latency ..

@Links2004
Copy link
Owner

no the ws TCP stream need to be interpreted complete, you only know where the next message will be if you have read the header of the last one.
may integrated an ACK command the you only send new date when the old is handled.
or more optimized, send 2 "LED frames" and only send a new one when the first one gets an ACK, by this you keep the transfer rate at he max without waiting unnecessary for an ACK, since there will always be one frame in transfer already.

@gnkarn
Copy link
Author

gnkarn commented Jul 7, 2017

good, will try that ..

other question , i have again the same problem with parsing the message , it is too strange , and debugging is not helping
Do you have any example , where the message parsing is performed on the onmessage function ?
i couldn't found any , and also on the JSON library , it could be good to add a web socket example
as , independently , both works , but when combined is not that easy .

in my example , i get this , where json2 is indicating the message received as payload , but the parser says it can parse it successfully , and no clue why ...

[WS][0][handleWebsocket] ------- read massage frame -------
[WS][0][handleWebsocket] fin: 1 rsv1: 0 rsv2: 0 rsv3 0 opCode: 1
[WS][0][handleWebsocket] mask: 0 payloadLen: 50
[readCb] n: 50 t: 302537
[WS][0][handleWebsocket] text: {"msgName":"time","type":3,"message":"8:16:56 PM"}
json2 :
{"msgName":"time","type":3,"message":"8:16:56 PM"}
Parse success ? 0

@gnkarn
Copy link
Author

gnkarn commented Jul 11, 2017

i found the problem , the parsing needs to be prepared exactly for the payload you are going to use ,
so if there two messages , that will need a different setup, then that needs to be managed separately , this bring to the problem , that you will need to detect the message before parsing it ( catch22 situation) , so the solution i will try is to add a first char code to my payload, that will indicate the type of parsing the message needs .

and probably , later will go to a more elaborated ( or simpler) protocol for leds ( OPC , tpm2net , artnet ,...)

@suryatmodulus
Copy link

suryatmodulus commented Oct 2, 2017

I know its really late but i hope its useful..... For ESP8266 WebSocketClientIO

#include <ArduinoJson.h> --- use this library

--> payload = 42["arduino",{"message":"forward"}]

case WStype_TEXT:
{
Serial.printf("[WSc] get text: %s\n", payload);
String text = ((const char *)&payload[0]);
int pos = text.indexOf('{');
String json = text.substring(pos,text.length()-1);

        const size_t BUFFER_SIZE =
             JSON_OBJECT_SIZE(10) 
             + MAX_CONTENT_SIZE;              
         DynamicJsonBuffer jsonBuffer(BUFFER_SIZE);

         JsonObject& root = jsonBuffer.parseObject(json);
         String msg = root["message"];

     }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants