Uploaded images are not shared/stored

[juliushaertl]

We need to figure out:

• how images are referenced or stored in the excalidraw format
• how we can persist them

[juliushaertl]

Files are identified by some hash value in excalidraw as a reference and need to be stored/managed separately. When inserting an image we can observe the file data being available though the onChange callback

https://github.com/excalidraw/excalidraw/blob/master/excalidraw-app/App.tsx#L462
https://github.com/excalidraw/excalidraw/blob/576bc0dbe52c705b5251a3599962d5d53749994d/excalidraw-app/collab/Portal.tsx#L179

We need to think about:

• Where do we store the images, can we use the node backend to also keep images and
  • let it write them to the .excalidraw file
  • or should we store them separately like text in a .attachments folder?
• Writing into the .excalidraw file would be nice (also supported by the schema) as it then would not have dependencies of external structures when copying or moving the file
• We need to be careful on how we handle those so that it is not included in every sync over the websocket, maybe also not being hold in memory on the node backend
• Maybe we can still distribute the files over the websocket

@hweihwang @grnd-alt Would be curious about your input here. Any thoughts around that?

[grnd-alt]

I would certainly try implementing it with writing to the .excalidraw file. There's a max size to websocket messages but that can be increased (though there will always be a limit). I think websocket distribution of images should be possible, we might have to improve our update mechanism to send fewer elements in updates which are not needed, but we should have a look at that in the future anyway.

[hweihwang]

Yes agreed, general issue as the board data grows, several concerns arise:

• Websocket bandwidth
• Frontend performance reapplying the big chunk of data (Not sure if currently excalidraw api caring about this well)
• Node/Redis memory
• Networking when sending big JSON between Node & PHP
• I/O when reading/writing those JSON to file

[hweihwang]

Some efforts refs:

• Complex Drawing Poor Performance excalidraw/excalidraw#8136
• Excalidraw getting slow in Obsidian excalidraw/excalidraw#6474

[juliushaertl]

My suggestion would be to split the actual image data transfer from the regular sync messages that distribute the elements on the board.

The node backend could extract the file data from the .excalidraw file and the frontend would request it separately once coming across an image node with the reference hash. Then we could send the individual images back in one websocket message per image.

Otherwise we could also consider a plain http endpoint for fetching the images, but that would require some extra handling for authentication compared to the websocket

[juliushaertl]

I'll have a look into this