AsyncSeek trait bound on Reader may limit options to stream bytes

[alice-i-cecile]

I'm concerned that this will prevent us from writing AssetReaders (and therefore loaders) that truly "stream" the bytes. For example, the http reader used by WASM currently collects bytes into a VecReader, which is why this works. But if we wanted to skip the Vec to avoid the up front allocation of the entire asset bytes, we now literally cannot do that because AssetReader requires seek, which an http request does not support.

Originally posted by @cart in #12547 (comment)

[alice-i-cecile]

We need an async / streaming abstraction for assets, but Reader may not be the right design. I'm currently inclined to ship with this for now, and then build out the right one once the rough edges become clearer.

[mockersf]

this is true for any asset reader that could work on a stream, it could make sense for large files on native too

[BeastLe9enD]

I'm just sharing my personal thoughts on it, maybe it will be interesting for others and exciting ideas will arise from it.

I see both sides, on the one hand it can of course limit options to stream bytes for example with an http request that does not support seek, on the other side I think it is really handy in some cases to have the AsyncSeek implementation, like seeking inside a file when only a specific region is needed.

I don't think AsyncSeek will be used often when loading assets in source format, but rather in processed assets. I think that in the future we also want to make it possible for different source assets to be processed with different asset processors, depending on which platform you are on. For example, on platforms that don't support seeking, we could split a large file into several files instead of seeking inside a big one.

Of course, we would still have the problem that all custom readers still have to implement the AsyncSeek trait. For me it feels wrong to have to implement dummy functions for all readers that don't support seeking. I think it would be good if the reader had a function from which you could possibly query optional seek functionality.

Maybe it's the wrong approach to have the seek functionality in the reader, but in my personal opinion it feels pretty intuitive :)

[ldubos]

Hi, I just wanted to share my thoughts on this subject. The new AsyncSeek trait bound on Reader has made my work more complicated for https://crates.io/crates/bevy-histrion-packer.

Since I "pack" assets together and support compression, it's not easy to allow seeking on compressed assets.

Currently, my workaround is to read the file into a temporary buffer to enable seeking. I chose this approach because I believe some "native" Bevy assets will likely take advantage of the seek feature, particularly meshlet meshes.

However, this hack (mainly due to time constraints on this part, which isn't my main priority at the moment) completely negates the benefits of asynchronous reading, as I first need to move all the data to a buffer.

This could be soblved with a custom trait like AsyncSeekForward instead of AsyncSeek. This more constrained trait would allow moving the cursor forward but not backward. It would enable "stream-based" readers and readers like mine to handle it properly without having to involves hack or simply not implements it and returning an error or worst, silently failing. The most probable use case for seeking in this context is moving forward directly to the sub-resource we need to load after reading some sort of entry table.