Apps ShelfPlayer and Plappa timeout on Android and iPhone

[feuerkind99]

I have the Docker on my Synology running version 2.17.5. I get library not available in both apps several times and if I try to reload it sometimes works and displays the books. I have currently 3 libraries with 3132 books, 1778 books and 10 books ... there have been many more but i reduced the amount of books to see if it gets better ... after adding the max_old_sace_size=10240 it got a bit better but still not fine ... got any ideas for me what to try? are this to many books? Most files are mp3 ... any advice is welcome. btw webinterface is always working fine ... takes some time to load but always works (on PC, iPhone, iPad and Android) so this seams to be only related to the apps

[feuerkind99]

Also interesting and maybe the problem:

2024-12-11 10:00:27.082 DEBUG Loaded 1 of 1 items for "Continue Listening/Reading" in 464.20s
2024-12-11 10:00:27.124 DEBUG Loaded 1 of 1 episodes for "Newest Episodes" in 337.00s
2024-12-11 10:00:27.132 DEBUG Loaded 0 of 0 items for "Continue Series" in 336.92s
2024-12-11 10:00:28.871 DEBUG Loaded 1 of 1 items for "Recently Added" in 282.04s
2024-12-11 10:01:55.678 DEBUG Loaded 3132 of 3132 items for libary page in 532.51s
2024-12-11 10:01:59.394 DEBUG [ApiCacheManager] Cache miss: {"user":"feuerkind99","url":"/libraries/62f007a6-740f-4b74-baf8-eed81ffb5f12/items"}
2024-12-11 10:01:59.496 DEBUG Loaded 1 of 1 items for "Recently Added" in 406.34s
2024-12-11 10:01:59.539 DEBUG [ApiCacheManager] count: 13 size: 9618747
2024-12-11 10:02:03.329 DEBUG Loaded 0 of 0 items for "Continue Series" in 354.84s
2024-12-11 10:02:03.350 DEBUG Loaded 1 of 1 items for "Recently Added" in 354.84s

I guess it is not supposed to have such a long loading time?

[nichwall]

How many files are in the library? Libraries with a high file count (multiple files per book) can get a lot slower.

We have been working on cleaning up the database and API to make things a lot faster and there are still things to do. I haven't seen load times take that long after recent updates. Did loading time get worse on a recent update, or has it always been about this slow?

Would you be willing to share your database to help us see what exactly is showing things down? You can zip the database (while the server is not running) to make it a lot smaller) and send the file or a download link via email on my GitHub profile or over Discord.

[feuerkind99]

in all three libraries I have about 5000 books ... that makes 1.8 TB in 310868 Files in 5759 Folders (Windows Explorer File Count :-) )

I would not say the problem started with a recent update but got worse over time ... I removed 244 GB files (30484 Files in 1172 Folders) but that didn't bring any significant change.

I also tried splitting the books to more different libraries but this did not help eighter

should I find a way to convert the mp3 to m4b? to get just 1 file per book? would this help?

I could provide a ABS-backup file with about 400 MB ... would this also help? Due to not having any idea how to get the database out from docker :-(

[nichwall]

Okay, thanks for confirming it wasn't a recent update that made load times much worse.

You can convert books to m4b using the built-in converter, but it does not let you bulk convert your books and is more focused on simple conversions (doesn't give you a lot of options). If you want to go that route, I would use Auto-M4b or something similar. I just have a custom ffmpeg script I run. Other users on Discord have reported converting to a single file per book improved performance a lot, but I don't have any numbers for how much it helps.

Yes, the backup from ABS works too.

[feuerkind99]

just sent the link to the backup file to your g*.com address ... please advice as soon as i can delete the file from my webserver :-)

I read something about the converter in ABS ... simply I did not find it up to now? guess I am blind :-(

[nichwall]

Thanks, I got the email, but I am currently at work and will download it tonight when I get home.

[feuerkind99]

no problem ... just a short message to know when I can delete the file :-)

[nichwall]

I loaded your database up. I am getting a load time of 1.05 seconds for the home page, and 0.35 seconds for loading the library page. The ABS web client and app do paginated loading, so the log shows only 30 items being loaded for the library page. In your original log, it looks like Plappa is trying to load everything for the library, which is probably what is slowing everything down. From the write-up here, multiple concurrent queries to the database slows things down a lot (so Plappa loading everything for the library may be what is slowing down all of the other queries if it is happening in parallel). It may be worth opening an issue with Plappa to implement paginated loading.

[2024-12-13 20:11:54.705] INFO: [Watcher] "Erwachsene" Ready
[2024-12-13 20:11:54.705] INFO: [Watcher] "Jugend" Ready
[2024-12-13 20:11:54.707] INFO: [Watcher] "Englisch" Ready
[2024-12-13 20:12:08.971] INFO: [Auth] User "root" logged in from ip ::ffff:127.0.0.1
[2024-12-13 20:12:09.088] DEBUG: [ApiCacheManager] count: 0 size: 0
[2024-12-13 20:12:10.350] DEBUG: Loaded filterdata in 1.25s
[2024-12-13 20:12:10.354] DEBUG: [ApiCacheManager] Cache miss: {"user":"root","url":"/libraries/62f007a6-740f-4b74-baf8-eed81ffb5f12?include=filterdata"}
[2024-12-13 20:12:10.534] INFO: [SocketAuthority] Socket Connected to /socket.io sweNiyc3tRNGt1BvAAAB
[2024-12-13 20:12:10.545] DEBUG: [ApiCacheManager] count: 1 size: 270717
[2024-12-13 20:12:10.549] DEBUG: [ApiCacheManager] count: 1 size: 270717
[2024-12-13 20:12:10.562] DEBUG: [ApiCacheManager] Cache miss: {"user":"root","url":"/libraries"}
[2024-12-13 20:12:10.588] DEBUG: Loaded 0 of 0 items for "Continue Listening/Reading" in 0.04s
[2024-12-13 20:12:10.599] DEBUG: [SocketAuthority] User Online root
[2024-12-13 20:12:10.706] DEBUG: Loaded 0 of 0 items for "Continue Series" in 0.12s
[2024-12-13 20:12:11.182] DEBUG: Loaded 10 of 107 items for "Recently Added" in 0.48s
[2024-12-13 20:12:11.244] DEBUG: Loaded 5 of 23 series for "Recent Series" in 0.06s
[2024-12-13 20:12:11.568] DEBUG: Loaded 10 of 831 items for "Discover" in 0.32s
[2024-12-13 20:12:11.590] DEBUG: Loaded 0 of 0 items for "Listen/Read Again" in 0.02s
[2024-12-13 20:12:11.603] DEBUG: Loaded 10 of 92 authors for "Newest Authors" in 0.01s
[2024-12-13 20:12:11.603] DEBUG: Loaded 4 personalized shelves in 1.05s
[2024-12-13 20:12:11.603] DEBUG: [ApiCacheManager] Cache miss: {"user":"root","url":"/libraries/62f007a6-740f-4b74-baf8-eed81ffb5f12/personalized?include=rssfeed,numEpisodesIncomplete,share"}
[2024-12-13 20:12:11.604] DEBUG: [ApiCacheManager] Caching with 1800000 ms TTL
[2024-12-13 20:12:25.433] DEBUG: [ApiCacheManager] count: 3 size: 346221
[2024-12-13 20:12:25.434] DEBUG: [ApiCacheManager] Cache hit: {"user":"root","url":"/libraries/62f007a6-740f-4b74-baf8-eed81ffb5f12?include=filterdata"}
[2024-12-13 20:12:25.564] DEBUG: [ApiCacheManager] count: 3 size: 346221
[2024-12-13 20:12:25.917] DEBUG: Loaded 30 of 3132 items for libary page in 0.35s
[2024-12-13 20:12:25.934] DEBUG: [ApiCacheManager] Cache miss: {"user":"root","url":"/libraries/62f007a6-740f-4b74-baf8-eed81ffb5f12/items?sort=media.metadata.title&desc=0&limit=30&page=0&minified=1&include=rssfeed,numEpisodesIncomplete,share"}