perf(dashboard): Optimize query to fetch recent pages

[juliusknorr]

The query turns out to be quite slow as the order by statement will cause the wrong index to be used which leads to long query times on larger filecache tables.

I only consider this a workaround, as changing the query like this will not catch pages that are only in the filecache and do not have an entry in the collectives table, but this is an acceptable tradeoff I think.

---

ANALYZE SELECT `p`.*, `f`.`mtime` as `timestamp`, `f`.`name` as `filename`, `f`.`path` as `path` FROM `oc_filecache` `f` LEFT JOIN `oc_collectives_pages` `p` ON `f`.`fileid` = `p`.`file_id` WHERE (`f`.`storage` = '2') AND (`f`.`path` LIKE 'appdata_ocmk9xspynvl/collectives/12/%') AND (`f`.`mimetype` = 1
4) ORDER BY timestamp limit 7;
+------+-------------+-------+--------+--------------------------------------------------------------------------------------------------------------------+------------------------------+---------+-------------+---------+------------+----------+------------+-------------+
| id   | select_type | table | type   | possible_keys                                                                                                      | key                          | key_len | ref         | rows    | r_rows     | filtered | r_filtered | Extra       |
+------+-------------+-------+--------+--------------------------------------------------------------------------------------------------------------------+------------------------------+---------+-------------+---------+------------+----------+------------+-------------+
|    1 | SIMPLE      | f     | index  | fs_storage_path_hash,fs_storage_mimetype,fs_storage_mimepart,fs_storage_size,store_mimepart,fs_storage_path_prefix | fs_mtime                     | 8       | const,const | 4119229 | 1158541.00 |     1.07 |       0.00 | Using where |
|    1 | SIMPLE      | p     | eq_ref | collectives_pages_file_index                                                                                       | collectives_pages_file_index | 8       | oc.f.fileid | 1       | 1.00       |   100.00 |     100.00 |             |
+------+-------------+-------+--------+--------------------------------------------------------------------------------------------------------------------+------------------------------+---------+-------------+---------+------------+----------+------------+-------------+
2 rows in set (2.811 sec)

MariaDB [oc]> ANALYZE SELECT `p`.*, `f`.`mtime` as `timestamp`, `f`.`name` as `filename`, `f`.`path` as `path` FROM `oc_collectives_pages` `p` LEFT JOIN `oc_filecache` `f` ON `f`.`fileid` = `p`.`file_id` WHERE (`f`.`storage` = '2') AND (`f`.`path` LIKE 'appdata_ocmk9xspynvl/collectives/12/%') AND (`f`.`mimetype`
= 14) ORDER BY timestamp limit 7;
+------+-------------+-------+--------+-----------------------------------------------------------------------------------------------------------------------------------------------+---------+---------+--------------+------+---------+----------+------------+---------------------------------+
| id   | select_type | table | type   | possible_keys                                                                                                                                 | key     | key_len | ref          | rows | r_rows  | filtered | r_filtered | Extra                           |
+------+-------------+-------+--------+-----------------------------------------------------------------------------------------------------------------------------------------------+---------+---------+--------------+------+---------+----------+------------+---------------------------------+
|    1 | SIMPLE      | p     | ALL    | collectives_pages_file_index                                                                                                                  | NULL    | NULL    | NULL         | 1779 | 1779.00 |   100.00 |     100.00 | Using temporary; Using filesort |
|    1 | SIMPLE      | f     | eq_ref | PRIMARY,fs_storage_path_hash,fs_storage_mimetype,fs_storage_mimepart,fs_storage_size,store_mimepart,fs_storage_path_prefix,fs_id_storage_size | PRIMARY | 8       | oc.p.file_id | 1    | 1.00    |     1.07 |      42.16 | Using where                     |
+------+-------------+-------+--------+-----------------------------------------------------------------------------------------------------------------------------------------------+---------+---------+--------------+------+---------+----------+------------+---------------------------------+
2 rows in set (0.009 sec)

[blizzz]

this will not catch pages that are only in the filecache and do not have an entry in the collectives table

To my understanding, any collective page must have an entry in the filecache. They reside in the user`s file system after all, right. How realistic is it then to have pages that are only in the filecache, but not in the collectives pages table? Would that be files that are created manually outside of collectives, directly in the file system?

[juliusknorr]

To my understanding, any collective page must have an entry in the filecache. They reside in the user`s file system after all, right. How realistic is it then to have pages that are only in the filecache, but not in the collectives pages table? Would that be files that are created manually outside of collectives, directly in the file system?

Yes, if there is a collective page entry there is also a filecache entry. There is a more general issue - if you upload a markdown file through the files app into the collectives folder. With this change it will not show up in the recent files, but by current implementation it is also not showing in the collectives app. Might have been a design decision, but unsure so I'd rather keep that for a separate discussion.