Skip to content

Commit

Permalink
dfs: add fast lookup table for files
Browse files Browse the repository at this point in the history
Currently, looking up files on dfs is quite slow. Every time a fopen()
is done (or a dfs_open), the filesystem code scans the whole filesystem
looking for a file matching the requested one (it actually does do
that in a directory aware way, but it still needs to scan linearly
directories to look for matches). "Scanning" means reading one filesystem
sector via PI for each file, linearly.

This basically happens because dfs doesn't have a real "directory"
sector that stores the list of all the files with pointers to them.

This commit adds that. It adds a global datastructure, indexed by
filename hash, loaded once at filesystem mount, that allows to quickly
lookup for a file. It also starts parting with the approach of DFS
having the concept of "current working directory", which is basically
never used in libdragon producitons (and also, if needed, shouldn't be
part of the filesystem code but possibly a higher level).

Doing a test with 1000 files in the filesystem (measures are CPU ticks):

OLD CODE:  best=10,357, worst=2,677,366
NEW CODE:  best=   980, worst=    3,976
  • Loading branch information
gamemasterplc authored and rasky committed Jan 11, 2025
1 parent 2787538 commit 4c62799
Show file tree
Hide file tree
Showing 4 changed files with 510 additions and 186 deletions.
27 changes: 26 additions & 1 deletion include/dfsinternal.h
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@
/** @brief The size of a sector payload */
#define SECTOR_PAYLOAD 252

/** @brief Prime number used for hash lookups */
#define DFS_LOOKUP_PRIME 31

/** @brief Representation of a directory entry */
struct directory_entry
{
Expand Down Expand Up @@ -52,6 +55,28 @@ typedef struct dfs_open_file_s
uint32_t cart_start_loc;
} dfs_open_file_t;

/** @brief Data for a single file in dfs_lookup_t */
typedef struct dfs_lookup_file_s {
/** @brief Hash of the path string */
uint32_t path_hash;
/** @brief Top 12 bits: length of the path string; lowest 20 bits: offset of the path string */
uint32_t path_ofs;
/** @brief Data offset for file */
uint32_t data_ofs;
/** @brief Data length for file */
uint32_t data_len;
} dfs_lookup_file_t;

/** @brief Data for DFS file lookup used to speed up file open performance */
typedef struct dfs_lookup_s {
/** @brief Number of files */
uint32_t num_files;
/** @brief Base offset for path data */
uint32_t path_ofs;
/** @brief Array of file entries */
dfs_lookup_file_t files[];
} dfs_lookup_t;

/** @} */ /* dfs */

#endif
#endif
Loading

0 comments on commit 4c62799

Please sign in to comment.