Add API to update a path on a watched library folder

[mikiher]

This adds a an API for notifying the server of an update to a watched library folder. Each call to the API essentially mimics a watcher event, and calls the same event listener that listens to real watcher events.

It is meant to be used by an external watcher (e.g. one that would run on a Docker host), in an attempt to mitigate #2204 (watcher not firing events when watching bind mounts on docker). I'm currently working on such an external watcher that could be run on node.js on the docker host and which will use this API. Hopefully this external watcher can also be packaged as an executable, and could be installed as a service on the host machine.

[advplyr]

This works for me. I did some work on the Watcher yesterday so that it will poll the mtime of newly added files until it stops changing. This is to handle #1362

I also learned about the @this JSDoc tag so that we can get correct intellisense

[iconoclasthero]

  /**
   * POST: /api/watcher/update
   * Update a watch path
   * Req.body { libraryId, path, type, [oldPath] } 
   * type = add, unlink, rename
   * oldPath = required only for rename
   * @this import('../routers/ApiRouter')
   * 
   * @param {import('express').Request} req 
   * @param {import('express').Response} res 
   */

It's adding the metadata.json to the author folder not to the book folder, i.e., the one that was sent as "path".

#!/bin/bash

. ~/.config/abc.conf

[[ ! "$abs_server" || ! "$abs_token" ]] && echo "Check ~/.config/abc.conf" && exit 1

abs_endpoint="/api/watcher/update"

JSON_PAYLOAD=$(cat <<EOF
{
  "libraryId": "$libid",
  "path": "$PWD",
  "type": "add"  `# or "unlink" or "rename"`
`# "oldPath": "your-old-path" # Uncomment this line if type is "rename"`
}
EOF
)

pause "$JSON_PAYLOAD"
pause "abs_response=curl -X POST "$abs_server$abs_endpoint"  -H "Content-Type: application/json" -H
"Authorization: Bearer $abs_token"  -d "$JSON_PAYLOAD")"

abs_response="$(curl -X POST "$abs_server$abs_endpoint" \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer $abs_token" \
     -d "$JSON_PAYLOAD")"

echo "$abs_response"

So the output of the pause functions (read -rp) and echo looks good:

pwd: /library/books/audiobooks/Lomax, Alan/The Land Where the Blues Began (1970) 
$ abs.updatewatchedfolder
{
  "libraryId": "7----<redacted>---9",
  "path": "/library/books/audiobooks/Lomax, Alan/The Land Where the Blues Began (1970)",
  "type": "add"  

}
abs_response=curl -X POST https://qqq.myurl.com/api/watcher/update  -H Content-Type: application/json -H Authorization: Bearer <redacted>  -d { "libraryId": "7---redacted ----9", "path": "/library/books/audiobooks/Lomax, Alan/The Land Where the Blues Began (1970)", "type": "add" })
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   168  100     2  100   166      5    465 --:--:-- --:--:-- --:--:--   470
OK

Looking at the resulting file:

$ ll --no-user; echo ..\/; ll --no-user ..; cat ../metadata.json |jq -r 
.rw-rw-r--  37k 27 Feb  2017 'The Land Where the Blues Began -- Cover 01.jpg'
.rw-rw-r-- 167M 18 May 19:05 'The Land Where the Blues Began -- Audiobook.opus'
../
drwxrwxr-x    - 18 May 20:29 'The Land Where the Blues Began (1970)'
.rw-r--r-- 2.3k 18 May 21:13 metadata.json

{
  "tags": [],
  "chapters": [
    {
      "id": 0,
      "start": 0,
      "end": 5222.171,
      "title": "Part 01a "
    },
    {
      "id": 1,
      "start": 5222.171,
      "end": 10374.517,
      "title": "Part 01b "
    },
    {
      "id": 2,
      "start": 10374.517,
      "end": 15535.745,
      "title": "Part 01c "
    },
    {
      "id": 3,
      "start": 15535.745,
      "end": 20713.717,
      "title": "Part 01d "
    },
    {
      "id": 4,
      "start": 20713.717,
      "end": 26034.762,
      "title": "Part 02a "
    },
    {
      "id": 5,
      "start": 26034.762,
      "end": 31327.83,
      "title": "Part 02b "
    },
    {
      "id": 6,
      "start": 31327.83,
      "end": 36634.56,
      "title": "Part 02c "
    },
    {
      "id": 7,
      "start": 36634.56,
      "end": 41938.912,
      "title": "Part 02d "
    },
    {
      "id": 8,
      "start": 41938.912,
      "end": 47155.441,
      "title": "Part 03a "
    },
    {
      "id": 9,
      "start": 47155.441,
      "end": 52416.405,
      "title": "Part 03b "
    },
    {
      "id": 10,
      "start": 52416.405,
      "end": 57691.97,
      "title": "Part 03c "
    },
    {
      "id": 11,
      "start": 57691.97,
      "end": 62892.8,
      "title": "Part 03d "
    },
    {
      "id": 12,
      "start": 62892.8,
      "end": 68132.284,
      "title": "Part 04a "
    },
    {
      "id": 13,
      "start": 68132.284,
      "end": 73306.123,
      "title": "Part 04b "
    },
    {
      "id": 14,
      "start": 73306.123,
      "end": 78493.572,
      "title": "Part 04c "
    },
    {
      "id": 15,
      "start": 78493.572,
      "end": 79914.359,
      "title": "Part 04d "
    }
  ],
  "title": "Lomax, Alan",
  "subtitle": null,
  "authors": [
    "Alan Lomax"
  ],
  "narrators": [],
  "series": [],
  "genres": [
    "Other"
  ],
  "publishedYear": "1970",
  "publishedDate": null,
  "publisher": "Pantheon",
  "description": "Lomax, who has done more than anyone else to make black music of the South known as a glorious expression of American art, summs up sixty years of \"discovering the African American musical heritage in this journey through the Mississippi Delta.",
  "isbn": "9780679404248",
  "asin": null,
  "language": null,
  "explicit": false,
  "abridged": false
}

and the log result:

[2024-05-19 01:31:36.995] [BackupManager] INFO Backup "2024-05-17T0130" Removed
[2024-05-19 02:02:42.640] [Watcher]   DEBUG File Added audiobooks/Lomax, Alan/The Land Where the Blues Began (1970)
[2024-05-19 02:02:42.640] [Watcher]   ERROR New file folder not found in library "Audiobook Library" with path "audiobooks/Lomax, Alan/The Land Where the Blues Began (1970)"
[2024-05-19 02:02:42.642] [fileUtils] ERROR Failed to getFileMtimeMs Error: ENOENT: no such file or directory, stat 'audiobooks/Lomax, Alan/The Land Where the Blues Began (1970)'
[2024-05-19 02:06:12.166] [Watcher]   DEBUG File Added /library/books/audiobooks/Lomax, Alan/The Land Where the Blues Began (1970)
[2024-05-19 02:06:12.166] [Watcher]   DEBUG Modified file in library "Audiobook Library" and folder "8b2c964b-a2ba-40de-909e-75914400d009" with relPath "/Lomax, Alan/The Land Where the Blues Began (1970)"
[2024-05-19 02:06:12.169] [Watcher]   DEBUG Waiting to add file at "/library/books/audiobooks/Lomax, Alan/The Land Where the Blues Began (1970)". mtimeMs=1716078548465.099 lastMTimeMs=0 (loop 0)
[2024-05-19 02:06:13.169] [Watcher]   DEBUG File finished adding at "/library/books/audiobooks/Lomax, Alan/The Land Where the Blues Began (1970)"
[2024-05-19 02:06:16.555] [libraryFilters] DEBUG Loaded filterdata in 0.38s
[2024-05-19 02:06:16.556] [LibraryScanner] DEBUG [Scanner] Scanning file update groups in folder "8b2c964b-a2ba-40de-909e-75914400d009" of library "Audiobook Library"
[2024-05-19 02:06:16.556] [LibraryScanner] DEBUG [Scanner] scanFolderUpdates fileUpdateGroup [object Object]
[2024-05-19 02:06:16.657] [LibraryScanner] DEBUG Folder update for relative path "Lomax, Alan" is in library item "Lomax, Alan" - scan for updates
[2024-05-19 02:06:16.718] [ScanLogger] DEBUG [Scan] "Lomax, Alan": "Lomax, Alan" Getting metadata with precedence [txtFiles, opfFile, audioMetatags, folderStructure, absMetadata]
[2024-05-19 02:06:16.719] [ScanLogger] DEBUG [Scan] "Lomax, Alan": setChapters: Using embedded chapters in first audio file /library/books/audiobooks/Lomax, Alan/The Land Where the Blues Began (1970)/The Land Where the Blues Began -- Audiobook.opus
[2024-05-19 02:06:16.720] [ScanLogger] DEBUG [Scan] "Lomax, Alan": Found metadata file "/library/books/audiobooks/Lomax, Alan/metadata.json"
[2024-05-19 02:06:16.721] [ScanLogger] DEBUG [Scan] "Lomax, Alan": Library item is up-to-date
[2024-05-19 02:06:16.721] [LibraryScanner] DEBUG Folder scan results [object Object]

[iconoclasthero]

For what it is worth, the watcher did not recognize these files, though inotify eventually did... at least the first time around. This was the second time around for the Lomax book, but everything looks the same.