Print more detailed ("extended") error code in SQLite3 code

[DL6ER]

Thank you for your contribution to the Pi-hole Community!

Please read the comments below to help us consider your Pull Request.

We are all volunteers and completing the process outlined will help us review your commits quicker.

Please make sure you

1. Base your code and PRs against the repositories developmental branch.
2. Sign Off all commits as we enforce the DCO for all contributions
3. Sign all your commits as they must have verified signatures
4. File a pull request for any change that requires changes to our documentation at our documentation repo

---

What does this PR aim to accomplish?:

Print more specific than the standard SQLite3 error codes when available

How does this PR accomplish the above?:

Print extended error code if anything in dbquery() fails

Link documentation PRs if any are needed to support this PR:

---

By submitting this pull request, I confirm the following:

1. I have read and understood the contributors guide, as well as this entire template. I understand which branch to base my commits and Pull Requests against.
2. I have commented my proposed changes within the code and I have tested my changes.
3. I am willing to help maintain this change if there are issues with it later.
4. It is compatible with the EUPL 1.2 license
5. I have squashed any insignificant commits. (git rebase)
6. I have checked that another pull request for this purpose does not exist.
7. I have considered, and confirmed that this submission will be valuable to others.
8. I accept that this submission may not be used, and the pull request closed at the will of the maintainer.
9. I give this submission freely, and claim no ownership to its content.

---

• I have read the above and my PR is ready for review. Check this box to confirm

[DL6ER]

Awaiting testing feedback from @rdwebdesign

[rdwebdesign]

Before:

   [2022-11-25 03:05:01.030 215/T253] ERROR: SQL query "END TRANSACTION" failed: database is locked
   [2022-11-25 03:05:01.030 215/T253] END TRANSACTION failed when trying to store queries to long-term database
   [2022-11-25 03:05:01.030 215/T253] Keeping queries in memory for later new attempt
   [2022-11-25 03:05:02.110 215/T253] ERROR: SQL query "END TRANSACTION" failed: database is locked
   [2022-11-25 03:05:02.110 215/T253] WARNING: Storing devices in network table failed: database is locked

Using the new branch:

   [2022-11-25 03:27:01.114 733/T771] ERROR: SQL query "END TRANSACTION" failed: database is locked (SQLITE_BUSY)
   [2022-11-25 03:27:01.115 733/T771] END TRANSACTION failed when trying to store queries to long-term database
   [2022-11-25 03:27:01.115 733/T771] Keeping queries in memory for later new attempt
   [2022-11-25 03:27:02.176 733/T771] ERROR: SQL query "END TRANSACTION" failed: database is locked (SQLITE_BUSY)
   [2022-11-25 03:27:02.177 733/T771] WARNING: Storing devices in network table failed: database is locked

Note:
I just posted the relevant lines

[yubiuser]

So the new information here is only (SQLITE_BUSY)

[DL6ER]

This PR had the intention to be useful when debugging a "I/O error" which been have a lot of reasons. I'll try to provide an example myself later.

[DL6ER]

Okay, what I did here was the following: Removed o-w from /etc/pihole so FTL wasn't able to write to the directory any longer (essential to create the auxiliary database journal file). The database file itself was still writable:

$ ls -lh /etc/ | grep pihole
dr-xrwxr-x 3 pihole pihole    4.0K Nov 25 09:46 pihole

$ ls -lh /etc/pihole | grep pihole-FTL.db
-rw-rw-r-- 1 pihole pihole 106M Nov 25 09:46 pihole-FTL.db

When starting FTL on development we see:

ERROR: SQL query "DELETE FROM message;" failed: attempt to write a readonly database

which is kind of confusing as the database file itself has the correct permissions. However, SQLite3 means the entirety of the database here (including the transient journal file) which is confusing if you are not aware of this.

On the new branch we get the slightly more detailed message:

ERROR: SQL query "DELETE FROM message;" failed: attempt to write a readonly database (SQLITE_READONLY_DIRECTORY)

The extended error code at the very end gives us a better picture of what to look out for during debugging.

---

Other examples where this might turn out to be useful are the mentioned very general "disk I/O error" which can have the following error codes underneath:

• SQLITE_IOERR_READ
• SQLITE_IOERR_SHORT_READ
• SQLITE_IOERR_WRITE
• SQLITE_IOERR_FSYNC
• SQLITE_IOERR_DIR_FSYNC
• SQLITE_IOERR_TRUNCATE
• SQLITE_IOERR_FSTAT
• SQLITE_IOERR_UNLOCK
• SQLITE_IOERR_RDLOCK
• SQLITE_IOERR_DELETE
• SQLITE_IOERR_NOMEM
• SQLITE_IOERR_ACCESS
• SQLITE_IOERR_CHECKRESERVEDLOCK
• SQLITE_IOERR_LOCK
• SQLITE_IOERR_CLOSE
• SQLITE_IOERR_DIR_CLOSE
• SQLITE_IOERR_SHMOPEN
• SQLITE_IOERR_SHMSIZE
• SQLITE_IOERR_SHMLOCK
• SQLITE_IOERR_SHMMAP
• SQLITE_IOERR_SEEK
• SQLITE_IOERR_DELETE_NOENT
• SQLITE_IOERR_MMAP
• SQLITE_IOERR_GETTEMPPATH
• SQLITE_IOERR_CONVPATH

Some other examples for the rather generic unable to open database file error message:

• SQLITE_CANTOPEN_NOTEMPDIR
• SQLITE_CANTOPEN_ISDIR
• SQLITE_CANTOPEN_FULLPATH
• SQLITE_CANTOPEN_CONVPATH
• SQLITE_CANTOPEN_SYMLINK

I will stop here and assume you might be convinced.

[yubiuser]

Thanks. This is really useful.

As the corresponding strings are manually copied from sqlite3.c: should we add a test, that they are up-to-date, so we notice if they change?

[DL6ER]

After thinking about it, I figured it's actually much easier to simply patch SQLite3 to offer this function publicly than writing a complicated test to reliably extract the two functions from common.c and sqlite3.c and compare them.

Whenever SQLite3 is updated in the future, it will be sufficient to replace the files, run sh patch/sqlite3.sh, and then commit the result. This is the same I've been doing locally all the time but this time we add the patch I',m using locally for years to the repository and create a second patch for this new feature. Both patches need to be applied for the compilation to succeed making this something that cannot be forgotten about. Automatic patching is not suitable here as it would require a recompliation of SQLite3 every time causing massive additional delays during development.

Force pushed because rebased to latest development (this branch was previously still on SQLite 3.39.4 instead of 3.40.0).

[pralor-bot]

This pull request has been mentioned on Pi-hole Userspace. There might be relevant details there:

https://discourse.pi-hole.net/t/pi-hole-ftl-v5-20-and-web-v5-18-released/59959/1