[BUG] rgw/sfs: process aborts when writing data if running as unprivileged user

[tserong]

Describe the bug

The radosgw binary can be run with --setuser and --setgroup parameters, to make it drop root privileges after the process is initialized. Unfortunately, the SFS data store is created before privileges are dropped, which means the data directory and sqlite database are owned by root. Note the log on startup, which shows this ordering:

2022-10-28T17:40:37.530+1100 7f0195381d40  0 rgw main: sfs init!
2022-10-28T17:40:37.534+1100 7f0195381d40  0 sfs serving data from "/data"
2022-10-28T17:40:37.534+1100 7f0195381d40  0 rgw main: Found default user "testid" in database.
2022-10-28T17:40:37.570+1100 7f0195381d40  0 framework: beast
2022-10-28T17:40:37.570+1100 7f0195381d40  0 framework conf key: ssl_certificate, val: config://rgw/cert/$realm/$zone.crt
2022-10-28T17:40:37.570+1100 7f0195381d40  0 framework conf key: ssl_private_key, val: config://rgw/cert/$realm/$zone.key
2022-10-28T17:40:37.570+1100 7f0195381d40  0 starting handler: beast
2022-10-28T17:40:37.570+1100 7f0195381d40  0 set uid:gid to 486:473 (s3gw:s3gw)

When run as an unprivileged user like this, the process starts just fine, but then if you try to write anything (create a bucket, put an object), the process will abort.

To Reproduce

1. Create a new, unprivileged user and group (groupadd -r s3gw ; useradd -r -g s3gw -s /sbin/nologin -d /var/lib/empty s3gw)
2. Run radosgw with radosgw -d --no-mon-config --id s3gw --rgw-data /data --run-dir /run --rgw-sfs-data-path /data --rgw-backend-store sfs --debug-rgw 1 --setuser s3gw --setgroup s3gw
3. Observe that /data/s3gw.db is created world readable, but only writable by root
4. Try creating a bucket and/or putting an object. Observe that the radosgw process aborts.

Expected behavior

The data directory and s3gw.db are owned by the user and group s3gw is running as, rather than being owned by root.

[tserong]

Looks like radosgw_Main() uses CINIT_FLAG_DEFER_DROP_PRIVILEGES to avoid dropping privileges in the global_init functions, then later AsioFrontend::init() calls drop_privileges() to do the job. One possible solution would be to chown() the data directory and sqlite database somewhere in the middle. This seems to be the approach taken in ceph's global_init code to change ownership of pidfiles and logfiles (look for chown_path in src/global/global_init.cc).