EES-5329 Create new database users for local environment and new public_data_read_write group role

[benoutram]

This PR adds new application user roles for running locally and a new public_data_read_write group role to be applied to application user or individual user roles needing read and write permissions on public schema objects.

New version of Public API seed data

There is a new version public-api-16.zip of the seed data to reflect the changes in ownership of the public schema objects and grants after introducing new application user roles.

New application user roles for running locally

Up until now we have configured application services running locally to use the postgres superuser. That doesn't reflect how we run the services in Azure where they each have their own application user role with privileges granted.

This PR adds a new Postgres initialisation script which is run by the Docker entrypoint and creates new application user roles:

• app_public_data_api
• app_public_data_processor
• app_admin
• app_publisher

Each of the services application settings files are changed to use these new user roles for the database logon.

The initialisation script grants the app_public_data_api role CREATE privilege to create new objects which it will own on the public schema, and grants all the other application user roles USAGE privilege to access objects on the public schema . This doesn't grant any additional permissions (e.g. SELECT).

Granting read and write privileges for public tables and sequences using the public_data_read_write role

We are currently limited by the amount of maintenance we can perform on the Postgres database because our own Azure administrator roles don't have any permission to write data. By default, every object created is owned by the role that created it. In our case that's the Public API's application role which is responsible for running database migrations.

Our administrator roles inherit role azure_pg_admin but this doesn't have sufficient rights to write data. Role azure_pg_admin inherits predefined role pg_read_all_data, the equivalent of having as SELECT rights on objects, and USAGE rights on all schemas. However Azure Database for PostgreSQL - Flexible Server does not allow users to be granted the predefined role pg_write_all_data.

In this PR we add the ability to manage data ourselves by granting privileges to a new public_data_read_write role which can be granted to our own individual Azure administrator roles if they require read and write privileges on public schema objects.

The initialisation script when running locally, or the initial migration run by Public.Data.Api in Azure environments will grant privileges by default to the 'public_data_read_write' group role for objects in the public schema subsequently created by the Public.Data.API applications user role.

Setting up public_data_read_write in Azure

In Azure environments this role needs to be created manually before the database migration is run which grants privileges:

CREATE ROLE public_data_read_write WITH NOLOGIN;

Membership of this role can be granted to our administrator roles in Azure who require rights to write data on public schema objects.

GRANT public_data_read_write TO <role>;

Using public_data_read_write locally

In the local environment the public_data_read_write role is created by the initialisation script. Using it locally is optional since you can continue to connect to the database with the postgres superuser using your preferred tool of choice.

If you do want to set up a personal user role which is a member of public_data_read_write:

psql -U postgres -d public_data <<-EOSQL
    CREATE ROLE my_user WITH LOGIN PASSWORD 'password';
    GRANT USAGE ON SCHEMA public TO my_user;
    GRANT public_data_read_write TO my_user;
EOSQL

Then use that to connect:

psql -U my_user -d public_data

Testing public_data_read_write

Create a new user test_user_1 without public_data_read_write role membership:

psql -U postgres -d public_data <<-EOSQL
    CREATE ROLE test_user_1 WITH LOGIN PASSWORD 'password';
    GRANT USAGE ON SCHEMA public TO test_user_1;
EOSQL

Attempt to SELECT from a table in the public schema logged in as test_user_1:

psql -U test_user_1 -d public_data <<-EOSQL
    SELECT COUNT(*) from "DataSets"
EOSQL

🔴 Prints

ERROR:  permission denied for table DataSets

Attempt to INSERT into a table in the public schema logged in as test_user_1:

psql -U test_user_1 -d public_data <<-EOSQL
    INSERT INTO "DataSets" ("Id", "Title", "Summary", "PublicationId", "Status", "Created")
    VALUES ('f76ea237-efda-47bc-b0d1-bcdc91f40bb8','Title','Summary','cbbd299f-8297-44bc-92ac-558bcf51f8ad','Published',now());
EOSQL

🔴 Prints

ERROR:  permission denied for table DataSets

Attempt to UPDATE into a table in the public schema logged in as test_user_1:

psql -U test_user_1 -d public_data <<-EOSQL
    UPDATE "DataSets" SET "Title" = 'Updated' WHERE "Id" = 'f76ea237-efda-47bc-b0d1-bcdc91f40bb8'
EOSQL

🔴 Prints

ERROR:  permission denied for table DataSets

Attempt to DELETE from a table in the public schema logged in as test_user_1:

psql -U test_user_1 -d public_data <<-EOSQL
    DELETE FROM "DataSets" WHERE "Id" = 'f76ea237-efda-47bc-b0d1-bcdc91f40bb8'
EOSQL

🔴 Prints

ERROR:  permission denied for table DataSets

Attempt to TRUNCATE a table in the public schema logged in as test_user_1:

psql -U test_user_1 -d public_data <<-EOSQL
    TRUNCATE "DataSets" CASCADE
EOSQL

🔴 Prints

ERROR:  permission denied for table DataSets

Create a new user test_user_2 with public_data_read_write role membership:

psql -U postgres -d public_data <<-EOSQL
    CREATE ROLE test_user_2 WITH LOGIN PASSWORD 'password';
    GRANT USAGE ON SCHEMA public TO test_user_2;
    GRANT public_data_read_write TO test_user_2;
EOSQL

Attempt to SELECT from a table in the public schema logged in as test_user_2:

psql -U test_user_2 -d public_data <<-EOSQL
    SELECT COUNT(*) from "DataSets"
EOSQL

🟢 Prints

 count
-------
     9
(1 row)

Attempt to INSERT into a table in the public schema logged in as test_user_2:

psql -U test_user_2 -d public_data <<-EOSQL
    INSERT INTO "DataSets" ("Id", "Title", "Summary", "PublicationId", "Status", "Created")
    VALUES ('f76ea237-efda-47bc-b0d1-bcdc91f40bb8','Title','Summary','cbbd299f-8297-44bc-92ac-558bcf51f8ad','Published',now());
EOSQL

🟢 Prints

INSERT 0 1

Attempt to UPDATE into a table in the public schema logged in as test_user_2:

psql -U test_user_2 -d public_data <<-EOSQL
    UPDATE "DataSets" SET "Title" = 'Updated' WHERE "Id" = 'f76ea237-efda-47bc-b0d1-bcdc91f40bb8'
EOSQL

🟢 Prints

UPDATE 1

Attempt to DELETE from a table in the public schema logged in as test_user_2:

psql -U test_user_2 -d public_data <<-EOSQL
    DELETE FROM "DataSets" WHERE "Id" = 'f76ea237-efda-47bc-b0d1-bcdc91f40bb8'
EOSQL

🟢 Prints

DELETE 1

Attempt to TRUNCATE a table in the public schema logged in as test_user_2:

psql -U test_user_2 -d public_data <<-EOSQL
    TRUNCATE "DataSets" CASCADE
EOSQL

🟢 Prints

NOTICE:  truncate cascades to table "DataSetVersions"
NOTICE:  truncate cascades to table "ChangeSetFilterOptions"
NOTICE:  truncate cascades to table "ChangeSetFilters"
NOTICE:  truncate cascades to table "ChangeSetIndicators"
NOTICE:  truncate cascades to table "ChangeSetLocations"
NOTICE:  truncate cascades to table "ChangeSetTimePeriods"
NOTICE:  truncate cascades to table "DataSetVersionImports"
NOTICE:  truncate cascades to table "FilterMetas"
NOTICE:  truncate cascades to table "GeographicLevelMetas"
NOTICE:  truncate cascades to table "IndicatorMetas"
NOTICE:  truncate cascades to table "LocationMetas"
NOTICE:  truncate cascades to table "TimePeriodMetas"
NOTICE:  truncate cascades to table "DataSetVersionMappings"
NOTICE:  truncate cascades to table "FilterOptionMetaLinks"
NOTICE:  truncate cascades to table "LocationOptionMetaLinks"
TRUNCATE TABLE