From a3c7e682acdffb324be3cd33a275da2f326c9ba6 Mon Sep 17 00:00:00 2001 From: Madison Swain-Bowden Date: Wed, 20 Dec 2023 17:49:20 -0800 Subject: [PATCH 1/3] Update copy changes implementation plan to avoid table rename --- ...-implementation_plan_mature_copy_change.md | 55 +++++++++++++++++-- 1 file changed, 50 insertions(+), 5 deletions(-) diff --git a/documentation/projects/proposals/trust_and_safety/content_report_moderation/20230517-implementation_plan_mature_copy_change.md b/documentation/projects/proposals/trust_and_safety/content_report_moderation/20230517-implementation_plan_mature_copy_change.md index 4cfac7dbc45..84479cd77cd 100644 --- a/documentation/projects/proposals/trust_and_safety/content_report_moderation/20230517-implementation_plan_mature_copy_change.md +++ b/documentation/projects/proposals/trust_and_safety/content_report_moderation/20230517-implementation_plan_mature_copy_change.md @@ -56,11 +56,22 @@ are broken into two aspects (code and copy). #### Code +```{note} +**Update 2023-12-20**: The maintainers determined after the fact that the multi-step +process for updating the table names themselves would be too complicated, so this +plan was modified to only change the names referenced within the code and within +the tables themselves. The previous plan is kept for posterity below. +``` + The following model names will need to be updated: -- [`AbstractMatureMedia`](https://github.com/WordPress/openverse/blob/2041c5df1e9d5d1f9f37e7c177f2e70f61ea5dba/api/api/models/media.py#L334-L333) -- [`MatureImage`](https://github.com/WordPress/openverse/blob/2041c5df1e9d5d1f9f37e7c177f2e70f61ea5dba/api/api/models/image.py#L83) -- [`MatureAudio`](https://github.com/WordPress/openverse/blob/2041c5df1e9d5d1f9f37e7c177f2e70f61ea5dba/api/api/models/audio.py#L263) +- [`AbstractMatureMedia`](https://github.com/WordPress/openverse/blob/aa16d4f1be7607b12c428886b9890bdd947cc71c/api/api/models/media.py#L332) +- [`MatureImage`](https://github.com/WordPress/openverse/blob/eb0e906e7300e32fae945eb9222ed47a50fb72a2/api/api/models/image.py#L83) +- [`MatureAudio`](https://github.com/WordPress/openverse/blob/aa16d4f1be7607b12c428886b9890bdd947cc71c/api/api/models/audio.py#L263) + +The database table name for these models will _not_ be changed, and they will +need to be referenced using +[the `db_table` model attribute](https://docs.djangoproject.com/en/5.0/ref/models/options/#django.db.models.Options.db_table). The [`mature` reason](https://github.com/WordPress/openverse/blob/2041c5df1e9d5d1f9f37e7c177f2e70f61ea5dba/api/api/models/media.py#L22-L21) @@ -70,8 +81,40 @@ value) in `AbstractMediaReport` definition will also need to be changed to `sensitive` and `sensitive_filtered` respectively. The migration necessary for changing the data is described below, however the API will need to continue to accept `mature` as a report reason and convert it internally to `sensitive` (at -least until we decide to make a version change to the API). The media report -tables already use table aliases as well +least until we decide to make a version change to the API). + +In order to perform both of these changes in a +[zero-downtime manner](https://docs.openverse.org/general/zero_downtime_database_management.html), +the following steps will need to be taken: + +1. Rename the models (without changing the underlying tables) and associated + references for: + - `AbstractMatureMedia` -> `AbstractSensitiveMedia` + - `MatureImage` -> `SensitiveImage` + - `MatureAudio` -> `SensitiveAudio` +2. Add `sensitive` and `sensitive_filtered` as new statuses for the + `AbstractMediaReport` class. Also add `sensitive_filtered` as to the check + for + [creating sensitive media records](https://github.com/WordPress/openverse/blob/aa16d4f1be7607b12c428886b9890bdd947cc71c/api/api/models/media.py#L224). + Instance of `mature` that are supplied for the reporting endpoints should be + converted to `sensitive`. +3. Add `sensitive` as an alias for the + [`mature` query parameter](https://github.com/WordPress/openverse/blob/3986e71c722e6cd8aca8a66a1f37d710aadc9a19/api/api/serializers/media_serializers.py#L120), + and deprecate the `mature` parameter. +4. Deploy this version of the API. +5. Create a + [data management command](https://docs.openverse.org/general/zero_downtime_database_management.html#django-management-command-based-data-transformations) + which converts all the `mature` and `mature_filtered` values to `sensitive` + and `sensitive_filtered` respectively for both report tables. +6. Deploy this version of the API and run the data management command to convert + all the values. +7. Remove the conversions from `mature` to `sensitive` and remove `mature` as a + choice for the `AbstractMediaReport` table. +8. Deploy the API. + +
Prior plan involving table changes + +The media report tables already use table aliases as well ([`ImageReport`](https://github.com/WordPress/openverse/blob/7b95a4c8eaa9804f53b4be7ac969e04ca437695a/api/api/models/image.py#L121-L122) and [`AudioReport`](https://github.com/WordPress/openverse/blob/7b95a4c8eaa9804f53b4be7ac969e04ca437695a/api/api/models/audio.py#L304-L305)) - @@ -120,6 +163,8 @@ the following steps will need to be taken: `NsfwReportAudio`) from the API codebase and deploy this version of the API. This will remove the old tables from the database. +
+ Per the [detecting sensitive textual content project plan](https://docs.openverse.org/projects/proposals/trust_and_safety/detecting_sensitive_textual_content/20230309-project_proposal_detecting_sensitive_textual_content.html#designation-of-results-with-sensitive-terms), the `mature` field of the media serializer will remain, and will eventually have From e77a323d88797cf9d17bdd967bc96da3f00f45e3 Mon Sep 17 00:00:00 2001 From: Madison Swain-Bowden Date: Tue, 26 Dec 2023 11:15:53 -0800 Subject: [PATCH 2/3] Clarify status vs reason --- .../20230517-implementation_plan_mature_copy_change.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/documentation/projects/proposals/trust_and_safety/content_report_moderation/20230517-implementation_plan_mature_copy_change.md b/documentation/projects/proposals/trust_and_safety/content_report_moderation/20230517-implementation_plan_mature_copy_change.md index 84479cd77cd..f7a9f4f65d8 100644 --- a/documentation/projects/proposals/trust_and_safety/content_report_moderation/20230517-implementation_plan_mature_copy_change.md +++ b/documentation/projects/proposals/trust_and_safety/content_report_moderation/20230517-implementation_plan_mature_copy_change.md @@ -92,9 +92,9 @@ the following steps will need to be taken: - `AbstractMatureMedia` -> `AbstractSensitiveMedia` - `MatureImage` -> `SensitiveImage` - `MatureAudio` -> `SensitiveAudio` -2. Add `sensitive` and `sensitive_filtered` as new statuses for the - `AbstractMediaReport` class. Also add `sensitive_filtered` as to the check - for +2. Add `sensitive` and `sensitive_filtered` as a new status and reason + respectively for the `AbstractMediaReport` class. Also add + `sensitive_filtered` as to the check for [creating sensitive media records](https://github.com/WordPress/openverse/blob/aa16d4f1be7607b12c428886b9890bdd947cc71c/api/api/models/media.py#L224). Instance of `mature` that are supplied for the reporting endpoints should be converted to `sensitive`. From 670ffdbe7af20fa6069aebc0314bcd8b08aba7bf Mon Sep 17 00:00:00 2001 From: Madison Swain-Bowden Date: Thu, 18 Jan 2024 07:46:46 -0800 Subject: [PATCH 3/3] Remove steps related to the media reports --- ...-implementation_plan_mature_copy_change.md | 39 +++---------------- 1 file changed, 5 insertions(+), 34 deletions(-) diff --git a/documentation/projects/proposals/trust_and_safety/content_report_moderation/20230517-implementation_plan_mature_copy_change.md b/documentation/projects/proposals/trust_and_safety/content_report_moderation/20230517-implementation_plan_mature_copy_change.md index f7a9f4f65d8..c7510529b97 100644 --- a/documentation/projects/proposals/trust_and_safety/content_report_moderation/20230517-implementation_plan_mature_copy_change.md +++ b/documentation/projects/proposals/trust_and_safety/content_report_moderation/20230517-implementation_plan_mature_copy_change.md @@ -72,45 +72,16 @@ The following model names will need to be updated: The database table name for these models will _not_ be changed, and they will need to be referenced using [the `db_table` model attribute](https://docs.djangoproject.com/en/5.0/ref/models/options/#django.db.models.Options.db_table). +This can be accomplished in a zero-downtime manner with a single deployment. The [`mature` reason](https://github.com/WordPress/openverse/blob/2041c5df1e9d5d1f9f37e7c177f2e70f61ea5dba/api/api/models/media.py#L22-L21) (and [`mature_filtered`](https://github.com/WordPress/openverse/blob/2041c5df1e9d5d1f9f37e7c177f2e70f61ea5dba/api/api/models/media.py#L18-L17) -value) in `AbstractMediaReport` definition will also need to be changed to -`sensitive` and `sensitive_filtered` respectively. The migration necessary for -changing the data is described below, however the API will need to continue to -accept `mature` as a report reason and convert it internally to `sensitive` (at -least until we decide to make a version change to the API). - -In order to perform both of these changes in a -[zero-downtime manner](https://docs.openverse.org/general/zero_downtime_database_management.html), -the following steps will need to be taken: - -1. Rename the models (without changing the underlying tables) and associated - references for: - - `AbstractMatureMedia` -> `AbstractSensitiveMedia` - - `MatureImage` -> `SensitiveImage` - - `MatureAudio` -> `SensitiveAudio` -2. Add `sensitive` and `sensitive_filtered` as a new status and reason - respectively for the `AbstractMediaReport` class. Also add - `sensitive_filtered` as to the check for - [creating sensitive media records](https://github.com/WordPress/openverse/blob/aa16d4f1be7607b12c428886b9890bdd947cc71c/api/api/models/media.py#L224). - Instance of `mature` that are supplied for the reporting endpoints should be - converted to `sensitive`. -3. Add `sensitive` as an alias for the - [`mature` query parameter](https://github.com/WordPress/openverse/blob/3986e71c722e6cd8aca8a66a1f37d710aadc9a19/api/api/serializers/media_serializers.py#L120), - and deprecate the `mature` parameter. -4. Deploy this version of the API. -5. Create a - [data management command](https://docs.openverse.org/general/zero_downtime_database_management.html#django-management-command-based-data-transformations) - which converts all the `mature` and `mature_filtered` values to `sensitive` - and `sensitive_filtered` respectively for both report tables. -6. Deploy this version of the API and run the data management command to convert - all the values. -7. Remove the conversions from `mature` to `sensitive` and remove `mature` as a - choice for the `AbstractMediaReport` table. -8. Deploy the API. +value) in `AbstractMediaReport` were initially going to be changed as part of +this, they will actually be deprecated in further steps as described by the +[Django admin tools and access control for moderators plan](20231208-implementation_plan_django_admin_moderator_access.md). +They will not be changed at this time.
Prior plan involving table changes