Opt out phones in EA use case and sample script

docs/use_cases/opt_outs_to_everyaction.rst

@@ -0,0 +1,17 @@
+===============
+Opt-outs to EveryAction
+===============
+
+As carriers tighten restrictions on peer-to-peer texting through 10DLC rules, it becomes more and more crucial for organizations to avoid texting people who have opted out of their communications. This is a challenge for organizations who pull outreach lists from EveryAction or VAN but run their actual texting program in another tool because their opt-outs are tracked in a different system from the one where they pull their lists. A number of commonly used texting tools have integrations with EveryAction and VAN but they don't always sync opt-out dispositions back into EveryAction/VAN.
+
+The Movement Cooperative worked with a couple of different member organizations to create a script using Parsons that would opt-out phone numbers in EveryAction to prevent them from being pulled into future outreach lists. The script only updates existing records, it does not create new ones, so it requires you to provide a VAN ID and assumes that the people you want to opt out already exist in EveryAction.
+
+The script requires the user to provide a table containing columns for phone number (must be named `phone`), committee ID (must be named `committeeid`), and vanid (must be named `vanid`).
+
+Some questions to consider when you construct this table are:
+
+- Which committees do you want to opt people out in?
+- Multiple people can have the same phone number assigned to them in EveryAction. Do you want to opt out a phone regardless of who it's associated with, or do you want to attempt to identify the specific person who opted out in your texting tool?
+- People can have multiple phone numbers associated with them in EveryAction. Do you want to opt out just the specific phone number that shows up in the texting tool data or all phones associated with a given person?
+
+The code we used is available as a `sample script <https://github.com/move-coop/parsons/tree/master/useful_resources/sample_code/opt_outs_everyaction.py>`_ for you to view, re-use, or customize to fit your needs.

useful_resources/sample_code/README.md

@@ -12,12 +12,15 @@ Please also add your new script to the table below.
 
 # Existing Scripts
 
-| File Name                   | Brief Description                                                              | Connectors Used       | Written For Parsons Version |
+| File Name                       | Brief Description                                                              | Connectors Used       | Written For Parsons Version |
 | --------------------------- | ------------------------------------------------------------------------------ | --------------------- | --------------------------- |
-| apply_activist_code.py      | Gets activist codes stored in Redshift and applies to users in Van             | Redshift, VAN         | unknown                     |
-| s3_to_redshift.py           | Moves files from S3 to Redshift                                                | Redshift, S3          | unknown                     |
-| s3_to_s3.py                 | Get files from vendor s3 bucket and moves to own S3 bucket                     | S3                    | unknown                     |
-| update_user_in_actionkit.py | Adds a voterbase_id (the Targetsmart ID) to users in ActionKit                 | Redshift, ActionKit   | unknown                     |
-| zoom_to_van.py              | Adds Zoom attendees to VAN and applies an activist code                        | Zoom, VAN             | 0.15.0                      |
-| ngpvan_sample_list.py       | Creates a new saved list from a random sample of an existing saved list in VAN | VAN                   | unknown                     |
-| actblue_to_google_sheets.py | Get information about contributions from ActBlue and put in a new Google Sheet | ActBlue, GoogleSheets | 0.18.0                      |
+| actblue_to_google_sheets.py     | Get information about contributions from ActBlue and put in a new Google Sheet | ActBlue, GoogleSheets | 0.18.0                       |
+| apply_activist_code.py          | Gets activist codes stored in Redshift and applies to users in Van             | Redshift, VAN         | unknown                     |
+| civis_job_status_slack_alert.py | Posts Civis job and workflow status alerts in Slack                            | Slack                 | unknown                     |
+| mysql_to_googlesheets.py        | Queries a MySQL database and saves the results to a Google Sheet               | GoogleSheets, MySQL   | unknown                     |
+| ngpvan_sample_list.py           | Creates a new saved list from a random sample of an existing saved list in VAN | VAN                   | unknown                     |
+| opt_outs_everyaction.py         | Opts out phone numbers in EveryAction from a Redshift table                    | Redshift, VAN         | 0.21.0                     |
+| s3_to_redshift.py               | Moves files from S3 to Redshift                                                | Redshift, S3          | unknown                     |
+| s3_to_s3.py                     | Get files from vendor s3 bucket and moves to own S3 bucket                     | S3                    | unknown                     |
+| update_user_in_actionkit.py     | Adds a voterbase_id (the Targetsmart ID) to users in ActionKit                 | Redshift, ActionKit   | unknown                     |
+| zoom_to_van.py                  | Adds Zoom attendees to VAN and applies an activist code                        | Zoom, VAN             | 0.15.0                       |

useful_resources/sample_code/opt_outs_everyaction.py

@@ -0,0 +1,187 @@
+import os
+import requests
+import time
+import json
+from parsons import Redshift, Table, VAN
+from parsons import logger
+from datetime import datetime
+
+# Committee Information and Credentials
+
+# This script can be run against multiple EveryAction committees.
+# The COMMITTEES_STR variable should be a list of committee information in JSON format.
+# The information should include the committee's name, ID, and API key.
+# [{"committee": "Committee 1", "committee_id": "12345", "api_key": "Committee 1 API key"},
+# {"committee": "Committee 2", "committee_id": "56789", "api_key": "Committee 2 API key"}]
+# This script was originally written to run in Civis Platform, which pulls environment variables
+# in as strings.
+COMMITTEES_STR = os.environ['COMMITTEES_PASSWORD']
+COMMITTEES = json.loads(COMMITTEES_STR)
+
+# Configuration Variables
+
+# It is assumed that the tables below live in a schema in Redshift
+# Therefore, in order to interact with them, we must name them using the format schema.table.
+# More on this here: https://docs.aws.amazon.com/redshift/latest/gsg/t_creating_schema.html
+
+# The OPT_OUT_TABLE is a table of phones to opt out.
+# The variable must be a string with the format schema.table.
+# The table must contain the columns phone, committeeid, and vanid.
+OPT_OUT_TABLE = os.environ['OPT_OUT_TABLE']
+
+# The SUCCESS_TABLE is a table where successful opt-outs will be logged.
+# The variable must be a string with the format schema.table.
+# This table's columns will be: vanid, phone, committeeid, and applied_at.
+SUCCESS_TABLE = os.environ['SUCCESS_TABLE']
+
+# The ERROR_TABLE is a table where errors will be logged.
+# The variable must be a string with the format schema.table.
+# This table's columns will be : vanid, phone, committeeid, errored_at, and error.
+ERROR_TABLE = os.environ['ERROR_TABLE']
+
+# To use the Redshift connector, set the following environmental variables:
+# REDSHIFT_USERNAME
+# REDSHIFT_PASSWORD
+# REDSHIFT_HOST
+# REDSHIFT_DB
+# REDSHIFT_PORT
+
+rs = Redshift()
+
+
+def attempt_optout(every_action, row, applied_at, committeeid,
+                   success_log, error_log, attempts_left=3):
+
+    vanid = row['vanid']
+    phone = row['phone']
+
+    # Documentation on this json construction is here
+    # https://docs.ngpvan.com/reference/common-models
+    match_json = {
+        "phones": [
+            {"phoneNumber": phone,
+             "phoneOptInStatus": "O"}
+        ]
+    }
+
+    try:
+        response = every_action.update_person_json(id=vanid, match_json=match_json)
+
+        # If the response is a dictionary the update was successful
+        if isinstance(response, dict):
+            success_log.append({
+                "vanid": response.get('vanId'),
+                "phone": phone,
+                "committeeid": committeeid,
+                "applied_at": applied_at
+            })
+
+            return response
+
+    # If we get an HTTP Error add it to the error log
+    # Usually these errors mean a vanid has been deleted from EveryAction
+    except requests.exceptions.HTTPError as error:
+        error_message = str(error)[:999]
+        error_log.append({
+            "vanid": vanid,
+            "phone": phone,
+            "committeeid": committeeid,
+            "errored_at": applied_at,
+            "error": error_message
+        })
+
+        return error_message
+
+    # If we get a connection error we wait a bit and try again.
+    except requests.exceptions.ConnectionError as connection_error:
+        logger.info("Got disconnected, waiting and trying again")
+
+        while attempts_left > 0:
+            attempts_left -= 1
+
+            # Wait 10 seconds, then try again
+            time.sleep(10)
+            attempt_optout(every_action, row, attempts_left)
+
+        else:
+            # If we are still getting a connection error after our maximum number of attempts
+            # we add the error to the log, save our full success and error logs in Redshift,
+            # and raise the error.
+            connection_error_message = str(connection_error)[:999]
+
+            error_log.append({
+                "vanid": vanid,
+                "phone": phone,
+                "committeeid": committeeid,
+                "errored_at": applied_at,
+                "error": connection_error_message
+            })
+
+            if len(success_log) > 0:
+                success_parsonstable = Table(success_log)
+                logger.info("Copying success data into log table...")
+                rs.copy(success_parsonstable, SUCCESS_TABLE, if_exists='append', alter_table=True)
+                logger.info("Success log complete.")
+
+            if len(error_log) > 0:
+                error_parsonstable = Table(error_log)
+                logger.info("Copying error data into log table...")
+                rs.copy(error_parsonstable, ERROR_TABLE, if_exists='append', alter_table=True)
+                logger.info("Error log complete.")
+
+            raise Exception(f"Connection Error {connection_error}")
+
+
+def main():
+    # Creating empty lists where we'll log successes and errors
+    success_log = []
+    error_log = []
+
+    # Get the opt out data
+    all_opt_outs = rs.query(f"select * from {OPT_OUT_TABLE}")
+
+    # Loop through each committee to opt-out phones
+    for committee in COMMITTEES:
+
+        api_key = committee['api_key']
+        committeeid = committee['committee_id']
+        committee_name = committee['committee']
+
+        every_action = VAN(db='EveryAction', api_key=api_key)
+
+        logger.info(f"Working on opt outs in {committee_name} committee...")
+
+        # Here we narrow the all_opt_outs table to only the rows that correspond
+        # to this committee.
+        opt_outs = all_opt_outs.select_rows(lambda row: str(row.committeeid) == committeeid)
+
+        logger.info(f"Found {opt_outs.num_rows} phones to opt out in {committee_name} committee...")
+
+        # Now we actually update the records
+
+        if opt_outs.num_rows > 0:
+
+            for opt_out in opt_outs:
+
+                applied_at = str(datetime.now()).split(".")[0]
+                attempt_optout(every_action, opt_out, applied_at,
+                               committeeid, success_log, error_log)
+
+    # Now we log results
+    logger.info(f"There were {len(success_log)} successes and {len(error_log)} errors.")
+
+    if len(success_log) > 0:
+        success_parsonstable = Table(success_log)
+        logger.info("Copying success data into log table...")
+        rs.copy(success_parsonstable, SUCCESS_TABLE, if_exists='append', alter_table=True)
+        logger.info("Success log complete.")
+
+    if len(error_log) > 0:
+        error_parsonstable = Table(error_log)
+        logger.info("Copying error data into log table...")
+        rs.copy(error_parsonstable, ERROR_TABLE, if_exists='append', alter_table=True)
+        logger.info("Error log complete.")
+
+
+if __name__ == '__main__':
+    main()