v3.2.0

What's Changed

This release includes enhancements to existing connectors, including our Action Network, NGPVAN, and Google Sheets connectors, bug fixes, a few updates, and a change in our base image to Python 3.8!

Also, a special shout-out to @anzelpwj, @jdw25, @strangeways, and @jeffjrodriguez for contributing their first parsons PR!

Enhancements and New Features

• Adding a data paste function to Google Sheets by @anzelpwj in #1045
• Allow passing google auth Credentials directly to BQ connector by @austinweisgrau in #1042
• EveryAction: Add email endpoint methods to retrieve email stats from TargetedEmail by @matthewkrausse in #1003
• Update GET helper function, add custom columns to create profile by @cmdelrio in #1055
• added encoding to s3_copy function in redshift class by @sharinetmc in #827
• New Feature: DBSync can upsert or append updated rows by @austinweisgrau in #1029
• [Enhancement] - Adding Create Ticket Function To Freshdesk by @NirTatcher in #1057
• Add option to disaggregate A/B test emails by @anzelpwj in #1065
• Add methods to update phone, order user detail, and import action by @technicalex in #1074
• Add ActionKit methods to add a phone number, create and update event fields, and search events in a campaign by @strangeways in #1076
• BigQuery extract table method by @austinweisgrau in #1041
• Features - ActionNetwork API - Unique ID Lists routes support + SQL Mirror support by @NirTatcher in #1025
• adding unique id lists support in ActionNetwork module by @NirTatcher in #1101
• ActionNetwork SQL Mirror querying support + general SSH util to query any db through ssh by @NirTatcher in #1102
• VAN Bulk Import Additions by @codygordon in #1078
• Add campaignId parameter to apply_responses method in People class by @matthewkrausse in #1089

Bug Fixes

• Remove self.project reference from GoogleBigQuery.get_columns() query by @coastlines in #1008
• add list data type to mapping by @KasiaHinkson in #1026
• Revert "add list data type to mapping" by @KasiaHinkson in #1027
• Map 'Decimal' type from incomping Table in BQ by @Jason94 in #1010
• Move flake8, black, and pytest configs to pyproject.toml by @bmos in #1013
• Bugfixes by @bmos in #1012
• Return PATCH response when updating AK user by @crayolakat in #1024
• Bump black from 22.12.0 to 24.3.0 by @dependabot in #1022
• Bump setuptools from 68.0.0 to 70.0.0 by @dependabot in #1094
• Change Import to Absolute by @IanRFerguson in #1096
• Revert "Features - ActionNetwork API - Unique ID Lists routes support… by @austinweisgrau in #1100
• Revert "New Feature: DBSync can upsert or append updated rows" by @shaunagm in #1064
• Hot fix for MobileCommons.create_profile() method by @cmdelrio in #1067
• If petl is loaded from a CSV, use the source file, don't re-write by @austinweisgrau in #1062
• Fix VAN create_event location_ids list bug by @codygordon in #1073
• Fix misspecified tests using MagicMock().called_with() by @austinweisgrau in #1072

Documentation

• Add cautionary note about performance in get_row_count method by @matthewkrausse in #998
• Move variable definition outside try/except block by @austinweisgrau in #1030
• Skip empty tables in GSheets append & overwrite methods by @codygordon in #1080
• Support for python3.12: Take 2 by @jdw25 in #1087
• Pass credentials directly to GCP connectors rather than through environment variable by @austinweisgrau in #1040
• Fix typo in environment variable by @austinweisgrau in #1093
• Remove unnecessary dependency by @austinweisgrau in #1088
• Change dict to json by @KasiaHinkson in #1028

Other Changes

• Removes bluelink connector because platform retired March 2024 by @elyse-weiss in #1034
• Update mac runner to macos-12 instead of mac-latest by @shaunagm in #1049
• Psycopg2 version bump by @anzelpwj in #1051
• Sqlalchemy Update to >= 1.4.22 by @matthewkrausse in #1050
• Update version on precommit hook to match requirements-dev, and make … by @shaunagm in #1053
• Bump azure-storage-blob from 12.3.2 to 12.13.0 by @dependabot in #1061
• bump braintree to 4.17.1 by @jdw25 in #1054
• Run upgraded black formatter on all files by @austinweisgrau in #1035
• add script by @sharinetmc in #1063
• Bump urllib3 from 1.26.18 to 1.26.19 by @dependabot in #1079
• Update census_geocoder.py by @jeffjrodriguez in #1085
• Airtable Updates by @codygordon in #1082
• Python3.8 Base Image by @IanRFerguson in #1090
• new release by @sharinetmc in #1107

New Contributors

• @anzelpwj made their first contribution in #1051
• @jdw25 made their first contribution in #1054
• @strangeways made their first contribution in #1076
• @jeffjrodriguez made their first contribution in #1085

Full Changelog: v3.1.0...v3.2.0

v3.1.0

What's Changed

New Connectors

• Census connector by @Charlie-Kramer in #978
• Formstack connector by @Jason94 in #991

New Features

• implement new hustle api call (create_group_membership) by @nmannes in #903
• Added get_representatives_by_address method to Google Civic Connector by @mkrausse-ggtx in #926
• Add utility method to format phone numbers by @mkrausse-ggtx in #928
• Feature/more auth0 functions by @talevy42 in #874
• Add actionnetwork methods by @NirTatcher in #940
• Action Builder Deactivate & Reactivate Connections by @ydamit in #965
• Create new feature for running and logging dbt commands by @austinweisgrau in #841
• Add head and tail methods to Parsons Table by @matthewkrausse in #1001

Bugfixes and other Changes

• Kasiahinkson/extend list blobs by @KasiaHinkson in #953
• Fix bug in NGPVAN apply_canvass_result method by @mkrausse-ggtx in #927
• Fix incorrect documentation on ActionNetwork upsert_person argument by @austinweisgrau in #915
• Explicitly name vars by @sharinetmc in #964
• Bump Multiple Docs Deps Around Sphinx by @SorenSpicknall in #975
• add client_options with default scopes to client by @graemelorimer in #969
• Update get_columns_list() to accept tables with spaces in the name by @coastlines in #973
• Fix ActionKit collect_upload_errors() Bug by @technicalex in #979
• BigQuery returns empty Table on query with no rows returned by @austinweisgrau in #972
• Miscellaneous fixes to BigQuery connector by @austinweisgrau in #959
• Remove doubled up imports to fix linting by @austinweisgrau in #982
• Bump python-dateutil from v2.8.1 to v2.8.2 by @bmos in #966
• Bump paramiko from 2.11.0 to 3.4.0 by @dependabot in #962
• Log progress when putting file through SFTP by @austinweisgrau in #985
• BigQuery - Call .result() regardless of compression by @IanRFerguson in #974
• Generalize the oauth_api_connector to accept authorization kwargs by @austinweisgrau in #918
• Enable email dict to be passed to VAN.upsert_person() by @austinweisgrau in #963
• Bump grpcio from 1.53.0 to 1.53.2 by @dependabot in #990
• BigQuery connector handles queries without return values (create table, create view) by @austinweisgrau in #1000
• Add config to setup.py so README appears on PyPI by @shaunagm in #996
• 992 bug parsons bigquery upsert is broken by @willyraedy in #994
• Simpler use of pip cache in actions by @bmos in #986
• Raise Error for Big Query .copy() when no GCS_TEMP_BUCKET environment variable or tmp_gcs_bucket parameter by @tracy-u in #997
• Refactor Table initialization to avoid iterating over every row by @austinweisgrau in #983
• Bump version number for minor release by @shaunagm in #1006

New Contributors

• @nmannes made their first contribution in #903
• @graemelorimer made their first contribution in #969
• @coastlines made their first contribution in #973
• @bmos made their first contribution in #966
• @Charlie-Kramer made their first contribution in #978
• @tracy-u made their first contribution in #997

Full Changelog: v3.0.0...v3.1.0

v3.0.0

What's Changed

Breaking Changes

3.0.0 is a major release, which means there are some breaking changes. You can see a full list of all PRs included in this release which have implications for breaking changes here.

1. We now parse boolean types by default instead of coercing them into strings by default. We detect boolean column types when copying a parsons Table to a database and create a boolean column in the database. If you want to maintain the old behavior, to convert the boolean columns in the table to strings before uploading it to the database, like this: table = table.convert(['bool', 'columns', 'here', ...], str) More: #943

2. We've made some major updates to the BigQuery and GoogleCloudStorage connecters:

GoogleBigQuery

The GoogleBigQuery connector was written with compatibility in mind, and utilizes many of the same functions as the Amazon Redshift connector in order to minimize the user experience between the two cloud service providers. GoogleBigQuery is authenticated with a service account JSON file, which can be generated in the GCP user interface and stored locally.

There are several subtle differences between GoogleBigQuery and Redshift, most notably in the .query() function, which runs asynchronously in Google. We recommend using the .delete_table() function rather than sending a DELETE TABLE SQL query through the .query() function for this reason, as the connector will raise an exception when the asynchronous task completes and the table no longer exists; alternatively, the user can pass in .query(sql=sql, return_values=False) to prevent this exception from raising.

In addition to the familiar .copy() function, the GoogleBigQuery connector includes a .copy_large_compressed_file_from_gcs() function to handle large files in cloud storage, such as the voter file. BigQuery streams large uncompressed files in batches, but cannot do so when a file in compressed. This function decompresses the file in question using the correct compression type parameter (gzip is default but zip is also accepted), copies the file to BigQuery, then deletes the decompressed file from cloud storage.

GoogleCloudStorage

Similarly, the GoogleCloudStorage connector provides an API to view and manipulate blobs in cloud storage with compatibility to Amazon's S3 connector. Users can create new storage buckets, load blobs into buckets, list their contents, acquire blob metadata, and download blobs from cloud storage to their local environments. This connector handles the decompression steps outlined above in the . copy_large_compressed_file_from_gcs() function outlined above, and also includes helpful utilities to aid in moving data to and from Google Cloud Storage.

New Connectors

Big thanks to @cmdelrio for adding the new MobileCommons connector (#896) and to @austinweisgrau for the new Catalist Match API connector (#912)!

Other Changes

• Zoom Polls by @IanRFerguson in #886
• Check for empty tables in zoom poll results by @Jason94 in #897
• #741 : Deprecate Slack chat.postMessage as_user argument and allow for new authorship arguments by @angloyna in #891
• #816 Airtable.get_records() fields argument can be either str or list by @angloyna in #892
• Nir's actionnetwork changes by @NirTatcher in #900
• Fix airtable.insert_records table arg by @codygordon in #907
• Add canales s3 functions by @sharinetmc in #885
• Enhancement: Action Network Connector: Added unpack_statistics param in get_messages method by @matthewkrausse in #917
• Adding rename_columns method to Parsons Table by @mkrausse-ggtx in #923
• Add http response to update_mailer by @sjwmoveon in #924
• Enable passing arbitrary additional fields to NGPVAN people match API by @austinweisgrau in #916
• Install dependencies for tests based on package install by @austinweisgrau in #951

New Contributors

Cheers to our newest contributors! 🎉 Thanks so much for your help.

• @NirTatcher made their first contribution in #900
• @matthewkrausse made their first contribution in #917
• @dexchan made their first contribution in #920

Full Changelog: v2.1.0...v3.0.0

v2.1.0

What's Changed

Documentation

• Update example VAN apply_response JSON to reflect accurate keys by @ydamit in #884

Other Changes

• Action Builder Entity Record Insert Name Key Hotfix by @ydamit in #887
• Add json as allowable file type to copy_s3 by @KasiaHinkson in #844
• Enable use of "extension" argument to Redshift unload by @austinweisgrau in #871
• Update docs to point to website by @shaunagm in #851
• update zoom docs by @sharinetmc in #888
• Add more ActionKit methods by @crayolakat in #870
• fix: handle 429 error code by @talevy42 in #877
• chore(actionnetwork): Edge case and cleaning up type checking by @talevy42 in #889
• Action Builder Remove Tag Method by @ydamit in #890
• Revert "Enable passing identifiers to ActionNetwork upsert_person() by @austinweisgrau in #876
• Update release by @KasiaHinkson in #894

New Contributors

• @KasiaHinkson made their first contribution in #844

Full Changelog: v2.0.0...v2.1.0

v2.0.0

What's Changed

This release addresses a breaking change to the Zoom connector. Starting on September 1st, 2023 JWT authentication will no longer be accepted when querying Zoom's REST API; this release moves the Zoom connector a server-to-server OAuth based authentication.

To query the Zoom API moving forward, users will need to:

• Setup an OAuth application in Zoom's app marketplace, including defining relevant endpoint scopes (meetings, users, etc.)
• Use the client ID, account ID, and account secret values to validate API access

Other Changes

• add release notes yml by @sharinetmc in #878
• Prepping for Major Release by @IanRFerguson in #881
• v2.0.0 by @IanRFerguson in #882

Full Changelog: v1.2.0...v2.0.0

v1.2.0 - New connectors, updates to testing, updates on our dependencies, and many more!

New Features

• Add upsert_user function by @talevy42 in #828
• add exists_ok param to add_column by @talevy42 in #839
• Add support for deleting person from NGPVAN by @sjwmoveon in #834
• feat: add Nation Builder Connector by @gcollazo in #837
• Implement delete actionfield for ActionKit by @crayolakat in #846
• Add nation builder connector by @gcollazo in 848
• Implement, test, and document Parsons Table deduplicate method by @jafayer in #842
• Action Builder connector by @ydamit in #826
• Feature/background processing by @talevy42 in #853
• NewMode - Add bulk targets endpoint by @IanRFerguson in #857
• Get and Update ActionKit orderrecurring by @technicalex in #859
• Enable passing identifiers to ActionNetwork upsert_person() by @austinweisgrau in #861
• Add Events Helpers to PDI Connector by @IanRFerguson in #865

Automated Testing

• Add multiple python versions to CI tests by @shaunagm in #858
• Add pre-commit hook config to run flake8 and black on commit by @austinweisgrau in #864

Dependencies

• Update Twilio Package Version by @ydamit in #821
• Bump grpcio from 1.51.1 to 1.53.0 by @dependabot in #852

Others

• Bump requests from 2.25.1 to 2.31.0 by @dependabot in #824

New Contributors

• @gcollazo, @jafayer, @technicalex

v1.1.0 - Adding Black formatting, some connector features, and lots of bug fixes

New Features

• Enable __len__ dunder method for parsons Table by @austinweisgrau in #830
• Add NGPVAN bulk apply suppressions function by @codygordon @crayolakat and @sjwmoveon in #823
• Catch File Extensions in S3 Prefix @IanRFerguson in #809
• Add Redash methods by @crayolakat in #802
• MoveOn Shopify / AK changes by @codygordon in #801

Automated Testing

• Use black formatting in addition to flake8 by @austinweisgrau in #796

Bug Fixes

• Remove hardcoded version number from conf.py and correct repo link in setup.py by @SorenSpicknail in #811
• Dependency cascade added to Redshift upsert by @elyse-weiss in #833 (see also #819 & #832)
• chore(Geocoder)L Add more informative error handling to geocode #729 by @talevy42 in #810
• Fix invalid setup.py syntax by @crayolakat in #831
• Merge main into major-release by @Jason94 in #814
• Fix Salesforce query output by @sjwmoveon in #829
• Specify version of slackclient in setup.py by @crayolakat in #825
• Pass use_env_token to to_s3_csv by @codygordon in #822
• Add Default Parameter Flexibility @IanRFerguson #807
• Standardize S3 Prefix Conventions @IanRFerguson #803

Documentation

• Delete docs for non-existing arg in MySql by @Jason94 in #813

New Contributors

• @austinweisgrau made their first contribution by adding Black formatting in #796
• @codygordon made their first contribution with updates to Shopify and ActionKit connectors in #801
• @IanRFerguson made their first contribution by standardizing S3 Prefix Conventions in #803
• @talevy42 made their first contribution by adding more informative error handling to geocode in #810

Welcome and thank you to Austin, Cody, Ian and Tal! 🎉🎉🎉

v1.0.0 - Major Release

New Connectors

• feat: scytl connector by @agreenspan24 in #737
• Donorbox connector by @shaunagm in #773

New Features

• Add use_env_token option to S3 object by @sjwmoveon in #745
• Updates to ActionKit, Braintree, and S3 connectors by @crayolakat in #754
• Add get_aliases() function to Google Admin connector by @crayolakat in #769
• merge contacts and test by @mkwoods927 in #776
• Add printed list functionality to VAN connector by @sharinef1 in #778
• Updates to ActionKit, S3, and NGPVan by @crayolakat in #775
• Opt out phones in EA use case and sample script by @mkwoods927 in #670

Automated Testing

• Unit tests for #685 by @alxmrs in #746
• Update cache keys for CircleCI configuration by @SorenSpicknall in #752
• Update CircleCI Docs key fingerprint by @neverett in #785

Bug Fixes

• Fix assert_matching_tables by @crayolakat in #759
• Scytl fix browser headers and requests lib w mock by @agreenspan24 in #762
• GitHub Connector Fix by @SorenSpicknall in #767
• NGPVAN: Saved List Overwrite Bug Fix by @jburchard in #770
• Upgrade dependencies by @crayolakat in #779
• Change release version to 1.0 by @Jason94 in #793
• Fix #780 and #781 - Google Sheets documentation by @ethyoo in #786

Documentation

• Typo fixes for ETL best practices guide. by @alxmrs in #747
• Fix #707 - Parsing of code blocks for Mobilize America documentation by @ethyoo in #749
• Simple docs for the SMTP connector by @AndrewRook in #735
• Update Code of Conduct contact email by @shaunagm in #756
• Update requirements.txt for joblib security vulnerability by @SorenSpicknall in #764
• Add docker image references, logo, scytl connector in sidebar by @shaunagm in #782

New Contributors

• @alxmrs made their first contribution in #747
• @ethyoo made their first contribution in #749
• @agreenspan24 made their first contribution in #737
• @sharinef1 made their first contribution in #778

Full Changelog: v0.21.0...v1.0.0

v0.21.0 - SmartMatch, psycopg2-binary installation fix, and lots of small bugfixes

New Connectors

• TargetSmart SmartMatch connector by @benstroud in #674

Enhancements

• More flexible importing in parsons/init.py by @cmc333333 in #702
• Sendmail connector is now an abstract base class by @AndrewRook in #717
• Convert None to "" in convert_columns_to_str by @Jason94 in #728
• Add upsert_person method to ActionNetwork by @shaunagm in #734
• Allow Subfolders for Temp S3 Bucket in Redshift Operations by @SorenSpicknall in #738
• Always use UTF-8 encoding writing to S3 via the Redshift connector by @Jason94 in #732

Fixes/Improvements

• Remove leading and trailing whitespace from table and schema name in Redshift utility method table_exists_with_connection() by @neverett in #715
• Fix Redshift column alteration bug by @SorenSpicknall in #709
• Fix small bug with postal addresses in Action Network add_person method by @shaunagm in #716
• Bugfix for Redshift upsert alter_varchar_columns cascade issue by @SorenSpicknall in #720
• Fixes for Sendmail tests linting by @SorenSpicknall in #721
• Don't auto-grab AWS session token if it's not passed in by @sjwmoveon in #740
• Update Mobilize connector by @Jason94 in #733
• Update psycopg2-binary from 2.8.5 to 2.9.3 by @shaunagm in #731

Documentation

• Update version number for 0.21.0 release by @SorenSpicknall in #743

New Contributors

• @benstroud made their first contribution in #674
• @sjwmoveon made their first contribution in #740

Full Changelog: v0.20.0...v0.21.0

v0.20.0 - New Auth0 Connector, Lots of Bugfixes, and Dependency Updates

New Connectors

• Auth0 connector by @crayolakat in #693

Enhancements

• Add phone to NGP VAN apply canvass result by @bzupnick in #705
• Add tests for SendMail notification connector by @AndrewRook in #703
• Re-name Phone2Action to Capitol Canary by @bzupnick in #687
• Update simple-salesforce version by @shaunagm in #682
• Update auth info for Google Sheets by @shaunagm in #677
• Update parakimo to latest by @bzupnick in #688
• Bump pyjwt from 2.0.1 to 2.4.0 by @dependabot in #692
• Filter expand_fields before stringifying in VAN get_person calls by @shaunagm in #700

Fixes/Improvements

• Updated Mobilize connector URI, fix display of errors, fix auth issue by @shaunagm in #708
• Fix error in VAN Target Export method by @cmc333333 in #704
• Fix Sendmail pep8 Compliance by @SorenSpicknall in #711
• Update docker image version referenced in testing by @SorenSpicknall in #712
• Fix BigQuery NoneType Mapping Bug by @scmora in #695
• Add missing check to distinguish between path string and json string in Google connector utility by @shaunagm in #701
• Fix issue templates by @shaunagm in #697
• Small fixes for VAN get_code and apply_code methods by @jburchard in #696

Documentation

• Update docs to account for wheels by @cmc333333 in #698
• Add materialization methods to docs by @AndrewRook in #694
• Add 'Intro to ETL' training guide by @shaunagm in #680

New Contributors

• @AndrewRook made their first contribution in #694
• @scmora made their first contribution in #695

Full Changelog: v0.19.0...v0.20.0