Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New connector for working with the Catalist Match API #912

Merged
merged 10 commits into from
Nov 14, 2023

Conversation

austinweisgrau
Copy link
Collaborator

@austinweisgrau austinweisgrau commented Oct 26, 2023

Connector for working with the Catalist Match API.

The Catalist Match tool requires OAuth2.0 client credentials for the API as well as
credentials for accessing the Catalist sftp bucket. Each Catalist client is given
their own bucket alias named after a tree species, used for constructing the
filepath within the sftp bucket.

Accessing the Catalist sftp bucket and Match API both require the source IP address
to be explicitly white-listed by Catalist.

Example usage:

tbl = Table.from_csv(...)
client = CatalistMatch(...)
match_result = client.match(tbl)

Tests

Tests are included in this PR.

Changes to parsons

Two changes are made to the parsons package outside of this new connector.

28949a9 makes a small change to the api_connector error handler to look for error messages in a response text attribute, where the error message is found in responses from the Catalist API.

1ffb046 adds a development dependency pytest-mock to support the ergonomic use of mocking in pytest. This dependency is then used in the tests on the catalist match connector in 31f3a18.

Expected future changes

Following a community development session, JSON responses from the API will be parsed using python dataclasses. Currently JSON responses are returned raw, which matches the normal pattern across parsons connectors.

@austinweisgrau
Copy link
Collaborator Author

austinweisgrau commented Oct 26, 2023

Working on updates to type hints to be backwards compatible with Python versions back to 3.8

@austinweisgrau
Copy link
Collaborator Author

@IanRFerguson or @shaunagm Do you know what's up with that docker failure? When I hit details I just get {"error": "Our service is temporarily unavailable. We'll be back soon"}

@austinweisgrau
Copy link
Collaborator Author

@IanRFerguson or @shaunagm Do you know what's up with that docker failure? When I hit details I just get {"error": "Our service is temporarily unavailable. We'll be back soon"}

NVM I got it with 001b527

Copy link
Contributor

@IanRFerguson IanRFerguson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks excellent overall - you crushed this! I have a few comments throughout but nothing structural or functional, the code is super legible and easy to navigate.

One note - I think adding a docs/catalist.rst file is required before merging. I left a comment on setup / obtaining client creds and I reckon the docs would be a good place to outline that

parsons/catalist/catalist.py Outdated Show resolved Hide resolved
parsons/catalist/catalist.py Show resolved Hide resolved
parsons/catalist/catalist.py Outdated Show resolved Hide resolved
parsons/catalist/catalist.py Show resolved Hide resolved
parsons/catalist/catalist.py Show resolved Hide resolved
parsons/catalist/catalist.py Outdated Show resolved Hide resolved
parsons/catalist/catalist.py Show resolved Hide resolved
@austinweisgrau
Copy link
Collaborator Author

Looks excellent overall - you crushed this! I have a few comments throughout but nothing structural or functional, the code is super legible and easy to navigate.

One note - I think adding a docs/catalist.rst file is required before merging. I left a comment on setup / obtaining client creds and I reckon the docs would be a good place to outline that

Added the docs/catalist.rst file with 9b73523, that was new for me though so not totally sure I did that correctly.

Copy link
Contributor

@IanRFerguson IanRFerguson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the comments so thoroughly! Looks great

@@ -47,6 +47,7 @@ black==22.12.0
testfixtures==6.18.5
pytest==7.1.1
pytest-datadir==1.3.0
pytest-mock>=3.0.0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this used in the tests? I see putest and requests-mock being imported but not pytest-mock

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It creates a pytest fixture called "mocker", which is used in the pytest fixtures in this file

@shaunagm shaunagm merged commit 97bc5c3 into move-coop:main Nov 14, 2023
6 checks passed
IanRFerguson added a commit that referenced this pull request Dec 1, 2023
* Update release (#894)

* Zoom Polls (#886)

* Merge main into major-release (#814)

* Use black formatting in addition to flake8 (#796)

* Run black formatter on entire repository

* Update requirements.txt and CONTRIBUTING.md to reflect black format

* Use black linting in circleci test job

* Use longer variable name to resolve flake8 E741

* Move noqa comments back to proper lines after black reformat

* Standardize S3 Prefix Conventions (#803)

This PR catches exception errors when a user does not exhaustive access to keys in an S3 bucket

* Add Default Parameter Flexibility (#807)

Skips over new `/` logic checks if prefix is `None` (which is true by default)

* MoveOn Shopify / AK changes (#801)

* Add access_token authentication option for Shopify

* Remove unnecessary check
The access token will either be None or explicitly set; don't worry about an empty string.

* Add get_orders function and test

* Add get_transactions function and test

* Add function and test to get order

* style fixes

* style fixes

---------

Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>

* Catch File Extensions in S3 Prefix (#809)

* add exception handling

* Shortened logs for flake8

* add logic for default case

* added file logic + note to user

* restructured prefix logic

This change moves the prefix -> prefix/ logic into a try/except block ... this will be more robust to most use cases, while adding flexibility that we desire for split-permission buckets

* drop nested try/catch + add verbose error log

* Add error message verbosity

Co-authored-by: willyraedy <wraedy@gmail.com>

---------

Co-authored-by: willyraedy <wraedy@gmail.com>

---------

Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: Ian <47256454+IanRFerguson@users.noreply.github.com>
Co-authored-by: Cody Gordon <13374656+codygordon@users.noreply.github.com>
Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>
Co-authored-by: willyraedy <wraedy@gmail.com>

* black format

* black format

* jwt -> s2s oauth

* scaffold new functions

* add docs

* return

* DatabaseConnector Interface to Major Release (#815)

* Create the DatabaseConnector

* Implement DatabaseConnector for the DB connectors

* Add DatabaseConnector to std imports

* Flake8 fix

* Remove reference to padding in copy()

* Add database_discover and fix inheritance

* Remove strict_length from copy()

* Put strict_length back in original order

* Remove strict_length stub from BQ

* Fix discover_database export statement

* Add return annotation to mysql table_exists

* Black formatter pass

* Add more documentation on when you should use

* Add developer notes.

* Fix code block documentation

* Enhance discover database

* Add unit tests for discover database

* Fix unit tests

* Add two more tests

* Reverse Postgres string_length change

---------

Co-authored-by: Jason Walker <jason@indivisible.org>

* add type handling

* pass in updated params

* move access token function

* ok let's rock!!

* make changes

* pass access token key only

* use temporary client to gen token

* mock request in constructor

* drop unused imports

* add changes

* scaffolding tests

* Add multiple python versions to CI tests (#858)

* Add multiple python versions to CI tests

* Remove duplicate key

* Combine CI jobs

* Update ubuntu image and actually install Python versions

* Replace pyenv with apt-get to install python versions

* Remove sudo

* Remove get from 'apt-get'

* Update apt before attempting to install

* Add ppa/deadsnakes repository

* Add prereq

* Fix typo

* Add -y to install command

* Move -y to correct spot

* Add more -ys

* Add some echoes to debug

* Switch back to pyenv approach

* Remove tests from circleci config and move to new github actions config

Note: no caching yet, this is more of a proof of concept

* Split out Mac tests into seaparate file

* Set testing environmental variable separately

* First attempt to add depdendency cache

* Remove windows tests for now

* Fix circleci config

* Fix circleci for real this time

* Add tests on merging of PRs and update readme to show we do not support for Python 3.7

* Enable passing `identifiers` to ActionNetwork `upsert_person()` (#861)

* Enable passing `identifiers` to ActionNetwork upsert_person

* Remove unused arguments from method

self.get_page method doesn't exist and that method call doesn't return
anything. The return statement works fine as-is to return all tags and
handles pagination on its own.

* Include deprecated per_page argument for backwards compatibility

Emit a deprecation warning if this argument is used

* Include examples in docstring for `identifiers` argument

* Expand documentation on ActionNetwork identifiers

* Add pre-commit hook config to run flake8 and black on commit (#864)

Notes added to README on how to install and set up

* black format

* black format

* jwt -> s2s oauth

* scaffold new functions

* add docs

* return

* add type handling

* pass in updated params

* move access token function

* ok let's rock!!

* make changes

* pass access token key only

* use temporary client to gen token

* mock request in constructor

* drop unused imports

* add changes

* scaffolding tests

* write unit tests

* added testing

* drop typing (for now)

* update docstring typing

* add tests

* write functions

* update typing

* add poll results

* update table output

* fix tests

* uhhh run it back

* add scope requirements

* add to docs

We can add more here if folks see fit

* one for the money two for the show

---------

Co-authored-by: Jason <JasonW94@gmail.com>
Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: Cody Gordon <13374656+codygordon@users.noreply.github.com>
Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>
Co-authored-by: willyraedy <wraedy@gmail.com>
Co-authored-by: Jason Walker <jason@indivisible.org>
Co-authored-by: Shauna <shaunagm@gmail.com>

* Check for empty tables in zoom poll results (#897)

Co-authored-by: Jason Walker <jason@indivisible.org>

* Bump urllib3 from 1.26.5 to 1.26.17 (#901)

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.5 to 1.26.17.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](urllib3/urllib3@1.26.5...1.26.17)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add MobileCommons Connector (#896)

* mobilecommons class

* Update __init__.py

* get broadcasts

* fix get broadcast request

* Add mc_get_request method

* Add annotation

* Incorporate Daniel's suggestions and finish up get_broadcasts

* A few more methods

Need to figure out page_count issue

* small fix

* Remove page_count, use page record num instead

* Add in page_count again

Not all get responses include num param, but do include page_count. wft

* Fix logging numbers

* Add create_profile

* Fix error message for post request

* Start tests

* Add some tests

* Continue testing

* Update test_mobilecommons.py

* functionalize status_code check

* break out parse_get_request function

* fix test data

* fix documentation typo

* Add several tests

* Update mobilecommons.py

* Fix limit and pagination logic

* debug unit testing

* better commenting and logic

* Documentation

* Add MC to init file

* Revert "Merge branch 'main' into cormac-mobilecommons-connector"

This reverts commit cad250f, reversing
changes made to 493e117.

* Revert "Add MC to init file"

This reverts commit 493e117.

* Revert "Revert "Add MC to init file""

This reverts commit 8f87ec2.

* Revert "Revert "Merge branch 'main' into cormac-mobilecommons-connector""

This reverts commit 8190052.

* Fix init destruction

* fix init yet again

* Update testing docs with underscores

* Lint

* Lint tests

* break up long responses

* Fix more linting issues

* Hopefully last linting issue

* DGJKSNCHIVBN

* Documentation fixes

* Remove note to self

* date format

* remove random notes

* Update test_mobilecommons.py

---------

Co-authored-by: sharinetmc <128429303+sharinetmc@users.noreply.github.com>

* #741 : Deprecate Slack chat.postMessage `as_user` argument and allow for new authorship arguments (#891)

* remove the argument and add a warning that the usage is deprecated

* remove usage of as_user from sample code

* add in the user customization arguments in lieu of the deprecated as_user argument

* add comment regarding the permissions required to use these arguments

* use kwargs

* surface the whole response

* allow usage of the deprecated argument but surface the failed response better

* add to retry

* delete test file

* fix linting

* formatting to fix tests

* fix if style

* add warning for using thread_ts

* move the documentation to the optional arguments

* #816 Airtable.get_records() fields argument can be either str or list (#892)

* all fields to be a str object

* remove newline

* Nir's actionnetwork changes (#900)

* working on adding a functio to an and took care of a lint issues

* init

* working on all get functions

* actionnetwork functions batch 1 is ready

* linting and black formatted compliance

* removed unwanted/unsused lines

* merged updated main

* did some linting

* added some more get functions to support all ActionNetwork objects (Advocacy Campaigns, Attendances, Campaigns, Custom Fields, Donations, Embeds, Event Campaigns, Events, Forms, Fundraising Pages, Items, Lists, Messages, Metadata, Outreaches, People, Petitions, Queries, Signatures, Submissions, Tags, Taggings, Wrappers)

* worked on linting again

* fix airtable.insert_records table arg (#907)

* Add canales s3 functions (#885)

* add raw s3 functions to parsons

* add selected functions to s3.py

* delte redundant functions and move drop_and_save function to redshift.py

* create test file

* add s3 unit tests

* add rs.drop_and_unload unit test

* add printing for debugging

* remove testing file

* unsaved changes

* remove unused packages

* remove unneeded module

* Bump urllib3 from 1.26.17 to 1.26.18 (#904)

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.17 to 1.26.18.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](urllib3/urllib3@1.26.17...1.26.18)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: sharinetmc <128429303+sharinetmc@users.noreply.github.com>

* New connector for working with the Catalist Match API (#912)

* Enable api_connector to return error message in `text` attribute

Some API error responses contain the error message in the `text`
attribute, so this update makes it possible to fetch that message if
it exists.

* New connector to work with the Catalist Match API

* Add pytest-mock to requirements to support mocking in pytests

* Tests on the catalist match connector

* More open ended pytest-mock version for compatibility

* Expand docstring documetation based on feedback in PR

* More verbose error on match failure

* Parameterize template_id variable

* Expand docstrings on initial setup

* Include Catalist documentation rst file

* Enhancement: Action Network Connector: Added unpack_statistics param in get_messages method (#917)

* Adds parameter to get_messages

This adds the ability to unpack the statistics which are returned as a nested dictionary in the response.

* added unpack_statistics to an.get_messages()

* added parameters to get_messages and built tests

* changes unpack_statistics to False by default.

* added tbl variable

* formatted with black

* fixed docs

---------

Co-authored-by: mattkrausse <106627640+mattkrausse@users.noreply.github.com>

* Adding rename_columns method to Parsons Table (#923)

* added rename_columns for multiple cols

* linted

* added clarification to docs about dict structure

* updated docs

---------

Co-authored-by: mattkrausse <106627640+mattkrausse@users.noreply.github.com>

* Add http response to update_mailer (#924)

Without returning the response, or at least the status code, it's impossible to check for errors.

* Enable passing arbitrary additional fields to NGPVAN person match API (#916)

* match gcs api to s3

* wip

* two different functions

* use csv as default

* drop unused var

* add docs

* use temp file

* add comments

* wip

* add docs + replicate in gzip

* boy howdy!

* set timeout

* Revert "Merge branch 'main' into ianferguson/gcs-pathing"

This reverts commit 5b1ef6e, reversing
changes made to f0eb3d6.

* black format

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Kasia Hinkson <52927664+KasiaHinkson@users.noreply.github.com>
Co-authored-by: Jason <JasonW94@gmail.com>
Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: Cody Gordon <13374656+codygordon@users.noreply.github.com>
Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>
Co-authored-by: willyraedy <wraedy@gmail.com>
Co-authored-by: Jason Walker <jason@indivisible.org>
Co-authored-by: Shauna <shaunagm@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Cormac Martinez del Rio <66973815+cmdelrio@users.noreply.github.com>
Co-authored-by: sharinetmc <128429303+sharinetmc@users.noreply.github.com>
Co-authored-by: Angela Gloyna <angela.gloyna@gmail.com>
Co-authored-by: NirTatcher <75395024+NirTatcher@users.noreply.github.com>
Co-authored-by: justicehaze <69082853+justicehaze@users.noreply.github.com>
Co-authored-by: mattkrausse <106627640+mattkrausse@users.noreply.github.com>
Co-authored-by: mkrausse-ggtx <131683556+mkrausse-ggtx@users.noreply.github.com>
Co-authored-by: Sophie Waldman <62553142+sjwmoveon@users.noreply.github.com>
IanRFerguson added a commit that referenced this pull request Dec 1, 2023
* Update release (#894)

* Zoom Polls (#886)

* Merge main into major-release (#814)

* Use black formatting in addition to flake8 (#796)

* Run black formatter on entire repository

* Update requirements.txt and CONTRIBUTING.md to reflect black format

* Use black linting in circleci test job

* Use longer variable name to resolve flake8 E741

* Move noqa comments back to proper lines after black reformat

* Standardize S3 Prefix Conventions (#803)

This PR catches exception errors when a user does not exhaustive access to keys in an S3 bucket

* Add Default Parameter Flexibility (#807)

Skips over new `/` logic checks if prefix is `None` (which is true by default)

* MoveOn Shopify / AK changes (#801)

* Add access_token authentication option for Shopify

* Remove unnecessary check
The access token will either be None or explicitly set; don't worry about an empty string.

* Add get_orders function and test

* Add get_transactions function and test

* Add function and test to get order

* style fixes

* style fixes

---------

Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>

* Catch File Extensions in S3 Prefix (#809)

* add exception handling

* Shortened logs for flake8

* add logic for default case

* added file logic + note to user

* restructured prefix logic

This change moves the prefix -> prefix/ logic into a try/except block ... this will be more robust to most use cases, while adding flexibility that we desire for split-permission buckets

* drop nested try/catch + add verbose error log

* Add error message verbosity

Co-authored-by: willyraedy <wraedy@gmail.com>

---------

Co-authored-by: willyraedy <wraedy@gmail.com>

---------

Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: Ian <47256454+IanRFerguson@users.noreply.github.com>
Co-authored-by: Cody Gordon <13374656+codygordon@users.noreply.github.com>
Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>
Co-authored-by: willyraedy <wraedy@gmail.com>

* black format

* black format

* jwt -> s2s oauth

* scaffold new functions

* add docs

* return

* DatabaseConnector Interface to Major Release (#815)

* Create the DatabaseConnector

* Implement DatabaseConnector for the DB connectors

* Add DatabaseConnector to std imports

* Flake8 fix

* Remove reference to padding in copy()

* Add database_discover and fix inheritance

* Remove strict_length from copy()

* Put strict_length back in original order

* Remove strict_length stub from BQ

* Fix discover_database export statement

* Add return annotation to mysql table_exists

* Black formatter pass

* Add more documentation on when you should use

* Add developer notes.

* Fix code block documentation

* Enhance discover database

* Add unit tests for discover database

* Fix unit tests

* Add two more tests

* Reverse Postgres string_length change

---------

Co-authored-by: Jason Walker <jason@indivisible.org>

* add type handling

* pass in updated params

* move access token function

* ok let's rock!!

* make changes

* pass access token key only

* use temporary client to gen token

* mock request in constructor

* drop unused imports

* add changes

* scaffolding tests

* Add multiple python versions to CI tests (#858)

* Add multiple python versions to CI tests

* Remove duplicate key

* Combine CI jobs

* Update ubuntu image and actually install Python versions

* Replace pyenv with apt-get to install python versions

* Remove sudo

* Remove get from 'apt-get'

* Update apt before attempting to install

* Add ppa/deadsnakes repository

* Add prereq

* Fix typo

* Add -y to install command

* Move -y to correct spot

* Add more -ys

* Add some echoes to debug

* Switch back to pyenv approach

* Remove tests from circleci config and move to new github actions config

Note: no caching yet, this is more of a proof of concept

* Split out Mac tests into seaparate file

* Set testing environmental variable separately

* First attempt to add depdendency cache

* Remove windows tests for now

* Fix circleci config

* Fix circleci for real this time

* Add tests on merging of PRs and update readme to show we do not support for Python 3.7

* Enable passing `identifiers` to ActionNetwork `upsert_person()` (#861)

* Enable passing `identifiers` to ActionNetwork upsert_person

* Remove unused arguments from method

self.get_page method doesn't exist and that method call doesn't return
anything. The return statement works fine as-is to return all tags and
handles pagination on its own.

* Include deprecated per_page argument for backwards compatibility

Emit a deprecation warning if this argument is used

* Include examples in docstring for `identifiers` argument

* Expand documentation on ActionNetwork identifiers

* Add pre-commit hook config to run flake8 and black on commit (#864)

Notes added to README on how to install and set up

* black format

* black format

* jwt -> s2s oauth

* scaffold new functions

* add docs

* return

* add type handling

* pass in updated params

* move access token function

* ok let's rock!!

* make changes

* pass access token key only

* use temporary client to gen token

* mock request in constructor

* drop unused imports

* add changes

* scaffolding tests

* write unit tests

* added testing

* drop typing (for now)

* update docstring typing

* add tests

* write functions

* update typing

* add poll results

* update table output

* fix tests

* uhhh run it back

* add scope requirements

* add to docs

We can add more here if folks see fit

* one for the money two for the show

---------

Co-authored-by: Jason <JasonW94@gmail.com>
Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: Cody Gordon <13374656+codygordon@users.noreply.github.com>
Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>
Co-authored-by: willyraedy <wraedy@gmail.com>
Co-authored-by: Jason Walker <jason@indivisible.org>
Co-authored-by: Shauna <shaunagm@gmail.com>

* Check for empty tables in zoom poll results (#897)

Co-authored-by: Jason Walker <jason@indivisible.org>

* Bump urllib3 from 1.26.5 to 1.26.17 (#901)

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.5 to 1.26.17.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](urllib3/urllib3@1.26.5...1.26.17)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add MobileCommons Connector (#896)

* mobilecommons class

* Update __init__.py

* get broadcasts

* fix get broadcast request

* Add mc_get_request method

* Add annotation

* Incorporate Daniel's suggestions and finish up get_broadcasts

* A few more methods

Need to figure out page_count issue

* small fix

* Remove page_count, use page record num instead

* Add in page_count again

Not all get responses include num param, but do include page_count. wft

* Fix logging numbers

* Add create_profile

* Fix error message for post request

* Start tests

* Add some tests

* Continue testing

* Update test_mobilecommons.py

* functionalize status_code check

* break out parse_get_request function

* fix test data

* fix documentation typo

* Add several tests

* Update mobilecommons.py

* Fix limit and pagination logic

* debug unit testing

* better commenting and logic

* Documentation

* Add MC to init file

* Revert "Merge branch 'main' into cormac-mobilecommons-connector"

This reverts commit cad250f, reversing
changes made to 493e117.

* Revert "Add MC to init file"

This reverts commit 493e117.

* Revert "Revert "Add MC to init file""

This reverts commit 8f87ec2.

* Revert "Revert "Merge branch 'main' into cormac-mobilecommons-connector""

This reverts commit 8190052.

* Fix init destruction

* fix init yet again

* Update testing docs with underscores

* Lint

* Lint tests

* break up long responses

* Fix more linting issues

* Hopefully last linting issue

* DGJKSNCHIVBN

* Documentation fixes

* Remove note to self

* date format

* remove random notes

* Update test_mobilecommons.py

---------

Co-authored-by: sharinetmc <128429303+sharinetmc@users.noreply.github.com>

* #741 : Deprecate Slack chat.postMessage `as_user` argument and allow for new authorship arguments (#891)

* remove the argument and add a warning that the usage is deprecated

* remove usage of as_user from sample code

* add in the user customization arguments in lieu of the deprecated as_user argument

* add comment regarding the permissions required to use these arguments

* use kwargs

* surface the whole response

* allow usage of the deprecated argument but surface the failed response better

* add to retry

* delete test file

* fix linting

* formatting to fix tests

* fix if style

* add warning for using thread_ts

* move the documentation to the optional arguments

* #816 Airtable.get_records() fields argument can be either str or list (#892)

* all fields to be a str object

* remove newline

* Nir's actionnetwork changes (#900)

* working on adding a functio to an and took care of a lint issues

* init

* working on all get functions

* actionnetwork functions batch 1 is ready

* linting and black formatted compliance

* removed unwanted/unsused lines

* merged updated main

* did some linting

* added some more get functions to support all ActionNetwork objects (Advocacy Campaigns, Attendances, Campaigns, Custom Fields, Donations, Embeds, Event Campaigns, Events, Forms, Fundraising Pages, Items, Lists, Messages, Metadata, Outreaches, People, Petitions, Queries, Signatures, Submissions, Tags, Taggings, Wrappers)

* worked on linting again

* fix airtable.insert_records table arg (#907)

* Add canales s3 functions (#885)

* add raw s3 functions to parsons

* add selected functions to s3.py

* delte redundant functions and move drop_and_save function to redshift.py

* create test file

* add s3 unit tests

* add rs.drop_and_unload unit test

* add printing for debugging

* remove testing file

* unsaved changes

* remove unused packages

* remove unneeded module

* Bump urllib3 from 1.26.17 to 1.26.18 (#904)

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.17 to 1.26.18.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](urllib3/urllib3@1.26.17...1.26.18)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: sharinetmc <128429303+sharinetmc@users.noreply.github.com>

* New connector for working with the Catalist Match API (#912)

* Enable api_connector to return error message in `text` attribute

Some API error responses contain the error message in the `text`
attribute, so this update makes it possible to fetch that message if
it exists.

* New connector to work with the Catalist Match API

* Add pytest-mock to requirements to support mocking in pytests

* Tests on the catalist match connector

* More open ended pytest-mock version for compatibility

* Expand docstring documetation based on feedback in PR

* More verbose error on match failure

* Parameterize template_id variable

* Expand docstrings on initial setup

* Include Catalist documentation rst file

* Enhancement: Action Network Connector: Added unpack_statistics param in get_messages method (#917)

* Adds parameter to get_messages

This adds the ability to unpack the statistics which are returned as a nested dictionary in the response.

* added unpack_statistics to an.get_messages()

* added parameters to get_messages and built tests

* changes unpack_statistics to False by default.

* added tbl variable

* formatted with black

* fixed docs

---------

Co-authored-by: mattkrausse <106627640+mattkrausse@users.noreply.github.com>

* Adding rename_columns method to Parsons Table (#923)

* added rename_columns for multiple cols

* linted

* added clarification to docs about dict structure

* updated docs

---------

Co-authored-by: mattkrausse <106627640+mattkrausse@users.noreply.github.com>

* Add http response to update_mailer (#924)

Without returning the response, or at least the status code, it's impossible to check for errors.

* Enable passing arbitrary additional fields to NGPVAN person match API (#916)

* match gcs api to s3

* Revert "Merge branch 'main' into ianferguson/gcs-pathing"

This reverts commit 5b1ef6e, reversing
changes made to f0eb3d6.

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Kasia Hinkson <52927664+KasiaHinkson@users.noreply.github.com>
Co-authored-by: Jason <JasonW94@gmail.com>
Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: Cody Gordon <13374656+codygordon@users.noreply.github.com>
Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>
Co-authored-by: willyraedy <wraedy@gmail.com>
Co-authored-by: Jason Walker <jason@indivisible.org>
Co-authored-by: Shauna <shaunagm@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Cormac Martinez del Rio <66973815+cmdelrio@users.noreply.github.com>
Co-authored-by: sharinetmc <128429303+sharinetmc@users.noreply.github.com>
Co-authored-by: Angela Gloyna <angela.gloyna@gmail.com>
Co-authored-by: NirTatcher <75395024+NirTatcher@users.noreply.github.com>
Co-authored-by: justicehaze <69082853+justicehaze@users.noreply.github.com>
Co-authored-by: mattkrausse <106627640+mattkrausse@users.noreply.github.com>
Co-authored-by: mkrausse-ggtx <131683556+mkrausse-ggtx@users.noreply.github.com>
Co-authored-by: Sophie Waldman <62553142+sjwmoveon@users.noreply.github.com>
shaunagm added a commit that referenced this pull request Dec 8, 2023
* Merge main into major-release (#814)

* Use black formatting in addition to flake8 (#796)

* Run black formatter on entire repository

* Update requirements.txt and CONTRIBUTING.md to reflect black format

* Use black linting in circleci test job

* Use longer variable name to resolve flake8 E741

* Move noqa comments back to proper lines after black reformat

* Standardize S3 Prefix Conventions (#803)

This PR catches exception errors when a user does not exhaustive access to keys in an S3 bucket

* Add Default Parameter Flexibility (#807)

Skips over new `/` logic checks if prefix is `None` (which is true by default)

* MoveOn Shopify / AK changes (#801)

* Add access_token authentication option for Shopify

* Remove unnecessary check
The access token will either be None or explicitly set; don't worry about an empty string.

* Add get_orders function and test

* Add get_transactions function and test

* Add function and test to get order

* style fixes

* style fixes

---------

Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>

* Catch File Extensions in S3 Prefix (#809)

* add exception handling

* Shortened logs for flake8

* add logic for default case

* added file logic + note to user

* restructured prefix logic

This change moves the prefix -> prefix/ logic into a try/except block ... this will be more robust to most use cases, while adding flexibility that we desire for split-permission buckets

* drop nested try/catch + add verbose error log

* Add error message verbosity

Co-authored-by: willyraedy <wraedy@gmail.com>

---------

Co-authored-by: willyraedy <wraedy@gmail.com>

---------

Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: Ian <47256454+IanRFerguson@users.noreply.github.com>
Co-authored-by: Cody Gordon <13374656+codygordon@users.noreply.github.com>
Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>
Co-authored-by: willyraedy <wraedy@gmail.com>

* DatabaseConnector Interface to Major Release (#815)

* Create the DatabaseConnector

* Implement DatabaseConnector for the DB connectors

* Add DatabaseConnector to std imports

* Flake8 fix

* Remove reference to padding in copy()

* Add database_discover and fix inheritance

* Remove strict_length from copy()

* Put strict_length back in original order

* Remove strict_length stub from BQ

* Fix discover_database export statement

* Add return annotation to mysql table_exists

* Black formatter pass

* Add more documentation on when you should use

* Add developer notes.

* Fix code block documentation

* Enhance discover database

* Add unit tests for discover database

* Fix unit tests

* Add two more tests

* Reverse Postgres string_length change

---------

Co-authored-by: Jason Walker <jason@indivisible.org>

* Zoom Authentication + Polling API (#873)

* Add multiple python versions to CI tests (#858)

* Add multiple python versions to CI tests

* Remove duplicate key

* Combine CI jobs

* Update ubuntu image and actually install Python versions

* Replace pyenv with apt-get to install python versions

* Remove sudo

* Remove get from 'apt-get'

* Update apt before attempting to install

* Add ppa/deadsnakes repository

* Add prereq

* Fix typo

* Add -y to install command

* Move -y to correct spot

* Add more -ys

* Add some echoes to debug

* Switch back to pyenv approach

* Remove tests from circleci config and move to new github actions config

Note: no caching yet, this is more of a proof of concept

* Split out Mac tests into seaparate file

* Set testing environmental variable separately

* First attempt to add depdendency cache

* Remove windows tests for now

* Fix circleci config

* Fix circleci for real this time

* Add tests on merging of PRs and update readme to show we do not support for Python 3.7

* Enable passing `identifiers` to ActionNetwork `upsert_person()` (#861)

* Enable passing `identifiers` to ActionNetwork upsert_person

* Remove unused arguments from method

self.get_page method doesn't exist and that method call doesn't return
anything. The return statement works fine as-is to return all tags and
handles pagination on its own.

* Include deprecated per_page argument for backwards compatibility

Emit a deprecation warning if this argument is used

* Include examples in docstring for `identifiers` argument

* Expand documentation on ActionNetwork identifiers

* Add pre-commit hook config to run flake8 and black on commit (#864)

Notes added to README on how to install and set up

* black format

* black format

* jwt -> s2s oauth

* scaffold new functions

* add docs

* return

* add type handling

* pass in updated params

* move access token function

* ok let's rock!!

* make changes

* pass access token key only

* use temporary client to gen token

* mock request in constructor

* drop unused imports

* add changes

* scaffolding tests

* Add multiple python versions to CI tests (#858)

* Add multiple python versions to CI tests

* Remove duplicate key

* Combine CI jobs

* Update ubuntu image and actually install Python versions

* Replace pyenv with apt-get to install python versions

* Remove sudo

* Remove get from 'apt-get'

* Update apt before attempting to install

* Add ppa/deadsnakes repository

* Add prereq

* Fix typo

* Add -y to install command

* Move -y to correct spot

* Add more -ys

* Add some echoes to debug

* Switch back to pyenv approach

* Remove tests from circleci config and move to new github actions config

Note: no caching yet, this is more of a proof of concept

* Split out Mac tests into seaparate file

* Set testing environmental variable separately

* First attempt to add depdendency cache

* Remove windows tests for now

* Fix circleci config

* Fix circleci for real this time

* Add tests on merging of PRs and update readme to show we do not support for Python 3.7

* Enable passing `identifiers` to ActionNetwork `upsert_person()` (#861)

* Enable passing `identifiers` to ActionNetwork upsert_person

* Remove unused arguments from method

self.get_page method doesn't exist and that method call doesn't return
anything. The return statement works fine as-is to return all tags and
handles pagination on its own.

* Include deprecated per_page argument for backwards compatibility

Emit a deprecation warning if this argument is used

* Include examples in docstring for `identifiers` argument

* Expand documentation on ActionNetwork identifiers

* Add pre-commit hook config to run flake8 and black on commit (#864)

Notes added to README on how to install and set up

* black format

* black format

* jwt -> s2s oauth

* scaffold new functions

* add docs

* return

* add type handling

* pass in updated params

* move access token function

* ok let's rock!!

* make changes

* pass access token key only

* use temporary client to gen token

* mock request in constructor

* drop unused imports

* add changes

* scaffolding tests

* write unit tests

* drop poll endpoints for now

---------

Co-authored-by: Shauna <shaunagm@gmail.com>
Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>

* Merging Main Before Release (#880)

* Add multiple python versions to CI tests (#858)

* Add multiple python versions to CI tests

* Remove duplicate key

* Combine CI jobs

* Update ubuntu image and actually install Python versions

* Replace pyenv with apt-get to install python versions

* Remove sudo

* Remove get from 'apt-get'

* Update apt before attempting to install

* Add ppa/deadsnakes repository

* Add prereq

* Fix typo

* Add -y to install command

* Move -y to correct spot

* Add more -ys

* Add some echoes to debug

* Switch back to pyenv approach

* Remove tests from circleci config and move to new github actions config

Note: no caching yet, this is more of a proof of concept

* Split out Mac tests into seaparate file

* Set testing environmental variable separately

* First attempt to add depdendency cache

* Remove windows tests for now

* Fix circleci config

* Fix circleci for real this time

* Add tests on merging of PRs and update readme to show we do not support for Python 3.7

* Enable passing `identifiers` to ActionNetwork `upsert_person()` (#861)

* Enable passing `identifiers` to ActionNetwork upsert_person

* Remove unused arguments from method

self.get_page method doesn't exist and that method call doesn't return
anything. The return statement works fine as-is to return all tags and
handles pagination on its own.

* Include deprecated per_page argument for backwards compatibility

Emit a deprecation warning if this argument is used

* Include examples in docstring for `identifiers` argument

* Expand documentation on ActionNetwork identifiers

* Add pre-commit hook config to run flake8 and black on commit (#864)

Notes added to README on how to install and set up

* Add Events Helpers to PDI Connector (#865)

* add helpers to Events object

* stage docstring

* add docs

* linting

* fix typo + enforce validation

* add return docs

* add events tests

* use mock pdi

* jk

* mark live tests

* add alias

* drop unused imports

* change release number (#872)

* add release notes yml (#878)

---------

Co-authored-by: Shauna <shaunagm@gmail.com>
Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: sharinetmc <128429303+sharinetmc@users.noreply.github.com>

* Switch from API key to Personal Access Token (#866)

* Wraedy/bigquery db connector (#875)

* Create the DatabaseConnector

* Implement DatabaseConnector for the DB connectors

* Add DatabaseConnector to std imports

* Flake8 fix

* Remove reference to padding in copy()

* Add database_discover and fix inheritance

* Remove strict_length from copy()

* Put strict_length back in original order

* Remove strict_length stub from BQ

* Fix discover_database export statement

* Add return annotation to mysql table_exists

* Black formatter pass

* create bigquery folder in databases folde

* create query parity between bigquery and redshift

* mock up copy functionality for bigquery

* fix typo

* add duplicate function to bigquery

* move transaction to helper function

* implement upsert

* fix imports and packages

* add get tables and views methods

* add query return flexibility

* match bigquery apis with redshift

* make s3 to gcs more generic

* add transaction support to bigquery

* remove logs

* add gcs docs

* process job config in function

* finish todo's (and add one more lol)

* [ wip ] AttributeError

* add raw download param

* drop raw download

* copy from GCS docstring

* copy s3 docs

* copy docs

* docstrings

* control flow

* add source path to aws transfer spec

* add Code object to imports

* cleaning up slightly

* check status code

* nice

* pass in required param

* add pattern handling

* add quote character to LoadJobConfig

* add schema to copy from gcs

* drop dist and sortkeys

No longer input params

* add delimiter param

* use schema definition

* write column mapping helper

* pass in formatted schema to load_uri fn

* rename new file

* move file with jason's changes

* move new changes back into file to maintain history

* remove extraneous fn and move project job config

* get back to test parity

* fix bad merge conflict

* remove extra params from copy sig

* clarify transaction guidance

* clean up list blobs

* clean up storage transfer polling

* upgrade cloud storage package

* use list of schema mappings

* scaffolded big file function 😎

* add to docs

* default to compression

we can make this more flexible, just scaffolding

* add temp logging

we can drop this later just trying to get a handle on cycle time

* use decompress

* add logging

* implement unzipping and reuploading cloud file

* logging error

* Add destination path

* Small fix

* add todo's

* drop max wait time

* add kwargs to put blob

Potentially useful for metadata (content type, etc.)

* add verbosity to description

* black formatted

* add gcs to/from helpers

* write to_bigquery function

* update big file logic

* allow jagged rows logic

* test additional methods

* add duplicate table test

* test drop flag for duplicate

* basic test for upsert

* add typing

* move non-essential logs to debug

* move logs to debug

* hey, it works!

* add UUID support for bigquery type map

* add datetime to bigquery type map

* address comments

* address comments

* drop GCS class function

we can pick this up later but it doesn't currently work

* move class back to old location with new import

* revert to old name

* remove transaction error handler

* add description conditional block for s3

* change one more conditional to s3

* handle empty source paths

* reverting new import path

---------

Co-authored-by: Jason Walker <jason@indivisible.org>
Co-authored-by: Ian <47256454+IanRFerguson@users.noreply.github.com>
Co-authored-by: Kasia Hinkson <52927664+KasiaHinkson@users.noreply.github.com>

* BigQuery - Add Column Helpers (#911)

* add column outlines

* optionally log query

* flip default params

* flip back

* Google BigQuery - Clean Up Autodetect Logic (#914)

* don't delete

* clean up schema autodetect logic

* undo comments

* Update stale references to parsons.databases.bigquery (#920)

* Fix BQ references in discover_database

* Update BQ references in tofrom.py

* Update BQ refs in test_discover_database.py

* Fix gcs hidden error (#930)

* logging

* edit flake8 max line for testing

* change flake8 for testing

* comment out unsused var

* add print to check branch

* change to logging

* more logging

* try printing

* more logging

* logging:

* more printing

* more logging

* print transfer job request

* change error message

* requested changes

* remove comment

* GoogleCloudStorage - Handle zip / gzip files flexibly (#937)

* Update release (#894)

* Zoom Polls (#886)

* Merge main into major-release (#814)

* Use black formatting in addition to flake8 (#796)

* Run black formatter on entire repository

* Update requirements.txt and CONTRIBUTING.md to reflect black format

* Use black linting in circleci test job

* Use longer variable name to resolve flake8 E741

* Move noqa comments back to proper lines after black reformat

* Standardize S3 Prefix Conventions (#803)

This PR catches exception errors when a user does not exhaustive access to keys in an S3 bucket

* Add Default Parameter Flexibility (#807)

Skips over new `/` logic checks if prefix is `None` (which is true by default)

* MoveOn Shopify / AK changes (#801)

* Add access_token authentication option for Shopify

* Remove unnecessary check
The access token will either be None or explicitly set; don't worry about an empty string.

* Add get_orders function and test

* Add get_transactions function and test

* Add function and test to get order

* style fixes

* style fixes

---------

Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>

* Catch File Extensions in S3 Prefix (#809)

* add exception handling

* Shortened logs for flake8

* add logic for default case

* added file logic + note to user

* restructured prefix logic

This change moves the prefix -> prefix/ logic into a try/except block ... this will be more robust to most use cases, while adding flexibility that we desire for split-permission buckets

* drop nested try/catch + add verbose error log

* Add error message verbosity

Co-authored-by: willyraedy <wraedy@gmail.com>

---------

Co-authored-by: willyraedy <wraedy@gmail.com>

---------

Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: Ian <47256454+IanRFerguson@users.noreply.github.com>
Co-authored-by: Cody Gordon <13374656+codygordon@users.noreply.github.com>
Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>
Co-authored-by: willyraedy <wraedy@gmail.com>

* black format

* black format

* jwt -> s2s oauth

* scaffold new functions

* add docs

* return

* DatabaseConnector Interface to Major Release (#815)

* Create the DatabaseConnector

* Implement DatabaseConnector for the DB connectors

* Add DatabaseConnector to std imports

* Flake8 fix

* Remove reference to padding in copy()

* Add database_discover and fix inheritance

* Remove strict_length from copy()

* Put strict_length back in original order

* Remove strict_length stub from BQ

* Fix discover_database export statement

* Add return annotation to mysql table_exists

* Black formatter pass

* Add more documentation on when you should use

* Add developer notes.

* Fix code block documentation

* Enhance discover database

* Add unit tests for discover database

* Fix unit tests

* Add two more tests

* Reverse Postgres string_length change

---------

Co-authored-by: Jason Walker <jason@indivisible.org>

* add type handling

* pass in updated params

* move access token function

* ok let's rock!!

* make changes

* pass access token key only

* use temporary client to gen token

* mock request in constructor

* drop unused imports

* add changes

* scaffolding tests

* Add multiple python versions to CI tests (#858)

* Add multiple python versions to CI tests

* Remove duplicate key

* Combine CI jobs

* Update ubuntu image and actually install Python versions

* Replace pyenv with apt-get to install python versions

* Remove sudo

* Remove get from 'apt-get'

* Update apt before attempting to install

* Add ppa/deadsnakes repository

* Add prereq

* Fix typo

* Add -y to install command

* Move -y to correct spot

* Add more -ys

* Add some echoes to debug

* Switch back to pyenv approach

* Remove tests from circleci config and move to new github actions config

Note: no caching yet, this is more of a proof of concept

* Split out Mac tests into seaparate file

* Set testing environmental variable separately

* First attempt to add depdendency cache

* Remove windows tests for now

* Fix circleci config

* Fix circleci for real this time

* Add tests on merging of PRs and update readme to show we do not support for Python 3.7

* Enable passing `identifiers` to ActionNetwork `upsert_person()` (#861)

* Enable passing `identifiers` to ActionNetwork upsert_person

* Remove unused arguments from method

self.get_page method doesn't exist and that method call doesn't return
anything. The return statement works fine as-is to return all tags and
handles pagination on its own.

* Include deprecated per_page argument for backwards compatibility

Emit a deprecation warning if this argument is used

* Include examples in docstring for `identifiers` argument

* Expand documentation on ActionNetwork identifiers

* Add pre-commit hook config to run flake8 and black on commit (#864)

Notes added to README on how to install and set up

* black format

* black format

* jwt -> s2s oauth

* scaffold new functions

* add docs

* return

* add type handling

* pass in updated params

* move access token function

* ok let's rock!!

* make changes

* pass access token key only

* use temporary client to gen token

* mock request in constructor

* drop unused imports

* add changes

* scaffolding tests

* write unit tests

* added testing

* drop typing (for now)

* update docstring typing

* add tests

* write functions

* update typing

* add poll results

* update table output

* fix tests

* uhhh run it back

* add scope requirements

* add to docs

We can add more here if folks see fit

* one for the money two for the show

---------

Co-authored-by: Jason <JasonW94@gmail.com>
Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: Cody Gordon <13374656+codygordon@users.noreply.github.com>
Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>
Co-authored-by: willyraedy <wraedy@gmail.com>
Co-authored-by: Jason Walker <jason@indivisible.org>
Co-authored-by: Shauna <shaunagm@gmail.com>

* Check for empty tables in zoom poll results (#897)

Co-authored-by: Jason Walker <jason@indivisible.org>

* Bump urllib3 from 1.26.5 to 1.26.17 (#901)

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.5 to 1.26.17.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](urllib3/urllib3@1.26.5...1.26.17)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add MobileCommons Connector (#896)

* mobilecommons class

* Update __init__.py

* get broadcasts

* fix get broadcast request

* Add mc_get_request method

* Add annotation

* Incorporate Daniel's suggestions and finish up get_broadcasts

* A few more methods

Need to figure out page_count issue

* small fix

* Remove page_count, use page record num instead

* Add in page_count again

Not all get responses include num param, but do include page_count. wft

* Fix logging numbers

* Add create_profile

* Fix error message for post request

* Start tests

* Add some tests

* Continue testing

* Update test_mobilecommons.py

* functionalize status_code check

* break out parse_get_request function

* fix test data

* fix documentation typo

* Add several tests

* Update mobilecommons.py

* Fix limit and pagination logic

* debug unit testing

* better commenting and logic

* Documentation

* Add MC to init file

* Revert "Merge branch 'main' into cormac-mobilecommons-connector"

This reverts commit cad250f, reversing
changes made to 493e117.

* Revert "Add MC to init file"

This reverts commit 493e117.

* Revert "Revert "Add MC to init file""

This reverts commit 8f87ec2.

* Revert "Revert "Merge branch 'main' into cormac-mobilecommons-connector""

This reverts commit 8190052.

* Fix init destruction

* fix init yet again

* Update testing docs with underscores

* Lint

* Lint tests

* break up long responses

* Fix more linting issues

* Hopefully last linting issue

* DGJKSNCHIVBN

* Documentation fixes

* Remove note to self

* date format

* remove random notes

* Update test_mobilecommons.py

---------

Co-authored-by: sharinetmc <128429303+sharinetmc@users.noreply.github.com>

* #741 : Deprecate Slack chat.postMessage `as_user` argument and allow for new authorship arguments (#891)

* remove the argument and add a warning that the usage is deprecated

* remove usage of as_user from sample code

* add in the user customization arguments in lieu of the deprecated as_user argument

* add comment regarding the permissions required to use these arguments

* use kwargs

* surface the whole response

* allow usage of the deprecated argument but surface the failed response better

* add to retry

* delete test file

* fix linting

* formatting to fix tests

* fix if style

* add warning for using thread_ts

* move the documentation to the optional arguments

* #816 Airtable.get_records() fields argument can be either str or list (#892)

* all fields to be a str object

* remove newline

* Nir's actionnetwork changes (#900)

* working on adding a functio to an and took care of a lint issues

* init

* working on all get functions

* actionnetwork functions batch 1 is ready

* linting and black formatted compliance

* removed unwanted/unsused lines

* merged updated main

* did some linting

* added some more get functions to support all ActionNetwork objects (Advocacy Campaigns, Attendances, Campaigns, Custom Fields, Donations, Embeds, Event Campaigns, Events, Forms, Fundraising Pages, Items, Lists, Messages, Metadata, Outreaches, People, Petitions, Queries, Signatures, Submissions, Tags, Taggings, Wrappers)

* worked on linting again

* fix airtable.insert_records table arg (#907)

* Add canales s3 functions (#885)

* add raw s3 functions to parsons

* add selected functions to s3.py

* delte redundant functions and move drop_and_save function to redshift.py

* create test file

* add s3 unit tests

* add rs.drop_and_unload unit test

* add printing for debugging

* remove testing file

* unsaved changes

* remove unused packages

* remove unneeded module

* Bump urllib3 from 1.26.17 to 1.26.18 (#904)

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.17 to 1.26.18.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](urllib3/urllib3@1.26.17...1.26.18)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: sharinetmc <128429303+sharinetmc@users.noreply.github.com>

* New connector for working with the Catalist Match API (#912)

* Enable api_connector to return error message in `text` attribute

Some API error responses contain the error message in the `text`
attribute, so this update makes it possible to fetch that message if
it exists.

* New connector to work with the Catalist Match API

* Add pytest-mock to requirements to support mocking in pytests

* Tests on the catalist match connector

* More open ended pytest-mock version for compatibility

* Expand docstring documetation based on feedback in PR

* More verbose error on match failure

* Parameterize template_id variable

* Expand docstrings on initial setup

* Include Catalist documentation rst file

* Enhancement: Action Network Connector: Added unpack_statistics param in get_messages method (#917)

* Adds parameter to get_messages

This adds the ability to unpack the statistics which are returned as a nested dictionary in the response.

* added unpack_statistics to an.get_messages()

* added parameters to get_messages and built tests

* changes unpack_statistics to False by default.

* added tbl variable

* formatted with black

* fixed docs

---------

Co-authored-by: mattkrausse <106627640+mattkrausse@users.noreply.github.com>

* Adding rename_columns method to Parsons Table (#923)

* added rename_columns for multiple cols

* linted

* added clarification to docs about dict structure

* updated docs

---------

Co-authored-by: mattkrausse <106627640+mattkrausse@users.noreply.github.com>

* Add http response to update_mailer (#924)

Without returning the response, or at least the status code, it's impossible to check for errors.

* Enable passing arbitrary additional fields to NGPVAN person match API (#916)

* match gcs api to s3

* wip

* two different functions

* use csv as default

* drop unused var

* add docs

* use temp file

* add comments

* wip

* add docs + replicate in gzip

* boy howdy!

* set timeout

* Revert "Merge branch 'main' into ianferguson/gcs-pathing"

This reverts commit 5b1ef6e, reversing
changes made to f0eb3d6.

* black format

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Kasia Hinkson <52927664+KasiaHinkson@users.noreply.github.com>
Co-authored-by: Jason <JasonW94@gmail.com>
Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: Cody Gordon <13374656+codygordon@users.noreply.github.com>
Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>
Co-authored-by: willyraedy <wraedy@gmail.com>
Co-authored-by: Jason Walker <jason@indivisible.org>
Co-authored-by: Shauna <shaunagm@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Cormac Martinez del Rio <66973815+cmdelrio@users.noreply.github.com>
Co-authored-by: sharinetmc <128429303+sharinetmc@users.noreply.github.com>
Co-authored-by: Angela Gloyna <angela.gloyna@gmail.com>
Co-authored-by: NirTatcher <75395024+NirTatcher@users.noreply.github.com>
Co-authored-by: justicehaze <69082853+justicehaze@users.noreply.github.com>
Co-authored-by: mattkrausse <106627640+mattkrausse@users.noreply.github.com>
Co-authored-by: mkrausse-ggtx <131683556+mkrausse-ggtx@users.noreply.github.com>
Co-authored-by: Sophie Waldman <62553142+sjwmoveon@users.noreply.github.com>

* GoogleCloudStorage - Add GCS Destination Path Param (#936)

* Update release (#894)

* Zoom Polls (#886)

* Merge main into major-release (#814)

* Use black formatting in addition to flake8 (#796)

* Run black formatter on entire repository

* Update requirements.txt and CONTRIBUTING.md to reflect black format

* Use black linting in circleci test job

* Use longer variable name to resolve flake8 E741

* Move noqa comments back to proper lines after black reformat

* Standardize S3 Prefix Conventions (#803)

This PR catches exception errors when a user does not exhaustive access to keys in an S3 bucket

* Add Default Parameter Flexibility (#807)

Skips over new `/` logic checks if prefix is `None` (which is true by default)

* MoveOn Shopify / AK changes (#801)

* Add access_token authentication option for Shopify

* Remove unnecessary check
The access token will either be None or explicitly set; don't worry about an empty string.

* Add get_orders function and test

* Add get_transactions function and test

* Add function and test to get order

* style fixes

* style fixes

---------

Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>

* Catch File Extensions in S3 Prefix (#809)

* add exception handling

* Shortened logs for flake8

* add logic for default case

* added file logic + note to user

* restructured prefix logic

This change moves the prefix -> prefix/ logic into a try/except block ... this will be more robust to most use cases, while adding flexibility that we desire for split-permission buckets

* drop nested try/catch + add verbose error log

* Add error message verbosity

Co-authored-by: willyraedy <wraedy@gmail.com>

---------

Co-authored-by: willyraedy <wraedy@gmail.com>

---------

Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: Ian <47256454+IanRFerguson@users.noreply.github.com>
Co-authored-by: Cody Gordon <13374656+codygordon@users.noreply.github.com>
Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>
Co-authored-by: willyraedy <wraedy@gmail.com>

* black format

* black format

* jwt -> s2s oauth

* scaffold new functions

* add docs

* return

* DatabaseConnector Interface to Major Release (#815)

* Create the DatabaseConnector

* Implement DatabaseConnector for the DB connectors

* Add DatabaseConnector to std imports

* Flake8 fix

* Remove reference to padding in copy()

* Add database_discover and fix inheritance

* Remove strict_length from copy()

* Put strict_length back in original order

* Remove strict_length stub from BQ

* Fix discover_database export statement

* Add return annotation to mysql table_exists

* Black formatter pass

* Add more documentation on when you should use

* Add developer notes.

* Fix code block documentation

* Enhance discover database

* Add unit tests for discover database

* Fix unit tests

* Add two more tests

* Reverse Postgres string_length change

---------

Co-authored-by: Jason Walker <jason@indivisible.org>

* add type handling

* pass in updated params

* move access token function

* ok let's rock!!

* make changes

* pass access token key only

* use temporary client to gen token

* mock request in constructor

* drop unused imports

* add changes

* scaffolding tests

* Add multiple python versions to CI tests (#858)

* Add multiple python versions to CI tests

* Remove duplicate key

* Combine CI jobs

* Update ubuntu image and actually install Python versions

* Replace pyenv with apt-get to install python versions

* Remove sudo

* Remove get from 'apt-get'

* Update apt before attempting to install

* Add ppa/deadsnakes repository

* Add prereq

* Fix typo

* Add -y to install command

* Move -y to correct spot

* Add more -ys

* Add some echoes to debug

* Switch back to pyenv approach

* Remove tests from circleci config and move to new github actions config

Note: no caching yet, this is more of a proof of concept

* Split out Mac tests into seaparate file

* Set testing environmental variable separately

* First attempt to add depdendency cache

* Remove windows tests for now

* Fix circleci config

* Fix circleci for real this time

* Add tests on merging of PRs and update readme to show we do not support for Python 3.7

* Enable passing `identifiers` to ActionNetwork `upsert_person()` (#861)

* Enable passing `identifiers` to ActionNetwork upsert_person

* Remove unused arguments from method

self.get_page method doesn't exist and that method call doesn't return
anything. The return statement works fine as-is to return all tags and
handles pagination on its own.

* Include deprecated per_page argument for backwards compatibility

Emit a deprecation warning if this argument is used

* Include examples in docstring for `identifiers` argument

* Expand documentation on ActionNetwork identifiers

* Add pre-commit hook config to run flake8 and black on commit (#864)

Notes added to README on how to install and set up

* black format

* black format

* jwt -> s2s oauth

* scaffold new functions

* add docs

* return

* add type handling

* pass in updated params

* move access token function

* ok let's rock!!

* make changes

* pass access token key only

* use temporary client to gen token

* mock request in constructor

* drop unused imports

* add changes

* scaffolding tests

* write unit tests

* added testing

* drop typing (for now)

* update docstring typing

* add tests

* write functions

* update typing

* add poll results

* update table output

* fix tests

* uhhh run it back

* add scope requirements

* add to docs

We can add more here if folks see fit

* one for the money two for the show

---------

Co-authored-by: Jason <JasonW94@gmail.com>
Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: Cody Gordon <13374656+codygordon@users.noreply.github.com>
Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>
Co-authored-by: willyraedy <wraedy@gmail.com>
Co-authored-by: Jason Walker <jason@indivisible.org>
Co-authored-by: Shauna <shaunagm@gmail.com>

* Check for empty tables in zoom poll results (#897)

Co-authored-by: Jason Walker <jason@indivisible.org>

* Bump urllib3 from 1.26.5 to 1.26.17 (#901)

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.5 to 1.26.17.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](urllib3/urllib3@1.26.5...1.26.17)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add MobileCommons Connector (#896)

* mobilecommons class

* Update __init__.py

* get broadcasts

* fix get broadcast request

* Add mc_get_request method

* Add annotation

* Incorporate Daniel's suggestions and finish up get_broadcasts

* A few more methods

Need to figure out page_count issue

* small fix

* Remove page_count, use page record num instead

* Add in page_count again

Not all get responses include num param, but do include page_count. wft

* Fix logging numbers

* Add create_profile

* Fix error message for post request

* Start tests

* Add some tests

* Continue testing

* Update test_mobilecommons.py

* functionalize status_code check

* break out parse_get_request function

* fix test data

* fix documentation typo

* Add several tests

* Update mobilecommons.py

* Fix limit and pagination logic

* debug unit testing

* better commenting and logic

* Documentation

* Add MC to init file

* Revert "Merge branch 'main' into cormac-mobilecommons-connector"

This reverts commit cad250f, reversing
changes made to 493e117.

* Revert "Add MC to init file"

This reverts commit 493e117.

* Revert "Revert "Add MC to init file""

This reverts commit 8f87ec2.

* Revert "Revert "Merge branch 'main' into cormac-mobilecommons-connector""

This reverts commit 8190052.

* Fix init destruction

* fix init yet again

* Update testing docs with underscores

* Lint

* Lint tests

* break up long responses

* Fix more linting issues

* Hopefully last linting issue

* DGJKSNCHIVBN

* Documentation fixes

* Remove note to self

* date format

* remove random notes

* Update test_mobilecommons.py

---------

Co-authored-by: sharinetmc <128429303+sharinetmc@users.noreply.github.com>

* #741 : Deprecate Slack chat.postMessage `as_user` argument and allow for new authorship arguments (#891)

* remove the argument and add a warning that the usage is deprecated

* remove usage of as_user from sample code

* add in the user customization arguments in lieu of the deprecated as_user argument

* add comment regarding the permissions required to use these arguments

* use kwargs

* surface the whole response

* allow usage of the deprecated argument but surface the failed response better

* add to retry

* delete test file

* fix linting

* formatting to fix tests

* fix if style

* add warning for using thread_ts

* move the documentation to the optional arguments

* #816 Airtable.get_records() fields argument can be either str or list (#892)

* all fields to be a str object

* remove newline

* Nir's actionnetwork changes (#900)

* working on adding a functio to an and took care of a lint issues

* init

* working on all get functions

* actionnetwork functions batch 1 is ready

* linting and black formatted compliance

* removed unwanted/unsused lines

* merged updated main

* did some linting

* added some more get functions to support all ActionNetwork objects (Advocacy Campaigns, Attendances, Campaigns, Custom Fields, Donations, Embeds, Event Campaigns, Events, Forms, Fundraising Pages, Items, Lists, Messages, Metadata, Outreaches, People, Petitions, Queries, Signatures, Submissions, Tags, Taggings, Wrappers)

* worked on linting again

* fix airtable.insert_records table arg (#907)

* Add canales s3 functions (#885)

* add raw s3 functions to parsons

* add selected functions to s3.py

* delte redundant functions and move drop_and_save function to redshift.py

* create test file

* add s3 unit tests

* add rs.drop_and_unload unit test

* add printing for debugging

* remove testing file

* unsaved changes

* remove unused packages

* remove unneeded module

* Bump urllib3 from 1.26.17 to 1.26.18 (#904)

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.17 to 1.26.18.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](urllib3/urllib3@1.26.17...1.26.18)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: sharinetmc <128429303+sharinetmc@users.noreply.github.com>

* New connector for working with the Catalist Match API (#912)

* Enable api_connector to return error message in `text` attribute

Some API error responses contain the error message in the `text`
attribute, so this update makes it possible to fetch that message if
it exists.

* New connector to work with the Catalist Match API

* Add pytest-mock to requirements to support mocking in pytests

* Tests on the catalist match connector

* More open ended pytest-mock version for compatibility

* Expand docstring documetation based on feedback in PR

* More verbose error on match failure

* Parameterize template_id variable

* Expand docstrings on initial setup

* Include Catalist documentation rst file

* Enhancement: Action Network Connector: Added unpack_statistics param in get_messages method (#917)

* Adds parameter to get_messages

This adds the ability to unpack the statistics which are returned as a nested dictionary in the response.

* added unpack_statistics to an.get_messages()

* added parameters to get_messages and built tests

* changes unpack_statistics to False by default.

* added tbl variable

* formatted with black

* fixed docs

---------

Co-authored-by: mattkrausse <106627640+mattkrausse@users.noreply.github.com>

* Adding rename_columns method to Parsons Table (#923)

* added rename_columns for multiple cols

* linted

* added clarification to docs about dict structure

* updated docs

---------

Co-authored-by: mattkrausse <106627640+mattkrausse@users.noreply.github.com>

* Add http response to update_mailer (#924)

Without returning the response, or at least the status code, it's impossible to check for errors.

* Enable passing arbitrary additional fields to NGPVAN person match API (#916)

* match gcs api to s3

* Revert "Merge branch 'main' into ianferguson/gcs-pathing"

This reverts commit 5b1ef6e, reversing
changes made to f0eb3d6.

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Kasia Hinkson <52927664+KasiaHinkson@users.noreply.github.com>
Co-authored-by: Jason <JasonW94@gmail.com>
Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: Cody Gordon <13374656+codygordon@users.noreply.github.com>
Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>
Co-authored-by: willyraedy <wraedy@gmail.com>
Co-authored-by: Jason Walker <jason@indivisible.org>
Co-authored-by: Shauna <shaunagm@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Cormac Martinez del Rio <66973815+cmdelrio@users.noreply.github.com>
Co-authored-by: sharinetmc <128429303+sharinetmc@users.noreply.github.com>
Co-authored-by: Angela Gloyna <angela.gloyna@gmail.com>
Co-authored-by: NirTatcher <75395024+NirTatcher@users.noreply.github.com>
Co-authored-by: justicehaze <69082853+justicehaze@users.noreply.github.com>
Co-authored-by: mattkrausse <106627640+mattkrausse@users.noreply.github.com>
Co-authored-by: mkrausse-ggtx <131683556+mkrausse-ggtx@users.noreply.github.com>
Co-authored-by: Sophie Waldman <62553142+sjwmoveon@users.noreply.github.com>

* Bump version number to 3.0.0

* Fix import statement in test_bigquery.py

* Add Tests to major-release Branch (#949)

* add major release branch to gh workflow

* add mac tests

* null changes (want to trigger test)

* remove temp change

* Resolve GCP Test Failures For Major Release (#948)

* add full import

* resolve bigquery unzip test

* remove keyword

* fix flake8 errors

* fix linting

* push docs

* fix flake8

* too long for flake8

* Install google-cloud-storage-transfer for google extras (#946)

This is required for the import of storage_transfer to work

Co-authored-by: Ian <47256454+IanRFerguson@users.noreply.github.com>

* Revert "Enable passing `identifiers` to ActionNetwork `upsert_person()` (#861)" (#945)

This reverts commit 77ead60.

Co-authored-by: Ian <47256454+IanRFerguson@users.noreply.github.com>

* BigQuery - Add row count function to connector (#913)

* add row count function

* use sql

* add unit test

* unit test

* whoops!

* add examples (#952)

* Parse Boolean types by default (#943)

* Parse Boolean types by default

Commit 766cfae created a feature for
parsing boolean types but turned it off by default. This commit turns
that feature on by default and adds a comment about how to turn it off
and what that does.

* Fix test expectations after updating boolean parsing behavior

* Only ever interpret python bools as SQL booleans

No longer coerce by default any of the following as booleans:
"yes", "True", "t", 1, 0, "no", "False", "f"

* Fix redshift test parsing bools

* Move redshift test into test_databases folder

* Remove retired TRUE_VALS and FALSE_VALS configuration variables

We now only use python booleans

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Jason <JasonW94@gmail.com>
Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: Ian <47256454+IanRFerguson@users.noreply.github.com>
Co-authored-by: Cody Gordon <13374656+codygordon@users.noreply.github.com>
Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>
Co-authored-by: willyraedy <wraedy@gmail.com>
Co-authored-by: Jason Walker <jason@indivisible.org>
Co-authored-by: sharinetmc <128429303+sharinetmc@users.noreply.github.com>
Co-authored-by: Kathy Nguyen <kathy.nguyen@moveon.org>
Co-authored-by: Kasia Hinkson <52927664+KasiaHinkson@users.noreply.github.com>
Co-authored-by: dexchan <2642977+dexchan@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Cormac Martinez del Rio <66973815+cmdelrio@users.noreply.github.com>
Co-authored-by: Angela Gloyna <angela.gloyna@gmail.com>
Co-authored-by: NirTatcher <75395024+NirTatcher@users.noreply.github.com>
Co-authored-by: justicehaze <69082853+justicehaze@users.noreply.github.com>
Co-authored-by: mattkrausse <106627640+mattkrausse@users.noreply.github.com>
Co-authored-by: mkrausse-ggtx <131683556+mkrausse-ggtx@users.noreply.github.com>
Co-authored-by: Sophie Waldman <62553142+sjwmoveon@users.noreply.github.com>
IanRFerguson added a commit that referenced this pull request Dec 11, 2023
* Merge main into major-release (#814)

* Use black formatting in addition to flake8 (#796)

* Run black formatter on entire repository

* Update requirements.txt and CONTRIBUTING.md to reflect black format

* Use black linting in circleci test job

* Use longer variable name to resolve flake8 E741

* Move noqa comments back to proper lines after black reformat

* Standardize S3 Prefix Conventions (#803)

This PR catches exception errors when a user does not exhaustive access to keys in an S3 bucket

* Add Default Parameter Flexibility (#807)

Skips over new `/` logic checks if prefix is `None` (which is true by default)

* MoveOn Shopify / AK changes (#801)

* Add access_token authentication option for Shopify

* Remove unnecessary check
The access token will either be None or explicitly set; don't worry about an empty string.

* Add get_orders function and test

* Add get_transactions function and test

* Add function and test to get order

* style fixes

* style fixes

---------

Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>

* Catch File Extensions in S3 Prefix (#809)

* add exception handling

* Shortened logs for flake8

* add logic for default case

* added file logic + note to user

* restructured prefix logic

This change moves the prefix -> prefix/ logic into a try/except block ... this will be more robust to most use cases, while adding flexibility that we desire for split-permission buckets

* drop nested try/catch + add verbose error log

* Add error message verbosity

Co-authored-by: willyraedy <wraedy@gmail.com>

---------

Co-authored-by: willyraedy <wraedy@gmail.com>

---------

Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: Ian <47256454+IanRFerguson@users.noreply.github.com>
Co-authored-by: Cody Gordon <13374656+codygordon@users.noreply.github.com>
Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>
Co-authored-by: willyraedy <wraedy@gmail.com>

* DatabaseConnector Interface to Major Release (#815)

* Create the DatabaseConnector

* Implement DatabaseConnector for the DB connectors

* Add DatabaseConnector to std imports

* Flake8 fix

* Remove reference to padding in copy()

* Add database_discover and fix inheritance

* Remove strict_length from copy()

* Put strict_length back in original order

* Remove strict_length stub from BQ

* Fix discover_database export statement

* Add return annotation to mysql table_exists

* Black formatter pass

* Add more documentation on when you should use

* Add developer notes.

* Fix code block documentation

* Enhance discover database

* Add unit tests for discover database

* Fix unit tests

* Add two more tests

* Reverse Postgres string_length change

---------

Co-authored-by: Jason Walker <jason@indivisible.org>

* Zoom Authentication + Polling API (#873)

* Add multiple python versions to CI tests (#858)

* Add multiple python versions to CI tests

* Remove duplicate key

* Combine CI jobs

* Update ubuntu image and actually install Python versions

* Replace pyenv with apt-get to install python versions

* Remove sudo

* Remove get from 'apt-get'

* Update apt before attempting to install

* Add ppa/deadsnakes repository

* Add prereq

* Fix typo

* Add -y to install command

* Move -y to correct spot

* Add more -ys

* Add some echoes to debug

* Switch back to pyenv approach

* Remove tests from circleci config and move to new github actions config

Note: no caching yet, this is more of a proof of concept

* Split out Mac tests into seaparate file

* Set testing environmental variable separately

* First attempt to add depdendency cache

* Remove windows tests for now

* Fix circleci config

* Fix circleci for real this time

* Add tests on merging of PRs and update readme to show we do not support for Python 3.7

* Enable passing `identifiers` to ActionNetwork `upsert_person()` (#861)

* Enable passing `identifiers` to ActionNetwork upsert_person

* Remove unused arguments from method

self.get_page method doesn't exist and that method call doesn't return
anything. The return statement works fine as-is to return all tags and
handles pagination on its own.

* Include deprecated per_page argument for backwards compatibility

Emit a deprecation warning if this argument is used

* Include examples in docstring for `identifiers` argument

* Expand documentation on ActionNetwork identifiers

* Add pre-commit hook config to run flake8 and black on commit (#864)

Notes added to README on how to install and set up

* black format

* black format

* jwt -> s2s oauth

* scaffold new functions

* add docs

* return

* add type handling

* pass in updated params

* move access token function

* ok let's rock!!

* make changes

* pass access token key only

* use temporary client to gen token

* mock request in constructor

* drop unused imports

* add changes

* scaffolding tests

* Add multiple python versions to CI tests (#858)

* Add multiple python versions to CI tests

* Remove duplicate key

* Combine CI jobs

* Update ubuntu image and actually install Python versions

* Replace pyenv with apt-get to install python versions

* Remove sudo

* Remove get from 'apt-get'

* Update apt before attempting to install

* Add ppa/deadsnakes repository

* Add prereq

* Fix typo

* Add -y to install command

* Move -y to correct spot

* Add more -ys

* Add some echoes to debug

* Switch back to pyenv approach

* Remove tests from circleci config and move to new github actions config

Note: no caching yet, this is more of a proof of concept

* Split out Mac tests into seaparate file

* Set testing environmental variable separately

* First attempt to add depdendency cache

* Remove windows tests for now

* Fix circleci config

* Fix circleci for real this time

* Add tests on merging of PRs and update readme to show we do not support for Python 3.7

* Enable passing `identifiers` to ActionNetwork `upsert_person()` (#861)

* Enable passing `identifiers` to ActionNetwork upsert_person

* Remove unused arguments from method

self.get_page method doesn't exist and that method call doesn't return
anything. The return statement works fine as-is to return all tags and
handles pagination on its own.

* Include deprecated per_page argument for backwards compatibility

Emit a deprecation warning if this argument is used

* Include examples in docstring for `identifiers` argument

* Expand documentation on ActionNetwork identifiers

* Add pre-commit hook config to run flake8 and black on commit (#864)

Notes added to README on how to install and set up

* black format

* black format

* jwt -> s2s oauth

* scaffold new functions

* add docs

* return

* add type handling

* pass in updated params

* move access token function

* ok let's rock!!

* make changes

* pass access token key only

* use temporary client to gen token

* mock request in constructor

* drop unused imports

* add changes

* scaffolding tests

* write unit tests

* drop poll endpoints for now

---------

Co-authored-by: Shauna <shaunagm@gmail.com>
Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>

* Merging Main Before Release (#880)

* Add multiple python versions to CI tests (#858)

* Add multiple python versions to CI tests

* Remove duplicate key

* Combine CI jobs

* Update ubuntu image and actually install Python versions

* Replace pyenv with apt-get to install python versions

* Remove sudo

* Remove get from 'apt-get'

* Update apt before attempting to install

* Add ppa/deadsnakes repository

* Add prereq

* Fix typo

* Add -y to install command

* Move -y to correct spot

* Add more -ys

* Add some echoes to debug

* Switch back to pyenv approach

* Remove tests from circleci config and move to new github actions config

Note: no caching yet, this is more of a proof of concept

* Split out Mac tests into seaparate file

* Set testing environmental variable separately

* First attempt to add depdendency cache

* Remove windows tests for now

* Fix circleci config

* Fix circleci for real this time

* Add tests on merging of PRs and update readme to show we do not support for Python 3.7

* Enable passing `identifiers` to ActionNetwork `upsert_person()` (#861)

* Enable passing `identifiers` to ActionNetwork upsert_person

* Remove unused arguments from method

self.get_page method doesn't exist and that method call doesn't return
anything. The return statement works fine as-is to return all tags and
handles pagination on its own.

* Include deprecated per_page argument for backwards compatibility

Emit a deprecation warning if this argument is used

* Include examples in docstring for `identifiers` argument

* Expand documentation on ActionNetwork identifiers

* Add pre-commit hook config to run flake8 and black on commit (#864)

Notes added to README on how to install and set up

* Add Events Helpers to PDI Connector (#865)

* add helpers to Events object

* stage docstring

* add docs

* linting

* fix typo + enforce validation

* add return docs

* add events tests

* use mock pdi

* jk

* mark live tests

* add alias

* drop unused imports

* change release number (#872)

* add release notes yml (#878)

---------

Co-authored-by: Shauna <shaunagm@gmail.com>
Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: sharinetmc <128429303+sharinetmc@users.noreply.github.com>

* Switch from API key to Personal Access Token (#866)

* Wraedy/bigquery db connector (#875)

* Create the DatabaseConnector

* Implement DatabaseConnector for the DB connectors

* Add DatabaseConnector to std imports

* Flake8 fix

* Remove reference to padding in copy()

* Add database_discover and fix inheritance

* Remove strict_length from copy()

* Put strict_length back in original order

* Remove strict_length stub from BQ

* Fix discover_database export statement

* Add return annotation to mysql table_exists

* Black formatter pass

* create bigquery folder in databases folde

* create query parity between bigquery and redshift

* mock up copy functionality for bigquery

* fix typo

* add duplicate function to bigquery

* move transaction to helper function

* implement upsert

* fix imports and packages

* add get tables and views methods

* add query return flexibility

* match bigquery apis with redshift

* make s3 to gcs more generic

* add transaction support to bigquery

* remove logs

* add gcs docs

* process job config in function

* finish todo's (and add one more lol)

* [ wip ] AttributeError

* add raw download param

* drop raw download

* copy from GCS docstring

* copy s3 docs

* copy docs

* docstrings

* control flow

* add source path to aws transfer spec

* add Code object to imports

* cleaning up slightly

* check status code

* nice

* pass in required param

* add pattern handling

* add quote character to LoadJobConfig

* add schema to copy from gcs

* drop dist and sortkeys

No longer input params

* add delimiter param

* use schema definition

* write column mapping helper

* pass in formatted schema to load_uri fn

* rename new file

* move file with jason's changes

* move new changes back into file to maintain history

* remove extraneous fn and move project job config

* get back to test parity

* fix bad merge conflict

* remove extra params from copy sig

* clarify transaction guidance

* clean up list blobs

* clean up storage transfer polling

* upgrade cloud storage package

* use list of schema mappings

* scaffolded big file function 😎

* add to docs

* default to compression

we can make this more flexible, just scaffolding

* add temp logging

we can drop this later just trying to get a handle on cycle time

* use decompress

* add logging

* implement unzipping and reuploading cloud file

* logging error

* Add destination path

* Small fix

* add todo's

* drop max wait time

* add kwargs to put blob

Potentially useful for metadata (content type, etc.)

* add verbosity to description

* black formatted

* add gcs to/from helpers

* write to_bigquery function

* update big file logic

* allow jagged rows logic

* test additional methods

* add duplicate table test

* test drop flag for duplicate

* basic test for upsert

* add typing

* move non-essential logs to debug

* move logs to debug

* hey, it works!

* add UUID support for bigquery type map

* add datetime to bigquery type map

* address comments

* address comments

* drop GCS class function

we can pick this up later but it doesn't currently work

* move class back to old location with new import

* revert to old name

* remove transaction error handler

* add description conditional block for s3

* change one more conditional to s3

* handle empty source paths

* reverting new import path

---------

Co-authored-by: Jason Walker <jason@indivisible.org>
Co-authored-by: Ian <47256454+IanRFerguson@users.noreply.github.com>
Co-authored-by: Kasia Hinkson <52927664+KasiaHinkson@users.noreply.github.com>

* BigQuery - Add Column Helpers (#911)

* add column outlines

* optionally log query

* flip default params

* flip back

* Google BigQuery - Clean Up Autodetect Logic (#914)

* don't delete

* clean up schema autodetect logic

* undo comments

* Update stale references to parsons.databases.bigquery (#920)

* Fix BQ references in discover_database

* Update BQ references in tofrom.py

* Update BQ refs in test_discover_database.py

* Fix gcs hidden error (#930)

* logging

* edit flake8 max line for testing

* change flake8 for testing

* comment out unsused var

* add print to check branch

* change to logging

* more logging

* try printing

* more logging

* logging:

* more printing

* more logging

* print transfer job request

* change error message

* requested changes

* remove comment

* Allow list blobs to return all info

* Return list

* List comprehension awesomeness

* trying something

* trying

* Moving things

* trying another thing

* return all blobs

* Forgot to change param back

* GoogleCloudStorage - Handle zip / gzip files flexibly (#937)

* Update release (#894)

* Zoom Polls (#886)

* Merge main into major-release (#814)

* Use black formatting in addition to flake8 (#796)

* Run black formatter on entire repository

* Update requirements.txt and CONTRIBUTING.md to reflect black format

* Use black linting in circleci test job

* Use longer variable name to resolve flake8 E741

* Move noqa comments back to proper lines after black reformat

* Standardize S3 Prefix Conventions (#803)

This PR catches exception errors when a user does not exhaustive access to keys in an S3 bucket

* Add Default Parameter Flexibility (#807)

Skips over new `/` logic checks if prefix is `None` (which is true by default)

* MoveOn Shopify / AK changes (#801)

* Add access_token authentication option for Shopify

* Remove unnecessary check
The access token will either be None or explicitly set; don't worry about an empty string.

* Add get_orders function and test

* Add get_transactions function and test

* Add function and test to get order

* style fixes

* style fixes

---------

Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>

* Catch File Extensions in S3 Prefix (#809)

* add exception handling

* Shortened logs for flake8

* add logic for default case

* added file logic + note to user

* restructured prefix logic

This change moves the prefix -> prefix/ logic into a try/except block ... this will be more robust to most use cases, while adding flexibility that we desire for split-permission buckets

* drop nested try/catch + add verbose error log

* Add error message verbosity

Co-authored-by: willyraedy <wraedy@gmail.com>

---------

Co-authored-by: willyraedy <wraedy@gmail.com>

---------

Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: Ian <47256454+IanRFerguson@users.noreply.github.com>
Co-authored-by: Cody Gordon <13374656+codygordon@users.noreply.github.com>
Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>
Co-authored-by: willyraedy <wraedy@gmail.com>

* black format

* black format

* jwt -> s2s oauth

* scaffold new functions

* add docs

* return

* DatabaseConnector Interface to Major Release (#815)

* Create the DatabaseConnector

* Implement DatabaseConnector for the DB connectors

* Add DatabaseConnector to std imports

* Flake8 fix

* Remove reference to padding in copy()

* Add database_discover and fix inheritance

* Remove strict_length from copy()

* Put strict_length back in original order

* Remove strict_length stub from BQ

* Fix discover_database export statement

* Add return annotation to mysql table_exists

* Black formatter pass

* Add more documentation on when you should use

* Add developer notes.

* Fix code block documentation

* Enhance discover database

* Add unit tests for discover database

* Fix unit tests

* Add two more tests

* Reverse Postgres string_length change

---------

Co-authored-by: Jason Walker <jason@indivisible.org>

* add type handling

* pass in updated params

* move access token function

* ok let's rock!!

* make changes

* pass access token key only

* use temporary client to gen token

* mock request in constructor

* drop unused imports

* add changes

* scaffolding tests

* Add multiple python versions to CI tests (#858)

* Add multiple python versions to CI tests

* Remove duplicate key

* Combine CI jobs

* Update ubuntu image and actually install Python versions

* Replace pyenv with apt-get to install python versions

* Remove sudo

* Remove get from 'apt-get'

* Update apt before attempting to install

* Add ppa/deadsnakes repository

* Add prereq

* Fix typo

* Add -y to install command

* Move -y to correct spot

* Add more -ys

* Add some echoes to debug

* Switch back to pyenv approach

* Remove tests from circleci config and move to new github actions config

Note: no caching yet, this is more of a proof of concept

* Split out Mac tests into seaparate file

* Set testing environmental variable separately

* First attempt to add depdendency cache

* Remove windows tests for now

* Fix circleci config

* Fix circleci for real this time

* Add tests on merging of PRs and update readme to show we do not support for Python 3.7

* Enable passing `identifiers` to ActionNetwork `upsert_person()` (#861)

* Enable passing `identifiers` to ActionNetwork upsert_person

* Remove unused arguments from method

self.get_page method doesn't exist and that method call doesn't return
anything. The return statement works fine as-is to return all tags and
handles pagination on its own.

* Include deprecated per_page argument for backwards compatibility

Emit a deprecation warning if this argument is used

* Include examples in docstring for `identifiers` argument

* Expand documentation on ActionNetwork identifiers

* Add pre-commit hook config to run flake8 and black on commit (#864)

Notes added to README on how to install and set up

* black format

* black format

* jwt -> s2s oauth

* scaffold new functions

* add docs

* return

* add type handling

* pass in updated params

* move access token function

* ok let's rock!!

* make changes

* pass access token key only

* use temporary client to gen token

* mock request in constructor

* drop unused imports

* add changes

* scaffolding tests

* write unit tests

* added testing

* drop typing (for now)

* update docstring typing

* add tests

* write functions

* update typing

* add poll results

* update table output

* fix tests

* uhhh run it back

* add scope requirements

* add to docs

We can add more here if folks see fit

* one for the money two for the show

---------

Co-authored-by: Jason <JasonW94@gmail.com>
Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: Cody Gordon <13374656+codygordon@users.noreply.github.com>
Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>
Co-authored-by: willyraedy <wraedy@gmail.com>
Co-authored-by: Jason Walker <jason@indivisible.org>
Co-authored-by: Shauna <shaunagm@gmail.com>

* Check for empty tables in zoom poll results (#897)

Co-authored-by: Jason Walker <jason@indivisible.org>

* Bump urllib3 from 1.26.5 to 1.26.17 (#901)

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.5 to 1.26.17.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](urllib3/urllib3@1.26.5...1.26.17)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add MobileCommons Connector (#896)

* mobilecommons class

* Update __init__.py

* get broadcasts

* fix get broadcast request

* Add mc_get_request method

* Add annotation

* Incorporate Daniel's suggestions and finish up get_broadcasts

* A few more methods

Need to figure out page_count issue

* small fix

* Remove page_count, use page record num instead

* Add in page_count again

Not all get responses include num param, but do include page_count. wft

* Fix logging numbers

* Add create_profile

* Fix error message for post request

* Start tests

* Add some tests

* Continue testing

* Update test_mobilecommons.py

* functionalize status_code check

* break out parse_get_request function

* fix test data

* fix documentation typo

* Add several tests

* Update mobilecommons.py

* Fix limit and pagination logic

* debug unit testing

* better commenting and logic

* Documentation

* Add MC to init file

* Revert "Merge branch 'main' into cormac-mobilecommons-connector"

This reverts commit cad250f, reversing
changes made to 493e117.

* Revert "Add MC to init file"

This reverts commit 493e117.

* Revert "Revert "Add MC to init file""

This reverts commit 8f87ec2.

* Revert "Revert "Merge branch 'main' into cormac-mobilecommons-connector""

This reverts commit 8190052.

* Fix init destruction

* fix init yet again

* Update testing docs with underscores

* Lint

* Lint tests

* break up long responses

* Fix more linting issues

* Hopefully last linting issue

* DGJKSNCHIVBN

* Documentation fixes

* Remove note to self

* date format

* remove random notes

* Update test_mobilecommons.py

---------

Co-authored-by: sharinetmc <128429303+sharinetmc@users.noreply.github.com>

* #741 : Deprecate Slack chat.postMessage `as_user` argument and allow for new authorship arguments (#891)

* remove the argument and add a warning that the usage is deprecated

* remove usage of as_user from sample code

* add in the user customization arguments in lieu of the deprecated as_user argument

* add comment regarding the permissions required to use these arguments

* use kwargs

* surface the whole response

* allow usage of the deprecated argument but surface the failed response better

* add to retry

* delete test file

* fix linting

* formatting to fix tests

* fix if style

* add warning for using thread_ts

* move the documentation to the optional arguments

* #816 Airtable.get_records() fields argument can be either str or list (#892)

* all fields to be a str object

* remove newline

* Nir's actionnetwork changes (#900)

* working on adding a functio to an and took care of a lint issues

* init

* working on all get functions

* actionnetwork functions batch 1 is ready

* linting and black formatted compliance

* removed unwanted/unsused lines

* merged updated main

* did some linting

* added some more get functions to support all ActionNetwork objects (Advocacy Campaigns, Attendances, Campaigns, Custom Fields, Donations, Embeds, Event Campaigns, Events, Forms, Fundraising Pages, Items, Lists, Messages, Metadata, Outreaches, People, Petitions, Queries, Signatures, Submissions, Tags, Taggings, Wrappers)

* worked on linting again

* fix airtable.insert_records table arg (#907)

* Add canales s3 functions (#885)

* add raw s3 functions to parsons

* add selected functions to s3.py

* delte redundant functions and move drop_and_save function to redshift.py

* create test file

* add s3 unit tests

* add rs.drop_and_unload unit test

* add printing for debugging

* remove testing file

* unsaved changes

* remove unused packages

* remove unneeded module

* Bump urllib3 from 1.26.17 to 1.26.18 (#904)

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.17 to 1.26.18.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](urllib3/urllib3@1.26.17...1.26.18)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: sharinetmc <128429303+sharinetmc@users.noreply.github.com>

* New connector for working with the Catalist Match API (#912)

* Enable api_connector to return error message in `text` attribute

Some API error responses contain the error message in the `text`
attribute, so this update makes it possible to fetch that message if
it exists.

* New connector to work with the Catalist Match API

* Add pytest-mock to requirements to support mocking in pytests

* Tests on the catalist match connector

* More open ended pytest-mock version for compatibility

* Expand docstring documetation based on feedback in PR

* More verbose error on match failure

* Parameterize template_id variable

* Expand docstrings on initial setup

* Include Catalist documentation rst file

* Enhancement: Action Network Connector: Added unpack_statistics param in get_messages method (#917)

* Adds parameter to get_messages

This adds the ability to unpack the statistics which are returned as a nested dictionary in the response.

* added unpack_statistics to an.get_messages()

* added parameters to get_messages and built tests

* changes unpack_statistics to False by default.

* added tbl variable

* formatted with black

* fixed docs

---------

Co-authored-by: mattkrausse <106627640+mattkrausse@users.noreply.github.com>

* Adding rename_columns method to Parsons Table (#923)

* added rename_columns for multiple cols

* linted

* added clarification to docs about dict structure

* updated docs

---------

Co-authored-by: mattkrausse <106627640+mattkrausse@users.noreply.github.com>

* Add http response to update_mailer (#924)

Without returning the response, or at least the status code, it's impossible to check for errors.

* Enable passing arbitrary additional fields to NGPVAN person match API (#916)

* match gcs api to s3

* wip

* two different functions

* use csv as default

* drop unused var

* add docs

* use temp file

* add comments

* wip

* add docs + replicate in gzip

* boy howdy!

* set timeout

* Revert "Merge branch 'main' into ianferguson/gcs-pathing"

This reverts commit 5b1ef6e, reversing
changes made to f0eb3d6.

* black format

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Kasia Hinkson <52927664+KasiaHinkson@users.noreply.github.com>
Co-authored-by: Jason <JasonW94@gmail.com>
Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: Cody Gordon <13374656+codygordon@users.noreply.github.com>
Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>
Co-authored-by: willyraedy <wraedy@gmail.com>
Co-authored-by: Jason Walker <jason@indivisible.org>
Co-authored-by: Shauna <shaunagm@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Cormac Martinez del Rio <66973815+cmdelrio@users.noreply.github.com>
Co-authored-by: sharinetmc <128429303+sharinetmc@users.noreply.github.com>
Co-authored-by: Angela Gloyna <angela.gloyna@gmail.com>
Co-authored-by: NirTatcher <75395024+NirTatcher@users.noreply.github.com>
Co-authored-by: justicehaze <69082853+justicehaze@users.noreply.github.com>
Co-authored-by: mattkrausse <106627640+mattkrausse@users.noreply.github.com>
Co-authored-by: mkrausse-ggtx <131683556+mkrausse-ggtx@users.noreply.github.com>
Co-authored-by: Sophie Waldman <62553142+sjwmoveon@users.noreply.github.com>

* GoogleCloudStorage - Add GCS Destination Path Param (#936)

* Update release (#894)

* Zoom Polls (#886)

* Merge main into major-release (#814)

* Use black formatting in addition to flake8 (#796)

* Run black formatter on entire repository

* Update requirements.txt and CONTRIBUTING.md to reflect black format

* Use black linting in circleci test job

* Use longer variable name to resolve flake8 E741

* Move noqa comments back to proper lines after black reformat

* Standardize S3 Prefix Conventions (#803)

This PR catches exception errors when a user does not exhaustive access to keys in an S3 bucket

* Add Default Parameter Flexibility (#807)

Skips over new `/` logic checks if prefix is `None` (which is true by default)

* MoveOn Shopify / AK changes (#801)

* Add access_token authentication option for Shopify

* Remove unnecessary check
The access token will either be None or explicitly set; don't worry about an empty string.

* Add get_orders function and test

* Add get_transactions function and test

* Add function and test to get order

* style fixes

* style fixes

---------

Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>

* Catch File Extensions in S3 Prefix (#809)

* add exception handling

* Shortened logs for flake8

* add logic for default case

* added file logic + note to user

* restructured prefix logic

This change moves the prefix -> prefix/ logic into a try/except block ... this will be more robust to most use cases, while adding flexibility that we desire for split-permission buckets

* drop nested try/catch + add verbose error log

* Add error message verbosity

Co-authored-by: willyraedy <wraedy@gmail.com>

---------

Co-authored-by: willyraedy <wraedy@gmail.com>

---------

Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: Ian <47256454+IanRFerguson@users.noreply.github.com>
Co-authored-by: Cody Gordon <13374656+codygordon@users.noreply.github.com>
Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>
Co-authored-by: willyraedy <wraedy@gmail.com>

* black format

* black format

* jwt -> s2s oauth

* scaffold new functions

* add docs

* return

* DatabaseConnector Interface to Major Release (#815)

* Create the DatabaseConnector

* Implement DatabaseConnector for the DB connectors

* Add DatabaseConnector to std imports

* Flake8 fix

* Remove reference to padding in copy()

* Add database_discover and fix inheritance

* Remove strict_length from copy()

* Put strict_length back in original order

* Remove strict_length stub from BQ

* Fix discover_database export statement

* Add return annotation to mysql table_exists

* Black formatter pass

* Add more documentation on when you should use

* Add developer notes.

* Fix code block documentation

* Enhance discover database

* Add unit tests for discover database

* Fix unit tests

* Add two more tests

* Reverse Postgres string_length change

---------

Co-authored-by: Jason Walker <jason@indivisible.org>

* add type handling

* pass in updated params

* move access token function

* ok let's rock!!

* make changes

* pass access token key only

* use temporary client to gen token

* mock request in constructor

* drop unused imports

* add changes

* scaffolding tests

* Add multiple python versions to CI tests (#858)

* Add multiple python versions to CI tests

* Remove duplicate key

* Combine CI jobs

* Update ubuntu image and actually install Python versions

* Replace pyenv with apt-get to install python versions

* Remove sudo

* Remove get from 'apt-get'

* Update apt before attempting to install

* Add ppa/deadsnakes repository

* Add prereq

* Fix typo

* Add -y to install command

* Move -y to correct spot

* Add more -ys

* Add some echoes to debug

* Switch back to pyenv approach

* Remove tests from circleci config and move to new github actions config

Note: no caching yet, this is more of a proof of concept

* Split out Mac tests into seaparate file

* Set testing environmental variable separately

* First attempt to add depdendency cache

* Remove windows tests for now

* Fix circleci config

* Fix circleci for real this time

* Add tests on merging of PRs and update readme to show we do not support for Python 3.7

* Enable passing `identifiers` to ActionNetwork `upsert_person()` (#861)

* Enable passing `identifiers` to ActionNetwork upsert_person

* Remove unused arguments from method

self.get_page method doesn't exist and that method call doesn't return
anything. The return statement works fine as-is to return all tags and
handles pagination on its own.

* Include deprecated per_page argument for backwards compatibility

Emit a deprecation warning if this argument is used

* Include examples in docstring for `identifiers` argument

* Expand documentation on ActionNetwork identifiers

* Add pre-commit hook config to run flake8 and black on commit (#864)

Notes added to README on how to install and set up

* black format

* black format

* jwt -> s2s oauth

* scaffold new functions

* add docs

* return

* add type handling

* pass in updated params

* move access token function

* ok let's rock!!

* make changes

* pass access token key only

* use temporary client to gen token

* mock request in constructor

* drop unused imports

* add changes

* scaffolding tests

* write unit tests

* added testing

* drop typing (for now)

* update docstring typing

* add tests

* write functions

* update typing

* add poll results

* update table output

* fix tests

* uhhh run it back

* add scope requirements

* add to docs

We can add more here if folks see fit

* one for the money two for the show

---------

Co-authored-by: Jason <JasonW94@gmail.com>
Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: Cody Gordon <13374656+codygordon@users.noreply.github.com>
Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>
Co-authored-by: willyraedy <wraedy@gmail.com>
Co-authored-by: Jason Walker <jason@indivisible.org>
Co-authored-by: Shauna <shaunagm@gmail.com>

* Check for empty tables in zoom poll results (#897)

Co-authored-by: Jason Walker <jason@indivisible.org>

* Bump urllib3 from 1.26.5 to 1.26.17 (#901)

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.5 to 1.26.17.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](urllib3/urllib3@1.26.5...1.26.17)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add MobileCommons Connector (#896)

* mobilecommons class

* Update __init__.py

* get broadcasts

* fix get broadcast request

* Add mc_get_request method

* Add annotation

* Incorporate Daniel's suggestions and finish up get_broadcasts

* A few more methods

Need to figure out page_count issue

* small fix

* Remove page_count, use page record num instead

* Add in page_count again

Not all get responses include num param, but do include page_count. wft

* Fix logging numbers

* Add create_profile

* Fix error message for post request

* Start tests

* Add some tests

* Continue testing

* Update test_mobilecommons.py

* functionalize status_code check

* break out parse_get_request function

* fix test data

* fix documentation typo

* Add several tests

* Update mobilecommons.py

* Fix limit and pagination logic

* debug unit testing

* better commenting and logic

* Documentation

* Add MC to init file

* Revert "Merge branch 'main' into cormac-mobilecommons-connector"

This reverts commit cad250f, reversing
changes made to 493e117.

* Revert "Add MC to init file"

This reverts commit 493e117.

* Revert "Revert "Add MC to init file""

This reverts commit 8f87ec2.

* Revert "Revert "Merge branch 'main' into cormac-mobilecommons-connector""

This reverts commit 8190052.

* Fix init destruction

* fix init yet again

* Update testing docs with underscores

* Lint

* Lint tests

* break up long responses

* Fix more linting issues

* Hopefully last linting issue

* DGJKSNCHIVBN

* Documentation fixes

* Remove note to self

* date format

* remove random notes

* Update test_mobilecommons.py

---------

Co-authored-by: sharinetmc <128429303+sharinetmc@users.noreply.github.com>

* #741 : Deprecate Slack chat.postMessage `as_user` argument and allow for new authorship arguments (#891)

* remove the argument and add a warning that the usage is deprecated

* remove usage of as_user from sample code

* add in the user customization arguments in lieu of the deprecated as_user argument

* add comment regarding the permissions required to use these arguments

* use kwargs

* surface the whole response

* allow usage of the deprecated argument but surface the failed response better

* add to retry

* delete test file

* fix linting

* formatting to fix tests

* fix if style

* add warning for using thread_ts

* move the documentation to the optional arguments

* #816 Airtable.get_records() fields argument can be either str or list (#892)

* all fields to be a str object

* remove newline

* Nir's actionnetwork changes (#900)

* working on adding a functio to an and took care of a lint issues

* init

* working on all get functions

* actionnetwork functions batch 1 is ready

* linting and black formatted compliance

* removed unwanted/unsused lines

* merged updated main

* did some linting

* added some more get functions to support all ActionNetwork objects (Advocacy Campaigns, Attendances, Campaigns, Custom Fields, Donations, Embeds, Event Campaigns, Events, Forms, Fundraising Pages, Items, Lists, Messages, Metadata, Outreaches, People, Petitions, Queries, Signatures, Submissions, Tags, Taggings, Wrappers)

* worked on linting again

* fix airtable.insert_records table arg (#907)

* Add canales s3 functions (#885)

* add raw s3 functions to parsons

* add selected functions to s3.py

* delte redundant functions and move drop_and_save function to redshift.py

* create test file

* add s3 unit tests

* add rs.drop_and_unload unit test

* add printing for debugging

* remove testing file

* unsaved changes

* remove unused packages

* remove unneeded module

* Bump urllib3 from 1.26.17 to 1.26.18 (#904)

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.17 to 1.26.18.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](urllib3/urllib3@1.26.17...1.26.18)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: sharinetmc <128429303+sharinetmc@users.noreply.github.com>

* New connector for working with the Catalist Match API (#912)

* Enable api_connector to return error message in `text` attribute

Some API error responses contain the error message in the `text`
attribute, so this update makes it possible to fetch that message if
it exists.

* New connector to work with the Catalist Match API

* Add pytest-mock to requirements to support mocking in pytests

* Tests on the catalist match connector

* More open ended pytest-mock version for compatibility

* Expand docstring documetation based on feedback in PR

* More verbose error on match failure

* Parameterize template_id variable

* Expand docstrings on initial setup

* Include Catalist documentation rst file

* Enhancement: Action Network Connector: Added unpack_statistics param in get_messages method (#917)

* Adds parameter to get_messages

This adds the ability to unpack the statistics which are returned as a nested dictionary in the response.

* added unpack_statistics to an.get_messages()

* added parameters to get_messages and built tests

* changes unpack_statistics to False by default.

* added tbl variable

* formatted with black

* fixed docs

---------

Co-authored-by: mattkrausse <106627640+mattkrausse@users.noreply.github.com>

* Adding rename_columns method to Parsons Table (#923)

* added rename_columns for multiple cols

* linted

* added clarification to docs about dict structure

* updated docs

---------

Co-authored-by: mattkrausse <106627640+mattkrausse@users.noreply.github.com>

* Add http response to update_mailer (#924)

Without returning the response, or at least the status code, it's impossible to check for errors.

* Enable passing arbitrary additional fields to NGPVAN person match API (#916)

* match gcs api to s3

* Revert "Merge branch 'main' into ianferguson/gcs-pathing"

This reverts commit 5b1ef6e, reversing
changes made to f0eb3d6.

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Kasia Hinkson <52927664+KasiaHinkson@users.noreply.github.com>
Co-authored-by: Jason <JasonW94@gmail.com>
Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: Cody Gordon <13374656+codygordon@users.noreply.github.com>
Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>
Co-authored-by: willyraedy <wraedy@gmail.com>
Co-authored-by: Jason Walker <jason@indivisible.org>
Co-authored-by: Shauna <shaunagm@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Cormac Martinez del Rio <66973815+cmdelrio@users.noreply.github.com>
Co-authored-by: sharinetmc <128429303+sharinetmc@users.noreply.github.com>
Co-authored-by: Angela Gloyna <angela.gloyna@gmail.com>
Co-authored-by: NirTatcher <75395024+NirTatcher@users.noreply.github.com>
Co-authored-by: justicehaze <69082853+justicehaze@users.noreply.github.com>
Co-authored-by: mattkrausse <106627640+mattkrausse@users.noreply.github.com>
Co-authored-by: mkrausse-ggtx <131683556+mkrausse-ggtx@users.noreply.github.com>
Co-authored-by: Sophie Waldman <62553142+sjwmoveon@users.noreply.github.com>

* Bump version number to 3.0.0

* Fix import statement in test_bigquery.py

* Add Tests to major-release Branch (#949)

* add major release branch to gh workflow

* add mac tests

* null changes (want to trigger test)

* remove temp change

* Resolve GCP Test Failures For Major Release (#948)

* add full import

* resolve bigquery unzip test

* remove keyword

* fix flake8 errors

* fix linting

* push docs

* fix flake8

* too long for flake8

* Install google-cloud-storage-transfer for google extras (#946)

This is required for the import of storage_transfer to work

Co-authored-by: Ian <47256454+IanRFerguson@users.noreply.github.com>

* Revert "Enable passing `identifiers` to ActionNetwork `upsert_person()` (#861)" (#945)

This reverts commit 77ead60.

Co-authored-by: Ian <47256454+IanRFerguson@users.noreply.github.com>

* BigQuery - Add row count function to connector (#913)

* add row count function

* use sql

* add unit test

* unit test

* whoops!

* add examples (#952)

* Parse Boolean types by default (#943)

* Parse Boolean types by default

Commit 766cfae created a feature for
parsing boolean types but turned it off by default. This commit turns
that feature on by default and adds a comment about how to turn it off
and what that does.

* Fix test expectations after updating boolean parsing behavior

* Only ever interpret python bools as SQL booleans

No longer coerce by default any of the following as booleans:
"yes", "True", "t", 1, 0, "no", "False", "f"

* Fix redshift test parsing bools

* Move redshift test into test_databases folder

* Remove retired TRUE_VALS and FALSE_VALS configuration variables

We now only use python booleans

* address comments

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Jason <JasonW94@gmail.com>
Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
Co-authored-by: Ian <47256454+IanRFerguson@users.noreply.github.com>
Co-authored-by: Cody Gordon <13374656+codygordon@users.noreply.github.com>
Co-authored-by: sjwmoveon <sophie.waldman@moveon.org>
Co-authored-by: Alex French <alexandra.j.french@gmail.com>
Co-authored-by: Kathy Nguyen <kathymn@gwu.edu>
Co-authored-by: willyraedy <wraedy@gmail.com>
Co-authored-by: Jason Walker <jason@indivisible.org>
Co-authored-by: Shauna <shaunagm@gmail.com>
Co-authored-by: sharinetmc <128429303+sharinetmc@users.noreply.github.com>
Co-authored-by: Kathy Nguyen <kathy.nguyen@moveon.org>
Co-authored-by: dexchan <2642977+dexchan@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Cormac Martinez del Rio <66973815+cmdelrio@users.noreply.github.com>
Co-authored-by: Angela Gloyna <angela.gloyna@gmail.com>
Co-authored-by: NirTatcher <75395024+NirTatcher@users.noreply.github.com>
Co-authored-by: justicehaze <69082853+justicehaze@users.noreply.github.com>
Co-authored-by: mattkrausse <106627640+mattkrausse@users.noreply.github.com>
Co-authored-by: mkrausse-ggtx <131683556+mkrausse-ggtx@users.noreply.github.com>
Co-authored-by: Sophie Waldman <62553142+sjwmoveon@users.noreply.github.com>
@austinweisgrau austinweisgrau deleted the match branch February 5, 2024 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants