Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

All repos for an organization #141

Closed
wants to merge 5 commits into from

Conversation

loeakaodas
Copy link
Contributor

Description of change

add parsing of "org/*" wildcard to retrieve all repos for an org as per TDL-6330

Manual QA steps

  • ran tap locally

Risks

Rollback steps

  • revert this branch

Copy link
Contributor

@KrisPersonal KrisPersonal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@loeakaodas Looks like your unit test case covers only extract_repos_from_config function. What about get_all_repos? Am i missing anything?

@loeakaodas
Copy link
Contributor Author

@KrisPersonal I originally didn't include a test for get_all_repos as it was only called from extract_repos_from_config and was indirectly tested as part of the test_org_all_repos method, but it was being mocked.

So, I went ahead and added specific unit test for get_all_repos now, but I did have to add a new dev dependency to mock the requests Response object easily.

@loeakaodas
Copy link
Contributor Author

Closing as PR was re-submitted in #146

@loeakaodas loeakaodas closed this Nov 12, 2021
loeakaodas added a commit that referenced this pull request Nov 17, 2021
* add parsing of "org/*" wildcard to retreive all repos for an org

* add unit test cases for extract_repos_from_config()

* add requests-mock for dev requirements

* add unit test for get_all_repos()
KrisPersonal pushed a commit that referenced this pull request Feb 14, 2022
* add mac to gitignore

* change testing repo

* adjust `start_date_2` for new repo

* add tap-tester automatic fields

* add tap-tester all fields

* add all expected streams to all fields test

* set specific bookmark for test-repo

* fix `collaborators` stream bookmark spelling  for tap-tester

* add more streams to automatic fields test

* add tap-tester bookmarks

* updates to automatic fields test:
* add check for unique primary keys in replicated records
* replace explicit set of expected streams with `expected_check_streams` from base
* update `test_run` doc string

* omit `team_memberships` stream from `expected_check_streams`

* build expected_check_streams() using expected_streams()

* add bug id and description

* pylint fixes:
* adjust imports
* use specific error class
* set encoding

* adjust circle config:
* use latest image
* trim pylint disable options
* add unit tests step
* make sure integration tests run always

* Re-submitting PR #141 (All repos for an organization) (#146)

* add parsing of "org/*" wildcard to retreive all repos for an org

* add unit test cases for extract_repos_from_config()

* add requests-mock for dev requirements

* add unit test for get_all_repos()

* Re-submitting PR #142 (Improve Rate Limiting and Retry Logic) (#143)

* add basic backoff retry

* add backoff to setup.py

* replace deprecated assertEquals method with assertEqual

* add MAX_SLEEP_SECONDS parsing from config and DEFAULT_MAX_SECONDS for rate limiting

* add comments for changes and use DEFAULT_SLEEP_SECONDS

* add pylint ignore for global-statement

* README updates: (#148)

- add full list of replicated streams
- update GitHub docs links

* add streams to excluded_streams that aren't respecting automatic fields

* add NotFoundException handling to collaborators stream

* add bug info

* adjust test for test-repo

* pylint fixes

* run unit and integations steps always

* don't raise a NotFoundException to deal with access issues to resources

* pylint fix and return empty response body for 404

* add collaborators stream to excluded set

* fixes to tap-tester tests

* adjust 2nd sync dates for data

* deal with None from get method

* adjust start date tap-tester dates

* actually deal with NoneType in get method

* FIX: sub_streams sync functions passed parent metadata

* remove expected_check_streams after bug identified and addressed

* use expected_streams after bug identified and addressed

* don't write a bookmark for FULL_TABLE streams

* update unit test expectations to recent changes

* updates to bookmarks tap-tester:
* adjust test expectatons for streams
* create simulated_states based on test data and tap behavior
* adjust tests based on commits and pr_commits schema

* update base based on tap behavior and test data

* adjust test expectations for team_members stream

* Exclude collaborators stream due to access issues in circle

* update circle config to include slack orb and tap-tester-user context

* add bug info

* add tap-tester-user to daily build context
AJWurts pushed a commit to villagelabsco/tap-github that referenced this pull request Oct 24, 2024
…singer-io#145)

* add mac to gitignore

* change testing repo

* adjust `start_date_2` for new repo

* add tap-tester automatic fields

* add tap-tester all fields

* add all expected streams to all fields test

* set specific bookmark for test-repo

* fix `collaborators` stream bookmark spelling  for tap-tester

* add more streams to automatic fields test

* add tap-tester bookmarks

* updates to automatic fields test:
* add check for unique primary keys in replicated records
* replace explicit set of expected streams with `expected_check_streams` from base
* update `test_run` doc string

* omit `team_memberships` stream from `expected_check_streams`

* build expected_check_streams() using expected_streams()

* add bug id and description

* pylint fixes:
* adjust imports
* use specific error class
* set encoding

* adjust circle config:
* use latest image
* trim pylint disable options
* add unit tests step
* make sure integration tests run always

* Re-submitting PR singer-io#141 (All repos for an organization) (singer-io#146)

* add parsing of "org/*" wildcard to retreive all repos for an org

* add unit test cases for extract_repos_from_config()

* add requests-mock for dev requirements

* add unit test for get_all_repos()

* Re-submitting PR singer-io#142 (Improve Rate Limiting and Retry Logic) (singer-io#143)

* add basic backoff retry

* add backoff to setup.py

* replace deprecated assertEquals method with assertEqual

* add MAX_SLEEP_SECONDS parsing from config and DEFAULT_MAX_SECONDS for rate limiting

* add comments for changes and use DEFAULT_SLEEP_SECONDS

* add pylint ignore for global-statement

* README updates: (singer-io#148)

- add full list of replicated streams
- update GitHub docs links

* add streams to excluded_streams that aren't respecting automatic fields

* add NotFoundException handling to collaborators stream

* add bug info

* adjust test for test-repo

* pylint fixes

* run unit and integations steps always

* don't raise a NotFoundException to deal with access issues to resources

* pylint fix and return empty response body for 404

* add collaborators stream to excluded set

* fixes to tap-tester tests

* adjust 2nd sync dates for data

* deal with None from get method

* adjust start date tap-tester dates

* actually deal with NoneType in get method

* FIX: sub_streams sync functions passed parent metadata

* remove expected_check_streams after bug identified and addressed

* use expected_streams after bug identified and addressed

* don't write a bookmark for FULL_TABLE streams

* update unit test expectations to recent changes

* updates to bookmarks tap-tester:
* adjust test expectatons for streams
* create simulated_states based on test data and tap behavior
* adjust tests based on commits and pr_commits schema

* update base based on tap behavior and test data

* adjust test expectations for team_members stream

* Exclude collaborators stream due to access issues in circle

* update circle config to include slack orb and tap-tester-user context

* add bug info

* add tap-tester-user to daily build context
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants