Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fetch databaseId for repos and users when initializing streams #112

Merged
merged 15 commits into from
Apr 29, 2022
Merged

Fetch databaseId for repos and users when initializing streams #112

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
f8c99ed
Add repo_id to context of all child streams of Repositories
laurentS Apr 26, 2022
8c13ccb
Add repo_id to PR stream context
laurentS Apr 26, 2022
d4c55e8
Refactor post_process onto parent class
laurentS Apr 27, 2022
23c3908
Fix mypy errors
laurentS Apr 27, 2022
b252307
Fix incorrect key in context
laurentS Apr 27, 2022
1763189
Add missing context for projects
laurentS Apr 27, 2022
3fb6e75
Fix config test
laurentS Apr 27, 2022
c897cf8
Correct context fetching
laurentS Apr 27, 2022
106d1cc
Rename test fixture
laurentS Apr 27, 2022
d10256b
Add user_id to user based streams
laurentS Apr 28, 2022
e88548c
Expand tests to check case correction and user streams
laurentS Apr 28, 2022
a36d67c
Fix config validation test
laurentS Apr 28, 2022
1a5c2ef
Add option to skip top level user stream with tests
laurentS Apr 28, 2022
40a9d11
Try to cache more requests
laurentS Apr 28, 2022
cbe6971
Update marker
ericboucher Apr 29, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions tap_github/client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,12 @@ def parse_response(self, response: requests.Response) -> Iterable[dict]:

yield from results

def post_process(self, row: dict, context: Optional[Dict[str, str]] = None) -> dict:
"""Add `repo_id` by default to all streams."""
if context is not None and "repo_id" in context:
row["repo_id"] = context["repo_id"]
return row


class GitHubGraphqlStream(GraphQLStream, GitHubRestStream):
"""GitHub Graphql stream class."""
Expand Down
Loading