Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pagination for GraphQl streams #118

Merged
merged 8 commits into from
May 18, 2022
Merged

Add pagination for GraphQl streams #118

merged 8 commits into from
May 18, 2022

Conversation

ericboucher
Copy link
Contributor

@ericboucher ericboucher commented May 13, 2022

Add deep pagination for Graphql streams. Fixes #89

This also fixes #106

This requires setting up extra pagination parameters in the query like this:

query userContributedTo($username: String! $nextPageCursor_1: String) {
            user (login: $username) {
              repositoriesContributedTo (first: 100 after: $nextPageCursor_1 includeUserRepositories: true orderBy: {field: STARGAZERS, direction: DESC}) {
                pageInfo {
                  hasNextPage_1: hasNextPage
                  startCursor_1: startCursor
                  endCursor_1: endCursor
                }
                nodes {
                  node_id: id
                  name_with_owner: nameWithOwner
                  open_graph_image_url: openGraphImageUrl
                  stargazer_count: stargazerCount
                  owner {
                    node_id: id
                    login
                  }
                }
              }
            }
          }

tap_github/client.py Outdated Show resolved Hide resolved
tap_github/client.py Outdated Show resolved Hide resolved
tap_github/client.py Outdated Show resolved Hide resolved
tap_github/client.py Outdated Show resolved Hide resolved
tap_github/user_streams.py Outdated Show resolved Hide resolved
@ericboucher
Copy link
Contributor Author

@laurentS thanks for your comments, I used the str.split in the end. It requires users to follow the imposed schema, but it should fail early if they don't and be safer.

@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 4 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@ericboucher ericboucher merged commit 38215c7 into main May 18, 2022
@ericboucher ericboucher deleted the graphql-pagination branch May 18, 2022 02:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

tap-github --about is broken Add pagination for the Graphql client
3 participants