Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stats and languages results incorrect due to graphql 100 records limit #1852

Open
2 of 3 tasks
rickstaa opened this issue Jul 7, 2022 · 14 comments
Open
2 of 3 tasks
Assignees
Labels
bug Something isn't working. hacktoberfest hacktoberfest-accepted help wanted Extra attention is needed. lang-card Issues related to the language card. ⭐ top bug Top bug. stats-card Feature, Enhancement, Fixes related to stats the stats card.

Comments

@rickstaa
Copy link
Collaborator

rickstaa commented Jul 7, 2022

Describe the bug

The current language and stats card results are incorrect for users with more than 100 repositories (see https://docs.github.com/en/graphql/overview/resource-limitations). This is because we are currently only fetching the first 100 repositories of a user account.

repositories(first: 100, ownerAffiliations: OWNER, orderBy: {direction: DESC, field: STARGAZERS}) {

repositories(ownerAffiliations: OWNER, isFork: false, first: 100) {

Because of this not all languages and stars of a user are represented.

Expected behaviour

I think we can use pagination options (i.e. cursor, after, endCursor objects) to loop through all repositories of a user account (see https://stackoverflow.com/a/70039506/8135687). @anuraghazra Let me know what you think.

TODOs

@rickstaa rickstaa added bug Something isn't working. stats-card Feature, Enhancement, Fixes related to stats the stats card. lang-card Issues related to the language card. labels Jul 7, 2022
@rickstaa
Copy link
Collaborator Author

rickstaa commented Jul 7, 2022

For the language card, a PR is already open to solve this #1178. For the stats card, there not yet exists a pull request. This is, however, less important since only people with more than 100 repositories with stars are affected.

@lumynou5
Copy link

For the language card, a PR is already open to solve this #1178. For the stats card, there not yet exists a pull request. This is, however, less important since only people with more than 100 repositories with stars are affected.

Why it is less important? Although people with more than 100 repositories with stars are less.

@rickstaa
Copy link
Collaborator Author

@Luminous-Coder Thanks for your question. What I meant is that it is probably affecting fewer people. Since the GraphQL call we make returns the repositories listed from most starred to least starred, a user must have 100 repositories with many stars for to score to be significantly affected. I hope that clears things up. Nevertheless, we should solve both cases. 👍🏻

@MatteoPierro
Copy link
Contributor

Hi @rickstaa, is it ok if I work on this for the stats card?

@rickstaa
Copy link
Collaborator Author

rickstaa commented Oct 3, 2022

@MatteoPierro of course! That would be amazing. An example of how this can be done using pagination can be found here. I think for the stats card, to be resource efficient, I think we can stop the pagination when the GraphQL API does not return repositories that have any stars.

@MatteoPierro
Copy link
Contributor

@rickstaa here it's my PR #2100

@MatteoPierro
Copy link
Contributor

@rickstaa Can I work on fixing the fetch all repos for top languages part as well? I see there is already a PR but it looks stale.

@rickstaa
Copy link
Collaborator Author

rickstaa commented Oct 4, 2022

@MatteoPierro, yea no problem. That would be great! 🚀 The pull request was created a long time ago, and I just reviewed it last week. The best thing to do is to work on this pull request so that @tc-imba gets co-authored. I think that #1178 should also be good if the syntax and tests are improved (see #1178 (comment)).

@MatteoPierro
Copy link
Contributor

ok. how can I work on @tc-imba PR in co-authored mode?

@rickstaa
Copy link
Collaborator Author

rickstaa commented Oct 4, 2022

The easiest way is to let git handle the co-authoring. This can be done as follows:

  1. Add @tc-imba's branch as a remote git remote add tcimba git@github.com:tc-imba/github-readme-stats.git.
  2. Fetch all new branches from the remotes git fetch -all.
  3. Checkout @tc-imba's master branch git checkout tcimba/master.
  4. Create a new branch based on @tc-imba's master branch git checkout -b name_this_however_you_want.
  5. Add the upstream master git remote add upstream git@github.com:anuraghazra/github-readme-stats.git.
  6. Fetch the upstream master git fetch upstream.
  7. Merge the upstream master into your new branch git merge upstream/master.
  8. Fix merge conflicts, and you are good to go! 🚀

Let me know if you run into any problems.

@MatteoPierro
Copy link
Contributor

Alright, probably I made it. #2111

@rickstaa
Copy link
Collaborator Author

Just a heads-up, I filled a feature request to make the total stars a specific user has received on his/her repositories available under the user object of the GraphQL API. Adding this feature would also enable #2111 on the public instance.community/community#35677. Please add your support there.

@rickstaa rickstaa mentioned this issue Oct 11, 2022
3 tasks
@milkshakegum
Copy link

milkshakegum commented Oct 12, 2022

@rickstaa
Copy link
Collaborator Author

rickstaa commented Oct 12, 2022

@milkshakegum Thanks for your report. Merging:

Will fix the issues you experience. You can already use these on your own fork. They are not yet merged into the master because we have to ensure the GraphQL and Vercel limits are not hit. We will likely implement this as an opt-in for Vercel instance and Github Action users (see #2179 and #2159).

rickstaa added a commit that referenced this issue Nov 21, 2022
This pull request adds experimental Organization support. Users can
include stats from Organization repositories to which they are
collaborators using the `include_orgs` card argument. Please be aware
that because of #1852, only the first 100 repositories are used.
Including organization, stats might therefore skew your results.

Co-authored-by: Raymond Nook <59678453+developStorm@users.noreply.github.com>
rickstaa added a commit that referenced this issue Nov 21, 2022
This pull request adds experimental Organization support. Users can
include stats from Organization repositories to which they are
collaborators using the `include_orgs` card argument. Please be aware
that because of #1852, only the first 100 repositories are used.
Including organization, stats might therefore skew your results.

Co-authored-by: Raymond Nook <59678453+developStorm@users.noreply.github.com>
rickstaa added a commit that referenced this issue Nov 22, 2022
This pull request adds experimental Organization support. Users can
include stats from Organization repositories to which they are
collaborators using the `include_orgs` card argument. Please be aware
that because of #1852, only the first 100 repositories are used.
Including organization, stats might therefore skew your results.

Co-authored-by: Raymond Nook <59678453+developStorm@users.noreply.github.com>
rickstaa added a commit that referenced this issue Nov 22, 2022
This pull request adds experimental Organization support. Users can
include stats from Organization repositories to which they are
collaborators using the `include_orgs` card argument. Please be aware
that because of #1852, only the first 100 repositories are used.
Including organization, stats might therefore skew your results.

Co-authored-by: Raymond Nook <59678453+developStorm@users.noreply.github.com>
@rickstaa rickstaa added the ⭐ top bug Top bug. label May 9, 2023
@rickstaa rickstaa added the ⭐ top issue Top issue. label Jun 15, 2023
@github-actions github-actions bot added ⭐ top issue Top issue. and removed ⭐ top issue Top issue. labels Jun 15, 2023
@github-actions github-actions bot removed the ⭐ top issue Top issue. label Nov 19, 2023
@github-actions github-actions bot added the ⭐ top issue Top issue. label Nov 22, 2023
@github-actions github-actions bot removed the ⭐ top issue Top issue. label Dec 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working. hacktoberfest hacktoberfest-accepted help wanted Extra attention is needed. lang-card Issues related to the language card. ⭐ top bug Top bug. stats-card Feature, Enhancement, Fixes related to stats the stats card.
Projects
None yet
Development

No branches or pull requests

4 participants