-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Leaderboard #179
Comments
@nasthagiri @nedbat @regisb @idegtiarov Following-up on an action item I took from the last contributor meetup, I've converted this card from the core committer program board into an issue to be able to comment on it. My action item was to add a mention of including badges there, which I've added to the description. Btw it could be worth starting to specify what we want for the leaderboard. Something like what the OpenStack project has, ie https://www.stackalytics.com/ ? |
Thanks for assigning this to me @antoviaque! I'm keen to work on this. |
I will take a look at this as well! Thanks for adding that ticket as a separate issue. |
I am currently looking at the Discourse API documentation to fetch badge and user information. I would like to be able to fetch the following information:
This is relatively easy to achieve, but there needs to be a bridge between Discourse and Github. For this, we can use the Discourse "Associated Accounts" (https://discuss.openedx.org/u/regis/preferences/account). Once we make that connection, we can use the Github API to fill in the remaining information. The only remaining field is the organization. I do not know yet how we can consistently associate a user to an organization. I would like to be able to list (at least) all organizations from the Open edX marketplace. Automatically finding the organization associated to a certain Github profile is imprecise and inconsistent. Thus, I think our best bet is to define a custom Discourse user field. This could either be a free-text field or a dropdown: https://discuss.openedx.org/admin/customize/user_fields @nedbat do you think this would be acceptable? EDIT: I'd also like to display the organization continent, but I don't have a clean solution for this. Ideas? |
I have made some progress on this. The idea is to generate a webpage that will display community members along with the number of likes received on the forums, the count of merged Github PRs, and other cool "vanity" metrics that show how engaged they are in the community. What I had in mind was to parse the Discourse bio summary and to gather extra information via hashtags. For instance, here's what I'd put in my bio:
The "corecommitter" and "overhangio" hashtags will be associated to my profile. The link to Github will also be parsed and the @regisb account name will be associated too. This means that it should be possible to expose the following information via a REST API:
Someone (else than me) will then be able to create a nice frontend where we can list and sort community members, search them by tags, etc. Thoughts? |
@regisb That sounds great! :) One comment is that it might be useful to tie the data to a specific time period - to allow to show the number of PRs, likes,etc over a specific year/month. This would allow newcomers to be able to get to a better position faster, and encourage old-timers to keep contributing :) |
FYI, on our side @symbolist might contribute some parts of this work -- though he would likely only become available from May. @idegtiarov @regisb Still interested to also do a part of this work? |
Actually, I have already written most of the backend code. I just need to implement some caching to make sure that we don't crawl the Discourse API too frequently, while still guaranteeing that we have fresh results at all times. |
@regisb Maybe we could develop this in the open so other people can help? :) |
@nedbat Yes, but I wanted to get the code in a presentable state, first. |
We are going to investigate Stackalitics service as a leaderboard option with one/couple of our internal repositories. The work is planned to start in April. |
Here's my what I got so far: https://github.com/openedx/oxct |
Adding to this thread, we already have an installation of the Grimoire Labs dashboard installed that I think can cover a bunch of the goals captured here. It currently isn't public, but that should be easily enough done. The project aims to implement the community metrics proposed by CHAOSS.
I was going to give Regis a "cooks tour" on a video call next week. If others are interested in joining, ping me on Slack? |
We just came out of a conversation with @e0d who presented your Grimoire instance. It was really interesting, and I'd like to recap here a few points which are close to my heart:
|
@e0d Thanks for the presentation of Grimoire, that was really useful to see! I only knew it through Cauldron -- I had tried to run it on the edX github orgs some time ago, but it is a bit limited in the type of sources it can import there: https://cauldron.io/project/3820 . The setup you have seem much more powerful in that regard: https://openedx-metrics.herokuapp.com/ (CC @bradenmacdonald @nasthagiri as this might be useful to gather data about the core committer program, which you are looking at for a blog post about the program.) Btw, would it be ok to post the recording of this meeting publicly here, in case others would like to watch it? A few ideas/comments that I've found interesting from what you, @regisb @symbolist @idegtiarov @arbrandes mentioned, or reactions to the points you've made:
+1 -- these might be things that we could be able to surface through tickets from bug reports, reports/likes on forums, maybe a role within the release working group? Badges are a good way too yes, maybe a stepped-up version of it could be a way to show the titles and responsibilities that any given person takes in the project? |
I spent some time over the weekend deploying an upgraded instance of Grimoire Labs. It is currently consuming all of the data and I'll share a link once it's done.
|
This is a great idea! |
I have been taking a deeper look at the CHAOSS project this week. To help others who would like to quickly understand what it is about so that they can participate in this discussion, I compiled together some highlights from my investigation here: https://openedx.atlassian.net/wiki/spaces/COMM/pages/2696446382/CHAOSS
I agree with this approach as well. It gives us a concrete starting point that has already been thought about deeply by many experts in the area and has been in use by other communities. We may want to additionally slice and dice the data for specific goals but the framework supports that as well (and so it does not constrain us). Also for the sake of thoroughness, I did try to see if there were any competing standards or options but this seems to be the only comprehensive one.
SortingHat is the part of the suite which is responsible for managing identities. From looking at its documentation it looks like it should support what we want and we just need to look into configuring that (looks like @e0d has already installed the user interface "hatstall" for that): "Sorting Hat maintains an SQL database of unique identities of communities members across (potentially) many different sources. Identities corresponding to the same real person can be merged in the same unique identity with a unique uuid. For each unique identity, a profile can be defined, with the name and other data shown for the corresponding person by default. In addition, each unique identity can be related to one or more affiliations, for different time periods. This will usually correspond to different organizations in which the person was employed during those time periods." https://www.researchgate.net/publication/331088184_SortingHat_Wizardry_on_Software_Project_Members has some more details.
Let me know if I can help with that. 🙂 To also start the conversation about the overall plan, if everyone is in agreement about this as a starting point, the next steps could be:
|
I've made progress getting Grimoire upgraded and configured against the core data sources. An outstanding item is to configure authentication, which I can look at over the weekend. Without that it is not simply a matter of the data being available to everyone, but that anyone would be able to alter dashboards. For CCs, I can send you a preview if Slack me directly. |
Is that even possible? I though that authentication was only available in the commercial edition of Kibana? |
Requiring login with a shared credential is possible, that's where we are right now. This is compatible with allowing readonly access to the views. This needs a little configuration change to work probably, but should be straight-forward. PM me if you want the credentials to view the data. |
That sums up what I remember, yes. |
@e0d @arbrandes I've added some suggestions and questions to your CHAOSS Cleanup spreadsheet, and would like to create a task to address some of these issues during our next sprint (30 June - 13 July). At a glance, I think "merging organizations" will be the easiest to do first since it's manual. But the others will require some (nice) contributions to sortinghat, like "sourcing organization for non-affiliated individuals from github". What do we need to get started on this? I could start by creating a github Project for this work, and start adding issues so we can discuss requirements with everybody. |
GitHub project sounds great.
I do think it is OK to have folks in a pseudo organization, say,
"individual.". But we want to classify whomever we can when they are
affiliated. There will be folks who.are legitimately individuals.
Do you have thoughts on which interventions will have the biggest quality
impacts? I think focusing on CCs and key firms will touch the majority of
contributions for example
…On Wed, Jun 23, 2021, 12:14 PM Jillian Vogel ***@***.***> wrote:
@e0d <https://github.com/e0d> @arbrandes <https://github.com/arbrandes>
I've added some suggestions and questions to your CHAOSS Cleanup
spreadsheet, and would like to create a task to address some of these
issues during our next sprint (30 June - 13 July). At a glance, I think
"merging organizations" will be the easiest to do first since it's manual.
But the others will require some (nice) contributions to sortinghat, like
"sourcing organization for non-affiliated individuals from github".
What do we need to get started on this? I could start by creating a github
Project for this work, and start adding issues so we can discuss
requirements with everybody.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#179 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAJWEAUUU4IC6WVASVSSP3TTUGXYZANCNFSM4VBPOKOQ>
.
|
@e0d question -- what are the source github projects included in this initial Grimoire deployment? Can we add non-edx repos like Tutor and the community-supported XBlocks? |
One more thought, the merged orgs is a good example of the type of change
that needs to be sticky. If we merge edX and edX inc. only for edX inc. to
be recreated during the next identity analysis that an issue. I'm not yet
sure where the two versions originated from. Do we need an aliases concept
for orga?
…On Wed, Jun 23, 2021, 12:31 PM Edward Zarecor ***@***.***> wrote:
GitHub project sounds great.
I do think it is OK to have folks in a pseudo organization, say,
"individual.". But we want to classify whomever we can when they are
affiliated. There will be folks who.are legitimately individuals.
Do you have thoughts on which interventions will have the biggest quality
impacts? I think focusing on CCs and key firms will touch the majority of
contributions for example
On Wed, Jun 23, 2021, 12:14 PM Jillian Vogel ***@***.***>
wrote:
> @e0d <https://github.com/e0d> @arbrandes <https://github.com/arbrandes>
> I've added some suggestions and questions to your CHAOSS Cleanup
> spreadsheet, and would like to create a task to address some of these
> issues during our next sprint (30 June - 13 July). At a glance, I think
> "merging organizations" will be the easiest to do first since it's manual.
> But the others will require some (nice) contributions to sortinghat, like
> "sourcing organization for non-affiliated individuals from github".
>
> What do we need to get started on this? I could start by creating a
> github Project for this work, and start adding issues so we can discuss
> requirements with everybody.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <#179 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAJWEAUUU4IC6WVASVSSP3TTUGXYZANCNFSM4VBPOKOQ>
> .
>
|
Currently it's every public project in the edX and Open edX GitHub orgs.
We can add other repos if that makes sense. I think we need to work out
that definition.
…On Wed, Jun 23, 2021, 12:31 PM Jillian Vogel ***@***.***> wrote:
@e0d <https://github.com/e0d> question -- what are the source github
projects included in this initial Grimoire deployment? Can we add non-edx
repos like Tutor and the community-supported XBlocks?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#179 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAJWEAXHOALYCH6HXAOASELTUGZZXANCNFSM4VBPOKOQ>
.
|
Can we export the number of contributions that are being counted against each non-org individual, so we can sort them and ensure the highest numbers are affiliated somewhere if that's appropriate? But yes, the CC people by definition will have the most contributions, so I've updated the "Recommended Organization" for all the core contributors I could identify. |
FYI I've created a github project to track these issues and ideas: https://github.com/orgs/edx/projects/6 Can people confirm they can edit those cards? I haven't converted any to proper issues yet, but I think that's what we have to do to allow comments. |
@pomegranited Thank you! 👍
I confirm that I can edit them yes. |
Same here. |
Hi everyone, this issue hasn't been touched since June 2021. Was there any enthusiasm/capacity to pick up on this idea, or should we close the issue? If we want to keep it I propose moving the issue to https://github.com/openedx/wg-coordination/issues since this issue doesn't pertain to an OEP. |
Closing per #227 (comment) |
As a contributor, I would like to see my achievements and compare myself with other contributors, in order to celebrate my wins and remain motivated for even more contributions.
To consider:
The text was updated successfully, but these errors were encountered: