Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[judge] Algorithm revamp #32

Open
sumanthratna opened this issue Dec 14, 2020 · 0 comments
Open

[judge] Algorithm revamp #32

sumanthratna opened this issue Dec 14, 2020 · 0 comments
Labels
enhancement New feature or request

Comments

@sumanthratna
Copy link
Member

sumanthratna commented Dec 14, 2020

from HackTJ 7.5 feedback from judges:

  • Hung on to the project that is overall best—if a project is gone, it's gone
  • Didn’t like single-elimination system: can’t give credit to good projects that had different strengths
  • "anti-bias" is cause for concern (I understand—but I think an explanation of Crowd-BT would mitigate this concern?)
  • old system, where you rank on a rubric, was better because of the above

ways to fix the above:

  • don't do an "overall" decision (?)
  • use a rubric (i.e., more than just 3 subcategories)
    • if we get rid of overall, aggregate the scoring from a single project into a single overall score to use for information gain
    • a bit more like the previous system (more nuance in voting) but doesn't come with the downside of "choose from 1 to 10"
  • allow re-making a comparison with an old project (but the same comparison shouldn't be made, e.g., current can be repeated but the same prev and current shouldn't)
@sumanthratna sumanthratna added the enhancement New feature or request label Dec 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant