Review process

[nuest]

@MarkusKonk @edzer Feedback welcome!

Draft for review process

This workshop runs a public peer review. The review process described here is therefore transparent for submitters and tries to be both brief and explicit.

1. Submitters create a pull request (PR) with their submission. By creating the PR the submitter accepts that their work will be published under a Creative Commons Attribution 4.0 International License.
2. The organization committee (@o2r-project/agile-2017-org-committee) checks the form of the submission and assigns the PR to at least one member of @o2r-project/agile-2017-reviewers to conduct the main review following the guidelines. Reviewers are welcome to follow new PRs and volunteer for a submission.
3. After the review is completed, the reviewer creates a the review recommendation comment on the PR mentioning the organizing committee (@o2r-project/agile-2017-org-committee) and including one of the following statements: Review recommendation: Accept for presentation, Review recommendation: Accept for presentation after revision, Review recommendation: Reject
4. All reviewers are welcome to give a short comment on a submission.
5. In case of Accept for presentation after revision, the PR is kept open until the recommendation is updated to either Accept for presentation, or Reject.
6. Once the review recommendations for all submissions are either acceptance or rejection, the @o2r-project/agile-2017-org-committee holds a meeting public Gitter chat and discusses all recommendation considering the overall programme and bounding conditions (numbers of possible slots, diversity of topics). The committee then make a decision comment for each submission, trying to follow the reviewers' recommendation: Decision: Accept for presentation, Decision: Reject
7. The decision comment invites all review committee members (@o2r-project/agile-2017-reviewers) to briefly check the review arguments and cast a vote using "thumb up/down" reactions on the comment. There must be a simple majority and the review committee is the tie breaker.
8. In case of acceptance of the submission, the PR is merged by @o2r-project/agile-2017-org-committee. In case of rejection, the pull request is closed without merge (a list of accepted and rejected submissions will be included in the workshop report.

[MarkusKonk]

Hier meine Kommentare:

• "[...] that her work will be published"
• heißt Punkt 3, dass Review guidelines #3 nicht mehr aktuell ist? Das widerspricht sich momentan etwas
• ab Punkt 5 bin ich etwas verwirrt. Wird in Punkt 7 über die Reviewer recommendation entschieden oder über organization commitee discussion? Müssen sich alle Reviewer durch alle Submissions klicken und voten oder wird das auch zugeorndet?

[nuest]

@MarkusKonk What exactly are the contradictions between 3. and #3 ?

Tried to clarify the language regarding the other comments.

[foost]

Suggest to rephrase
5. "In case of Accept for presentation after revision, the PR is kept open until the recommendation is updated to either Accept for presentation, or Reject."
6. "Once the review recommendations ...."

Step 7 seems a bit complicated - is it really necessary to describe it in that much detail now?

[MarkusKonk]

No difference between them, just a missing #.

[nuest]

I simplified Step 7 a bit and made the changes suggested in #2 (comment)

[nuest]

Moved to wiki https://github.com/o2r-project/agile-2017/wiki/Review-process