2021 BenchCouncil International Symposium on Benchmarking, Measuring and Optimizing (Bench 2021)
Nov 14-16, 2021
1. The online discussion is blind. While the reviewers discuss the papers, they don’t know others’ identities beyond reviewer #A, #B, …. Hence, a single reviewer cannot easily assert seniority and silence other voices, or influence them beyond the strength of their arguments.
2. When the reviewers pointing out closeness to prior work that informs the reviewer’s decision to lower the novelty and contribution of a paper, they should provide a full citation to that prior work.
3. When the reviewers asking authors to draw a comparison with concurrent work (e.g., work that was published or appeared online *after* the paper submission deadline) or with preliminary work (e.g., a poster or abstract that is not archival), this comparison should not inform a lower score by the reviewer.
4. Provide useful and constructive feedback to the authors. Be respectful, professional and positive in your reviews and provide suggestions for the authors to improve their work.
5. Score the paper absolutely and relative to the group of papers you are reviewing.
Absolute overall merit - There are 4 grades you can give to each paper for absolute overall merit; the top 2 ratings mean that you think the paper is acceptable to the conference and the bottom 2 ratings mean that in your opinion the paper is below the threshold for the conference. Please assign these values thinking, whether the paper is above the threshold for the conference or below.
Relative overall merit – is based on the papers that you are reviewing. You can rank your papers and then group the papers into the 4 bins. Except for fractional errors, you should be dividing your papers equally into the 4 categories.
6. Reviewers must treat all submissions as strictly confidential and destroy all papers once the technical program has been finalized.
7. Reviewers must contact the PC chair or EIC if they feel there is an ethical violation of any sort (e.g., authors seeking support for a paper, authors seeking to identify who the reviewers are).
8. Do not actively look for author identities. Reviewers should judge a paper solely on its merits.
9. If you know the authors, do not publicize the authors. If you would like to recuse yourself from the review task, contact the PC Chair.
10. Reviewers should review the current submission. If you have reviewed a previous submission, make sure your review is based on the current submission.
11. Reviewers must not share the papers with students/colleagues.
12. Reviewers must compose the reviews themselves and provide unbiased reviews.
13. Do not solicit external reviews without consulting the PC chairs or EIC. If you regularly involve your students in the review process as part of their PhD training, contact the PC chairs. You are still responsible for the reviews. You may do this on no more than one of your reviews.
14. Reviewers must keep review discussions (including which papers you reviewed) confidential.
15. Do not discuss the content of a submitted paper/reviews with anyone other than officially on the submission management system like HotCRP or EasyChair during the online discussion period or the PC meeting (from now until paper publication in any venue).
16. Do not reveal the name of paper authors in case reviewers happen to be aware of author identity. (Author names of accepted papers will be revealed after the PC meeting; author names of rejected papers will never be revealed.)
17. Do not disclose the outcome of a paper until its authors are notified of its acceptance or rejection.
18. Do not download or acquire material from the review site that you do not need access to.
19. Do not disclose the content of reviews, including the reviewers' identities, or discussions about a paper.
This set of review ethics is derived and based on the MICRO 2020, ASPLOS 2020-2021, ISCA 2020-21 review guidelines.