In the following, we describe in more details the guidelines to ensure a good review for HHAI2023. This is also done for transparency w.r.t. authors of the papers.
Relevance
What is the relevance of this paper to the HHAI audience? Note that the field of HHAI is driven by current developments in AI, but also requires fundamentally new approaches and solutions. In addition, we need collaboration with areas such as HCI, cognitive and social sciences, philosophy & ethics, complex systems, and others.
Possible scores:
- 1) Not relevant
- 2) Of limited interest to HHAI audience
- 3) Relevant to researchers in a subarea only
- 4) Likely to be of interest to a large proportion of the community
Significance
Do the contributions address an outstanding problem in science or society? Are other people (practitioners or researchers) likely to use these ideas or build on them? Does the paper address a difficult problem in a better way than previous research? Does it provide unique data, unique conclusions on existing data, or a unique theoretical or pragmatic approach? Does the paper provide complementary or novel perspectives to the current scientific discourse?
Possible scores:
- 1) Not significant
- 2) Moderately significant
- 3) Significant
- 4) Highly significant
Novelty
Papers should present novel research. Novelty could be in i) the techniques developed for HHAI contexts and presented, ii) in the application of existing and emerging techniques to a novel domain within HHAI, iii) in a novel experimental comparison of different HHAI techniques (either through computational/simulation experiments or empirical studies), or iv) novel analyses, theories or models that expand the understanding of HHAI, among others. In all cases, papers should be substantiated by facts or principled arguments. Please judge the following: Are the problems or approaches novel? Is this a novel combination of familiar techniques or theories? Is it clear how this work differs from previous contributions? Is related work adequately referenced?
Possible scores:
- 1) Not novel
- 2) Moderately novel
- 3) Novel
- 4) Very novel
Clarity
Is the paper clearly written? Is it well-organised? (If not, feel free to make suggestions to improve the manuscript.) Does it adequately inform the reader? (A superbly written paper provides enough information for the expert reader to reproduce its results.)
Possible scores:
- 1) Poor
- 2) Satisfactory
- 3) Good
- 4) Excellent
Overall Evaluation (followed by a detailed review)
Please use this to provide an overall evaluation including a rationale for the above criteria ratings. Also use the area for other comments, questions, and suggestions for how to improve the paper. Consider trying to articulate the strengths and weaknesses of the paper in advance of forming your overall assessment. Questions for the authors to address during the revised version of the manuscript (if accepted) should be addressed here, too.
Possible scores:
- An accept: A seminal paper for the ages. Clearly an outstanding paper. I assume no further discussion is needed.
- A weak accept: I vote and argue for acceptance.
- A borderline paper: I vote for acceptance, although I would not be upset if it were rejected.
- A weak rejection: I vote and argue for rejection.
- A rejection: I will strongly argue for rejection.
Reviewer’s Confidence
Possible scores
- 1: The reviewer’s evaluation is an educated guess and it is quite likely that the reviewer did not understand central parts of the paper. Either the paper is not in the reviewer’s area, or it was extremely difficult to understand.
- 2: The reviewer is knowledgeable but the topic or methods of the paper are outside their primary area of research. They are fairly confident that the evaluation is correct. It is possible that the reviewer did not understand certain parts of the paper, or that the reviewer was unfamiliar with a piece of relevant literature.
- 3: The reviewer is knowledgeable in the area of research of the paper and is confident but not absolutely certain that the evaluation is correct. It is unlikely but conceivable that the reviewer did not understand certain parts of the paper, or that the reviewer was unfamiliar with a piece of relevant literature.
- 4: The reviewer is an expert in the area of research and is absolutely certain that the evaluation is correct and very familiar with the relevant literature.
Paper Awards
Outstanding papers can receive a paper award for their submission.
Please check the box to suggest a paper that should be considered for an award.