CANDIDATE SELECTION

Candidates are nominated by the technical program committee chair based on recommendations of technical program committee members and reviewers. The general chair together with the technical program committee chair are responsible for nominating a best paper award committee and sending a final shortlist to this committee for selection. The number of the papers in the shortlist shall represent the top 1% of all the submissions in the year (or 3 papers, whichever is smaller) for one category.

 

BEST PAPER AWARD COMMITTEE

After the candidate list has been established, the general chair and technical program committee chair propose a best paper award committee. Members of the committee cannot have a paper in the shortlist. The committee has at least 3 members and is approved by the organizing committee. The committee should be broad enough in expertise to represent the IEEE TALE areas and should as much as possible represent a mix of academic lineage and demographics. Committee members should have a record of serving on the IEEE TALE or related conferences by the IEEE Education Society. One of the members should take up the role as the chairperson for making the final judgment after considering the reviews by the members.

 

BEST PAPER COMMITTEE DECISION MAKING PROCESS

The committee receives the shortlisted papers with paper metadata including but not limiting to anonymized reviews and best paper nomination scores, before the first day of the conference welcoming for the initial review of these papers.

The main task of the committee is to select the best papers from the shortlist to receive the award. All these papers should be eligible candidates to receive the award. The reviewing process typically involves the following steps: (1) Gathering preliminary brief review comments from the award committee members prior to the conference; (2) Attending the oral presentation and asking probing questions for clarification; (3) Giving independent overall rating on each candidate; then 3) Selecting candidates for the best paper award. The final selection should be sent by the chair of the award committee to the technical program committee chair and the general chair for confirmation.

The number of best paper awards given in a year should not be significantly higher than a usual practice. For reference, usually the top 5 papers were shortlisted to be considered for the award in the past TALE conferences. The committee may use their discretion and professional judgment to decide if the awardees could represent the best work worth to be promoted by the conference.

The award committee members may consider the merits of the following criteria:

  • Contribution (15%): Have the authors explicitly and clearly articulated how their research will contribute to current work?
  • Theoretical and Methodological Orientation (15%): Have the authors explicitly and clearly described how they performed their research? To what extent does their approach align with currently accepted theories and methodologies?
  • Findings and Conclusions (15%): To what extent are research findings compelling and different from what has been previously published? To what extent have the research findings contributed to existing knowledge and/or literature? Would the findings be worthwhile to be extended and published in any journal? Are implications for future research considered? If appropriate, are implications for practice considered?
  • Organization, Illustrations and Oral Presentation Clarity (55%): To what extent does the paper and demonstrate effectively organized, clear, and concise presentation appropriate for the readership of the Proceedings? To what extent do tables and figures meaningfully add to the narrative? Is the author as the presenter able to present the paper with clarity to demonstrate a full knowledge of the work? Can the presenter answer the questions from the audience clearly?
  • Any New Contribution or existing one with new methodology / execution

The score in each category should be given by the committee members after attending the oral presentation, and the overall score should be used in determining the best paper awards regardless of the peer review scores. In other words, the paper with a higher peer review score should not be more advantageous than the one with lower peer review score. The category 4 is given the highest weight, carrying the common practice in the conference that the best paper is mainly reviewed based on the oral presentation clarity.

Each category should hold a sub-score based on the 5-Likert scale, as follows:

1 = Poor
2 = Fair
3 = Good
4 = Excellent
5 = Outstanding

After each sub-score is collected, the following formula should be used to compute the overall score:

Total Score = (Subscore_1 + Subscore_2 + Subscore_3) x 15% + Subscore_4 x 55%

All the total scores from each committee member will be averaged as the resulting final score. If the resulting final score is at least 3 or above, it should be eligible for the best paper award. The committee should meet and discuss with each case, such that the decision can be made beyond the quantitative results.

Would you like your work to be considered for an IEEE TALE Award? Submit your paper before the deadline to be considered!