Talk to experienced colleagues before examining your first thesis
Novice examiners may find it overwhelming to evaluate a thesis and award a grade of either “passed” or “not passed”. A good strategy is to talk to experienced colleagues about how they approach the examination process. Familiarizing yourself with McGill’s thesis examiner report is crucial to accurately assessing a thesis.
Before examining a thesis, be sure to read McGill’s criteria for thesis examination. Examiners are required to complete an examiner report (PhD or Master’s), which includes rating the thesis on several criteria, giving an overall evaluation of “passed” or “not passed”, and listing recommendations for revision.
Reviewing definitions of concepts such as originality, significance and quality can help clarify how they can be applied to thesis evaluation. For example, here is how the Oxford Learning Institute defines originality, significance, and quality based on Lovitts (2007).
(Photo credit: Allen McInnis)
If you are a new supervisor and do not have much experience examining theses, it may be helpful to talk to more senior colleagues before you examine your first thesis.
Questions to ask:
- How do they define originality, significance and quality?
- What method do they use when reading theses (e.g., what section do they read first)?
- Do their approaches differ between Master’s and doctoral theses?
- Are there any particularly strong or weak theses that they can recall examining? What were the qualities that made them stand out positively or negatively?
How will you cope with personal biases?
All examiners will agree that fairness to the thesis being examined is important. Yet personal biases sometimes affect examiners’ discretion. What would you do if you disagree with the approach, the paradigm, or the conclusion of the thesis? Would you be annoyed if your own work or a friend’s work is not listed in the bibliography? Would you be able to examine a thesis fairly if it is radically unconventional?
As an examiner, you may ask yourself the following questions when examining a doctoral dissertation. Many of these questions are also helpful to Master’s thesis examiners.
- When you are asked to examine a thesis, do you feel dread at the work or interest? What either puts you off or encourages you within a few pages of reading?
- How short would be too short? If a thesis was original, competent and valuable, how short might it be?
- If an idea is brilliant, how important is the standard of presentation?
- To what extent do you check the literature review material? Would you look up works that you do not know?
- How do you cope with personal bias, if, for example, your own work or a friend’s work is not listed in the bibliography?
- How do you cope with personal bias if you disagree with the approach, the paradigm or the conclusion of the thesis?
- What about when a thesis departs radically from the template for some reason? How can you ensure that you examine it fairly?
- Do you have any pet dislikes or preferences that you suspect might be common in doctoral examiners?
Adapted from Carter (2008, pp. 366-367), reprinted with permission from Taylor & Francis Ltd.
Examiners’ motivations and methods
Examiners undertake this role for a variety of reasons, take the role very seriously, and generally spend a great deal of time examining a thesis. Although there is variation in individual style, many examiners use similar strategies when reading and evaluating theses.
Examiners report that they undertake the examination of theses for several reasons including, firstly, a sense of duty regarding:
maintaining standards within the discipline;
the belief that one of the roles of an academic is to examine theses; and
one's students, or the quid pro quo concept: "It's ... a reciprocal obligation from having one's own students examined" and "I have eight students at the moment which means I need 16 examiners soon ... so I need to reciprocate." (p. 375, reprinted with permission from the Society for Research into Higher Education).
Adapted from Mullins and Kiley (2002).
Other sources of motivation include (Mullins & Kiley, 2002; Tinkler & Jackson, 2004; Wisker & Kiley, 2014):
intellectual interest, including the opportunity to read at a level of detail not included in day-to-day professional reading, and to learn about very recent research;
gaining recognition in the academic and disciplinary community; and
learning the qualities of a strong thesis as well as the examination process and integrating this knowledge into supervisory practices.
Different examiners approach the task differently, but according to Mullins and Kiley (2002), most examiners follow the strategy below.
Begin by reading the abstract, introduction & conclusion to gauge the scope of the work and whether what candidates say they are going to do is actually done
Look at the references to see what sources have been used and whether they need to follow up on any of them
Then read from cover to cover taking detailed notes
Finally, go back over the thesis to check whether their questions have been answered or whether their criticisms are justified
There are different standards for Master’s theses and doctoral dissertations. Typically, originality is not essential for Master’s theses (Anderson, Day, & McLaughlin, 2006; Pilcher, 2011). However, research suggests that Master’s theses and doctoral dissertations may be examined based on a similar framework.
Bourke and Holbrook (2013), based on the survey results from over 300 PhD and Master’s supervisors (who were also examiners), found that they applied the same indicators in evaluating theses.
Contributions (originality, substantive, advance knowledge)
Literature review (coverage, accuracy, use/application)
Methods (appropriateness, effective application)
Analysis/results (appropriateness, effective interpretation)
Presentation (correct expression, communicative competence)
At the same time, their data suggested that examiners expected a slightly lower quality for Master’s theses for each of these indicators.
Anderson, C., Day, K., & McLaughlin, P. (2006). Mastering the dissertation: Lecturers’ representations of the purposes and processes of Master’s level dissertation supervision. Studies in Higher Education, 31(2), 149-168.
Bourke, S., & Holbrook, A. P. (2013). Examining PhD and research Master’s theses. Assessment & Evaluation in Higher Education, 38(4), 407-416.
Carter, S. (2008). Examining the doctoral thesis: A discussion. Innovations in Education and Teaching International, 45(4), 365-374.
Lovitts, B. (2007). Making the implicit explicit: Creating performance expectations for the dissertation. Sterling, VA: Stylus Publishing.
Mullins, G., & Kiley, M. (2002). ‘It's a PhD, not a Nobel Prize’: How experienced examiners assess research theses. Studies in Higher Education, 27(4), 369-386.
Pilcher, N. (2011). The UK postgraduate Master's dissertation: An "elusive chameleon"? Teaching in Higher Education, 16(1), 29-40.
Tinkler, P., & Jackson, C. (2004). The Doctoral Examination Process: A handbook for students, examiners and supervisors. Buckingham: SRHE and Open University Press.
Wisker, G., & Kiley, M. (2014). Professional learning: lessons for supervision from doctoral examining. International Journal for Academic Development, 19(2), 125-138.
Kiley, M., & Mullins, G. (2004). Examining the examiners: How inexperienced examiners approach the assessment of research theses. International Journal of Educational Research, 41(2), 121-135.