The Modified 5-Year-Olds’ Index – testing reliability and the impact of training level

  • Philip J Cooper

Student thesis: Doctoral ThesisDoctor of Dental Surgery (DDS)


Aim: To test the reliability of the Modified 5-Year-Olds’ Index using assessors with a range of experience, and to determine whether calibration improves reliability.
Design: Prospective reliability study.
Setting: Bristol Dental School, University of Bristol.
Methods: Fifty study models of non-syndromic 5-year-olds with a repaired UCLP were selected from the CCUK archives. Fifteen participants with a range of clinical experience in orthodontics were divided into three groups of equal experience. Each group was given differing amounts of information: Group 1 information sheet; Group 2 information sheet and reference models; Group 3 calibration course, information sheet and reference models. Each participant scored the 50 models using the Modified 5-Year-Olds’ Index on two occasions at least four weeks apart. ICCs calculated from a two-way random effects model were used to calculate intra-rater reliability and inter-rater reliability comparing assessors’ scoring to the experts’ consensus scores (gold standard).
Results: Group 2 (ICC 0.80 – 0.93) and Group 3 (ICC 0.80 – 0.91) demonstrated high levels of intra-rater agreement, with lower levels shown by Group 1 (0.68 – 0.93). Inter-rater agreement was high in Group 2 (ICC 0.87 – 0.93) and Group 3 (0.82 – 0.91), with Group 1 showing the lowest levels of agreement (ICC 0.69 – 0.94). The level of training of the assessors in Group 1 appears to influence reliability scores, with high intra- and inter-rater reliability scores of the consultant and post-CCST trainee at a similar level to those of the same grade in Groups 2 and 3.
Conclusions: The Modified 5-Year-Olds’ Index is a reliable method of assessing outcomes when model scoring is carried out by consultants or post-CCST trainees. Calibration in use of the Index does not improve reliability.
Date of Award26 Nov 2020
Original languageEnglish
Awarding Institution
  • The University of Bristol
SupervisorTony Ireland (Supervisor), Sam D Leary (Supervisor), Jonathan Sandy (Supervisor) & Nicola Atack (Supervisor)


  • cleft
  • UCLP
  • 5-Year-Olds' Index
  • Modified 5-Year-Olds' Index
  • cleft lip and palate
  • unilateral cleft lip and palate
  • orthodontics
  • calibration
  • reliability
  • Index
  • Indices

Cite this