Lots of British assessment research is high quality

The Education Research Assessment Exercise (RAE) panel report commented that the gap between innovative and pedestrian research was acute in assessment and noted that there was a relative lack of assessment research presented under the RAE, given the importance of this topic for the country’s education system. A key reason for this is that much high quality assessment research is applied rather than theoretical and is published in the grey literature, if at all. The 2008 RAE criteria, upon which researchers and their institutions were judged and financed, valued peer-reviewed publications in high-impact international academic journals (see box for criteria). Some of our assessment research is world-leading. But we also produce high quality, policy-relevant research that would be acclaimed nationally, rather than internationally, for which the RAE would award a 1*. In fact, due to cultural and structural differences in educational practice, it is highly likely that lots of educational research conducted in Britain might not be readily translated for an international audience. For example, the fact that we assess using extended answer formats as the norm and assume that you need teaching experience to mark examinations are an anathema in other cultural contexts. Lack of international transferability of many of our research agendas may well be the reason that the RAE Education panel awarded fewer 4*s than other subject panels. (However, this has resulted in HEFCE awarding the universities 5% less funding for education in this RAE than in the last round.) Structural factors also affect the international reach of British assessment research. Sixty per cent of education research funding in universities comes from Government sources and this figure is likely to be higher in assessment research. Government-funded, policy- and practice-relevant research tends to be short-term, as the policy context moves rapidly. Policy-makers want answers to new questions yesterday so that they can move on apace with their (always) urgent reforms on the basis of research and evaluation evidence. The result of this can be, at best, high quality, highly specialised research that is not always well linked with the wider literature. This kind of research has its place and has informed critical decisions in recent examination developments, such as the grading of the Diplomas, or the introduction of single level tests. But extra work is required to embed this research in the academic literature and link it to theory or make explicit how it advances knowledge in a way that is not pedestrian. But there’s another snag with Government-funded projects: the Government own the intellectual property and they might not agree to publication of politically sensitive findings. Although they are rarely explicitly recognised as such, awarding bodies are another major assessment research funder – employing teams of researchers to evaluate forthcoming and on-going assessments. Again, only some of their research is published, largely because their main aim is to improve the quality of educational assessments, rather than to disseminate their findings to the academic community. Many of the leading assessment academics worked in examination boards for at least part of their research careers. Awarding bodies provide a valuable public service in this regard. It is worth a moment’s reflection about the likely state of British assessment research if awarding bodies decided at a stroke that they no longer wished to fund research. Quite simply, lots of policy decisions would have to be taken without evidence. So, does the future look bright for high quality educational research? Not necessarily. Assessment research is a niche area and it is hard to fill job vacancies. Routes for developing a research career in this field are fewer than they were in the past. Teachers are now taught very little assessment theory and practice. And what has happened to all of the assessment courses at universities? Without adequate training it will be hard to maintain quality in assessment research. The CIEA’s efforts to galvanize Masters degrees in assessment in universities across the country should help, some of which are being offered for the first time in October 2009. Whilst I am not advocating complacency, we should recognise that a lot of British educational assessment research is valuable – research that is internationally recognised as well as research that is worthwhile nationally.