A study into the feasibility and validity of using alternatives to questionnaires to evaluate the impact of a selection of physics & chemistry science communication activities

Abstract

One of the current issues in the field of Science Communication is that of evaluating the impact of activities designed to engage people with science and technology. Evaluation is widely seen as the primary tool to identify best practice and provide empirical evidence of the positive impact of such activities.
However, there is no standard method for doing this or for comparing the results. The majority of published evaluations involve the use of quantitative data generated through written questionnaires.
Using such methods to evaluate the impact of short-duration activities is often inappropriate or impractical. This dissertation investigates alternatives to the traditional questionnaire for obtaining feedback about the impact of some types of science communication activity. It also attempts to establish what the effects on participants are.
A number of Institute of Physics and Bristol ChemLabS (Chemistry) events have been evaluated using a variety of unconventional techniques. These included postcard writing, children’s drawings, comments walls and other informal feedback, and generated largely qualitative data. A methodological and theoretical justification for their use is outlined.
The data has been analysed using the Generic Learning Outcomes (GLO) framework. The results were assessed to ascertain whether the evaluation methods studied are reliable and valid ways of measuring impact. Practical considerations, such as the ease of data collection have also been examined. In addition the impacts themselves are identified and discussed.
The study has identified those techniques which provide a range and depth of information about the learning outcomes experienced by respondents. They can also easily be embedded into science communication activities and do not require time-consuming analysis.
The impact of the physics and chemistry activities studied was found to be very positive for the majority of participants. A myriad of effects are seen at a variety of levels and over a range of timescales.

Date of Award	25 Nov 2009
Original language	English
Awarding Institution	University of Bristol
Supervisor	Jocelyn M Wishart (Supervisor) & Tim G Harrison (Supervisor)