A different take on the role of evaluation in arts education

RSA Fellow Nicola Wallis shares her views on evaluation.

Being part of the Evidence Champions Network, alongside the AHRC Cultural Value Project (University of Warwick, National Gallery & University of Cambridge Museums collaboration) has inspired me to really interrogate the ‘why’ behind my own practice and the activities of our organisations. Effective evaluation is only possible when we have a good understanding of what we are trying to achieve. How else will we know whether or not we have been successful?

What does successful arts education look like? As experienced practitioners, we have a sense of what this is, and whether things have gone well or not, but when this knowledge remains embodied in individuals only, we cannot build on it collaboratively as a sector. The process of learning more about evaluation was revealing as we set out to understand how things had come to be as they are, and to begin to articulate and align our individual views  as to what our priorities should be going forward.

We are finding that as the rationale for our activity becomes clearer and more defined, so does the rationale for our evaluation process. For example, if we aim to promote children’s rights to free and full participation in the arts then we can use our evaluation procedures not only to assess whether or not we are being successful in this, but also as a further tool which itself is part of the way in which we honour and actively promote children’s rights

For me, this is the ultimate purpose of all evaluation: clarifying our values. Not just as a check that we are making good use of time and resources, or meeting promised deliverables, important though these things are for accountability and for quality assurance. Neither is evaluation simply a tool from which educators and organisations can plan their next steps – searching for gaps, or for less successful elements of activity that can be refined and developed in order that they might meet stated aims more completely.

Now I consider evaluation as a pedagogical tool in itself. When we question children and young people about certain aspects of our programmes, we have the potential opportunity to show them that we value their views, and to highlight what we value ourselves. As arts educators, we are used to enabling dialogue and knowledge exchange to occur using a variety of different media and techniques. Evaluation is another mode of communication.

It is important, as in so much of arts education, that the conversations that we invite people to are genuinely two-way. Whatever methods we choose should have room not just for organisations and audiences to find their shared values, but also to introduce new perspectives from either side.

Where’s a logical place to start?

When embarking on this new process of incorporating evaluation into our pedagogy, colleagues and I began working on a logic model by listing out current activity streams and trying to map them backwards onto our stated objectives. This proved to be pretty unsatisfactory, and of course, starting with what you are aiming for and then selecting activities that willhelp you get there is a much more conventional route!

However, as colleagues and other evidence champions in the network pointed out, current activities have not, for the most part, arisen at random or by chance. They have been developed over many years, and contain within them the traces of much professional expertise and bespoke pragmatic judgements tailored precisely to the unique context of the organization.

Simply rejecting this out of hand, in order not to fall into ‘but we’ve always done it this way’ laziness, could risk missing out on valuable rich and longitudinal evidence gained over time by practitioners constantly honing and reshaping their practice in the light of real-time feedback from participants. Equally, if we simply accept current practice as the best we can ever do, we give ourselves no opportunity to improve, or to adapt our work as contexts and audience themselves change.

A plea for sitting by

What I’m wondering is whether we need to take a fresh look at the kind of evaluation that we value. Of course we need to have accurate assessment of our work against clear outcomes in order to meet the goals that are important to us and our audiences. But the ‘truest’ assessment is not always the one that employs the most highly-rated validated tools, or has been subjected to the highest levels of statistical scrutiny.

The word ‘assessment’, which we use nowadays to mean the evaluation, measurement, and documentation of learning, has its origins in the Latin ‘assidere’ meaning ‘to sit by’. I’m trying to spend more time ‘sitting by’ the children and young people I work with: listening to their priorities, goals and values with the intention that these will have an impact on what my own become too.

I know that many other arts educators using evaluation to improve their pedagogical practice are doing exactly the same too. The challenge now is how to document and share this ‘sitting by’ in ways that are recognized and valued as the important evidence gathering and professional development that it is. This might need a new shared language beyond traditional understandings of ‘validity’ and ‘robustness’. Still thinking what that would be, but I think a start might be ‘care’.