Using evidence to improve practice – a collective endeavour?

In early 2018, the RSA and the Arts Council Bridge Organisations launch an important new programme of work to improve how arts practitioners and teachers integrate the use of evidence of effective practice into arts and cultural learning. Laura Partridge reflects on what they can learn from other sectors that have trodden similar paths.

There is an inescapable truth, as Dan Corry notes in a recent NPC report on innovations in evaluation, that ‘for many, the words measurement and evaluation spell despair and are met with a deep sigh’. This is perhaps unsurprising if these are understood as things that have to be done in order to meet someone else’s requirements; to ensure the continuity of your funding; to prove that your work is worthy. These sentiments might explain, in part, why our recent survey of organisations delivering arts and cultural learning found that many are using relatively light-touch approaches to evaluation. 

But for those of us doing work with a social purpose, if we can reclaim the evaluation agenda for ourselves – “own” it as something that helps us, rather than seeing it a mere accountability mechanism – we might make even more of a difference in the lives of the people we reach. If we understand how our programmes work, for whom, and in what context(s), we can do more of what works. Similarly, if we know more about where our programmes are less effective, for which groups, and why, we can improve them.

The reality is that if you believe that your organisation alone has to find all the answers, this journey will feel lonely, and those aforementioned deep sighs will be frequent. There will always be others who have taken similar approaches, and learnt something along the way about what makes a difference. And yet, when we surveyed organisations working in arts and cultural learning earlier this year, half of all respondents found it ‘difficult’ or ‘very difficult’ to find evidence from other organisations or academic sources about when, how and why their project might make a difference to children and young people; or had never tried to seek out existing evidence at all.

Not surprisingly, when we asked arts and cultural organisations what would best support them to improve their work, they most commonly chose ‘opportunities to develop practice and share findings with other similar organisations’.

The Learning About Culture programme is designed with this issue in mind. The broad range of strategic and delivery partners behind the programme are committed to learning from what has gone before. We have been speaking to leaders from other sectors who’ve been working collaboratively to encourage more widespread use of research and evidence with a view to drawing lessons from their experience.

Over the coming months, we will share more examples, but a really impressive one comes from international development. Sarah Mistry, Director of Effectiveness and Learning at the sector’s membership organisation and Fellow of the RSA, recently explained to me that since the early 2000s, Bond have supported a long process in partnership with international development organisations and funders towards achieving a shared approach to evaluation. This was when international development organisations first expressed that they, as Sarah put it, ‘wanted common language and tools, high standards and a means of sharing best practice in order to speak with a more unified voice that would make a stronger case for its work’.

Along the way, Bond have learnt important lessons. At the outset, there was a lot of demand for evaluation tools, but these alone cannot support a sector to develop its practice. Sarah Mistry explained that what organisations really want and need is to ‘share knowledge and experience and solve problems together’ through learning cohorts and for this to be supplemented with training courses and some specialist consultancy where needed.

The impacts of these collaborations are many, but one clear example involves joint working between the organisations funded by DFID. By combining learning from across their work, these organisations were able to demonstrate the importance of unrestricted funding to achieving positive outcomes in international development projects. This led DFID to include unrestricted funding in their major grants programme.  

Bond’s journey still continues today, building on the many lessons learnt along the way, including the following:

• It is crucial to engage funders in discussions about evaluation and learning if practice is to change;
• Innovation must be underpinned by evaluation and learning from prior work (of others as well as yourselves);
• Organisations need more help to commission effective independent evaluations of their work that support continuous learning and project improvements.

Today, Bond is doing increasing amounts of work with organisation leaders and funders to explore how innovation can be informed by evidence and has developed a series of guides for commissioning independent evaluation to ensure that it supports organisations to reflect and improve. There is much here for the cultural sector to build on in order to, in Sarah’s words, ‘build sensible and mature reflective behaviours that value learning’.

In designing the Learning About Culture programme, we are learning from the prior experience of Bond and others who have attempted to develop evidence-based practice across their sectors, such as Project Oracle whose Impact Pioneers programme evaluation has informed our approach. We are also drawing on research such as the outcomes of a recent EEF trial on improving use of evidence in schools, which found that schools that were successful in embedding use of evidence in to their practice did so through structured collaboration and enquiry. Or that of the Alliance for Useful Evidence whose review of approaches to improving use of evidence among (mostly medical) practitioners demonstrates the importance of opinion leaders taking up roles as ‘evidence champions’, as well as highlighting the value of opportunities to develop critical appraisal skills and getting in place organisational/systemic structures and processes that support evidence-use.

Drawing on this existing knowledge about how to build evidence-rich practice, the Learning about Culture programme will create regional networks of arts practitioners and educators, who commit to extending their own use of evidence in their work and championing the role of evidence in their organisations. The structure of these networks and the activity that the groups undertake will be designed to encourage reflection that informs both innovation and continuous improvement. The groups will be carefully curated to ensure that members have common interests to support the sustainability of relationships beyond the lifecycle of the programme. We have ensured that funders are involved in the conversation from the very beginning.

Yet, we are keen to keep learning. If you know of other sectors that have developed an exciting approach to evaluating and learning together? If so, please comment below and we’ll follow up to find out more!

How else can you get involved?

• If you are a school leader, teacher or practitioner involved in arts and cultural learning and you would like to be part of our Evidence Champions Network, you can sign up to hear more using the form below.

• If you are a corporate or charitable funder of arts and cultural learning, please grab a cuppa and take 5-6mins to fill in our survey about how you support your grantees.

• Read more about Learning about Culture.