Outcome Bound

A new drive to embed efficacy into daily work life is  at the heart of Pearson’s cultural shift

By Sir Michael Barber

With contributions from Vaithegi Vasanthakumar

Now understand efficacy and the importance of learning outcomes, I realise I am part of education in my country.  My children, and even my grandchildren, will be benefited by the job I do. I thought my job was only driving; now I realise my job has to do with education.
– Juan Hernández, Pearson truck driver, Mexico

More than a decade into the 21st century, according to the United Nations, there are still 781 million adults lacking basic reading and writing skills and 290 million young people (aged 15–24) neither working nor studying. We need dramatic and significant innovation to tackle these pressing challenges and a new focus on learning outcomes from one based on inputs to far greater emphasis on outcomes; what Pearson, the world’s largest education company, calls “efficacy”.

I joined the company in September 2011, to create a team which, among other things, would drive this change of emphasis in the company. Placing efficacy – whether our products have a measurable impact on improving someone’s life through learning – at the heart of the business is challenging.  It requires greater rigour and emphasis on outcomes, and a solid evidence base. We used four broad categories of outcomes that we wanted to impact: access, completion, achievement  and progression.

It is worth pausing to consider how radical this idea is. It is not just that education systems often still work to qualifications and/or time-served (or “spent time” as it is called in the US) rather than proven outcomes; it is also that this is radical in business terms, a major step beyond simply controlling inputs and outputs.

During those early days, we came across Jugaad Innovation, a book by Jaideep Prabhu, Navi Radjou and Simone Ahuja, which inspired our thinking. The book describes jugaad as: “A colloquial Hindi word that roughly translates as ‘an innovative fix; an improvised solution born from ingenuity’. Jugaad is, quite simply, a unique way of thinking and acting in response to challenges; it is the gutsy art of spotting opportunities in… adverse circumstances and resourcefully improvising solutions… it is about doing more with less.” The characteristics described by jugaad – being resourceful, gutsy, innovative – were those to which our small team aspired.

It has been a challenging and rewarding journey and, in November 2013, Pearson made a commitment to report publicly by 2018 on the impact that its products and services have on learners, through an external audit, just as it does in relation to its financial outcomes. This commitment is the first of its kind in the education industry and drives our global business. We are moving in that direction starting now.

When we began, there was theoretical commitment to efficacy in Pearson but also scepticism about its benefits and  questions about how it could be turned into a practical proposition. Would it be seen as just another burden and would it get support from every business leader? What practical steps could we take to ensure that the idea of efficacy made a difference to the way we redesigned products and built relationships with customers? The process we went through in addressing these questions and more taught us much  about the nature of institutional culture and how change can occur.

In developing solutions to address these questions, we were determined to design and implement something that would not drain significant time or resources. We wanted to avoid creating bureaucratic and expensive top-down processes,  through simultaneously engaging Pearson’s executive team, and the hearts and minds of frontline employees. This has allowed us to begin to unleash the passion of our employees to deliver evidence-based solutions to pressing challenges in education.

Achieving initial scale

To start, we needed a simple tool that could be understood by all levels of the company and therefore allow us to rapidly scale awareness and understanding. We therefore developed the efficacy framework, a tool that asks a set of questions based around four things, that we use to assess what likelihood a product or service has, to deliver measurable outcomes. First, we need to decide on the outcomes – what do we want to achieve? Criteria for this are the intended outcomes, the overall design and value for money. Second is evidence. What evidence do we need to show to support the outcomes? We judge this by looking at the comprehensiveness, quality and application of evidence. Third comes planning and implementation, where we ask what mechanisms and systems are necessary to deliver the outcomes? This is done by forming an action plan, looking at governance and judging monitoring and reporting. Last is the capacity to deliver these outcomes. This is measured by assessing internal capacity and culture, user capacity and culture, and stakeholder relationships. The fourth point bears further comment. Ensuring the product delivers outcomes for students depends not just on the capacity of the relevant Pearson team but also on other key actors such as university lecturers, school district officials and teachers. This means we, Pearson, have an obligation to ensure they use our products in the right way.

We also developed the efficacy review: a process that prompted teams to apply the efficacy framework to their product,  with assistance at first from our team and later from  the dozens of efficacy reviewers we trained in the process. This enables teams to decide on improvements that will  increase efficacy.

The culture of these sessions was built on a transparent and plain-speaking ethos that resonated with participants. Over time, we expect to be able to predict with increasing accuracy what factors would result in what intended outcomes, so that we could harness the predictive power of the efficacy framework.  The dialogue often reconnected Pearson employees with their original motive for joining the company.

So we had a tool and a process. We needed to put them to work. Instead of developing the perfect strategy before we executed (by which time it would have be outdated anyway), we started with something reasonably formed and then refined as we implemented and learned.

As a result of this approach and the simplicity of the tool and process, efficacy spread quickly. Teams uncovered how it could help them deliver outcomes. By the summer of 2012, we were able to report to the executive team on what had been learnt, resulting in increased credibility and support. Our efficacy agenda was also significantly spurred on by John Fallon, who, when appointed CEO in January 2013, initiated  a transformation of Pearson that made it a single global operating company focused on achieving efficacy. 

To be truly transformative, it was not enough that we had built support at the executive level and among a network of  a few thousand efficacy enthusiasts. Nor was it enough that the new CEO was committed to the agenda. Success now required all 40,000 employees to embrace efficacy.

Reaching irreversibility

We had achieved initial scale and arrived at another phase of the organisational transformation: we needed to go beyond frontline employees simply understanding what efficacy meant. To achieve irreversibility, the focus on outcomes would need to be so deeply embedded and integral to what each employee did on a daily basis that reliance on a central efficacy team would eventually diminish. To achieve this, we focused on three major transformations: of product, culture and the wider industry.

First, in relation to product transformation, we continuously need to improve our goods to secure better outcomes. To this end, efficacy and research are now embedded at each stage of Pearson’s product development life cycle. A product team has to make explicit how it believes any given product will deliver significant learner outcomes. In addition, in all acquisitions and investment decisions, efficacy considerations are central, while all senior leaders have efficacy goals among their performance objectives. We are also strengthening our data systems, which are critical to measuring learner outcomes.

Second, we set out to transform the culture of the organisation so that efficacy was built into all of the company’s processes and became a day-to-day reality. New senior leaders of HR and corporate communications, among others, led this work. We created the Efficacy and Research Council, which consists of efficacy and research leaders who report into Pearson’s matrix structure and have dotted line reporting into our central efficacy team. In 2015 this Council needs to become more a driver of implementation and less a place of dialogue.

Starting in 2013, we also activated employee capacity in various ways. This included the creation of five e-learning modules to scale understanding of efficacy to a wider audience than just those our efficacy leadership could reach: over 20,000 colleagues have completed one of the modules, ‘Efficacy for Everyone’. We shared practical ‘efficacy stories’ of how others were changing aspects of what they did to impact learners and we were active on the internal company portal, answering questions and posting new content continually. In 2015, at our leadership summit for the company’s top 150 people, we focused on efficacy and how it could become a practical daily activity for everyone. This was live-streamed publicly to our employees across the company, building further connections to reinforce the spirit of openness which infuses our approach. In addition to the top 150 people, the summit involved  a sample of employees from across the company, building further connections.

Finally, we wanted to influence the broader industry, opening up a dialogue with the education industry about how to improve learner outcomes. There is much that we need to learn as our own understanding of efficacy continues to evolve.  In our dialogue with stakeholders and experts in the field, we have received varying feedback that has helped to open up conversation on the education industry’s shift from inputs and outputs to outcomes. We have made public (and freely available) the efficacy framework and other tools we have developed as well as a publication, The Incomplete Guide to Delivering Learning Outcomes, that sets out how Pearson has gone about the organisational change.

The external feedback has also been helpful in enabling us to refine our own approaches. So for example, Michael Feldstein, a leading US education blogger, has been running focus groups with the academic community to obtain feedback on the efficacy mission as it relates to higher education in North America. More broadly, product and customer-facing teams are testing the intended outcomes of individual products with learners in order to incorporate their feedback into the efficacy process.

Ultimately our goals are, on one hand, to supply better products that deliver outcomes for learners and, on the other hand, to encourage customers and learners to demand better outcomes from us, and indeed, other businesses.

In just two and a half years, the efficacy movement has gained significant momentum at Pearson; from piloting the efficacy framework in 2012, to committing, in 2013, to report on the learner outcomes of our products in an externally audited manner, to scaling efficacy across the company in 2014. As a result, outcomes, metrics and targets have been defined for our ‘first wave’ of priority products and efficacy embedded in critical processes such as the product life cycle.

There is still a long way to go. 2015 will be the decisive year for implementation. We must build capacity to design impactful products and also build new kinds of customer relationships. To achieve our goals, this will be the year that we solidify the kind of culture at Pearson, where efficacy is not an add-on to business as usual but rather is business as usual. As always, we welcome feedback and conversations but hope that the process itself gives some insight into what can be done together, if we are to change learner lives for the better.

Efficacy and research are now embedded at each stage of our product development life cycle