This week's big shifts on welfare policy are an unavoidable reconciliation with reality

Without much warning, this is turning into a momentous week in British politics.  The Party which did more than any other to expand the welfare state and uphold its core principal of universalism has signalled its desire to move on. By proposing a cap on welfare spending while accepting the removal of certain benefits for richer families and pensioners, Ed Miliband and Ed balls are moving their party and the country into uncharted territory.  In short, a new political consensus has rapidly emerged on the welfare state which accepts a radical break with the status quo.

For many this will be a moment of deep sadness. Indeed, it is difficult to look back on a century of policy making motivated by the noble idea of shared contributions and shared benefits and not feel rueful about what is being lost and anxious about what may be to come. The truth is, however, that Labour’s shift is an unavoidable reconciliation with reality.

Like all with strong political views that run against the grain of public opinion, those who hold a torch for the welfare state have convinced  themselves that voters simply fail to understand the nature of welfare largely as a result of the misinformation spread by the press.  There is certainly evidence that the British public do consistently overestimate the size of the welfare bill and how it is spent. But those who cling to this analysis need to recognise that these popular views are indicative of a deeper shift in public opinion driven not by The Sun and The Daily Mail but by profound changes in British society.

The welfare state went through its period of greatest consolidation in the late 1940s and 1950s.  This was a time of unprecedented social solidarity resulting from the shared effort and trauma of World War II. As the generation that fought in that conflict and built the post-war state has grown old and died so have their values. The sense of national community that was so important to the creation and acceptance of the taxes and structures that underpinned the welfare state no longer exists to nearly the same degree.

The paternalism that also informed the creation of the welfare state has also disappeared almost entirely. The belief that individuals and their families could look to a set of institutions to protect and support them throughout their lives was hard-wired into British society.  The offer of cradle to grave support from the state mirrored what many people expected and received from their employer, their church and their extended family.  The welfare state was created in an era of lower geographical mobility and where many worked at the same job for the same company for their whole lives.

This is not our world today. The proportion of self-employed has risen (PDF) from 7% of the population in the early 1970s to 14% in 2011.  In 1971 there were around 800,000 small businesses in the UK, today it is closer to 5 million – 40% of that rise has occurred in just the last decade or so (PDF). In addition, the employee relation with their employer is no longer so cosy: the latter offers less benefits and security and the former offers less loyalty in return.

As a result, we live in an economy where people are required to fall back on their own resources and live more independent, self-determined lives. In such an environment, it is hardly surprising that the paternalistic welfare state has lost its resonance and those who are seen to rely too heavily upon it are derided.

And yet despite this erosion of the principles that underpin the universal welfare state, the size of the welfare bill itself has remained largely unaltered since the 1950s (although it has grown very substantially if one includes pensioner benefits). In effect, we have a welfare state with a structure and a scale that has varied little over six decades even though the social values and institutions that once underpinned it have changed beyond recognition.  It is this, not scurrilous stories in the press, that explains the well-evidenced and long-term decline in positive attitudes to welfare.

However, we do live in a world where people still need support and help during the difficult periods of their lives. The great majority of the unemployed are not ‘scroungers’ but are desperate to get back into work. There are regions of the UK where joblessness has been endemic since the 1980s – their revival will not happen overnight however smart your growth policy may be.  And in an economy where creative destruction is a permanent feature, short to medium term unemployment can never be eliminated entirely.

The rational response to this is not to gnash teeth and wail at the loss of a welfare state that no longer has sufficient public support but to use theopportunity presented by Ed Miliband to engage in a serious conversation about what social security should look like in the 21^st Century.

Some, including Miliband see a central role for the contributory principle. But there is also a wider debate to be had about the part that mutual associations, civil society bodies, community groups and insurance schemes can play. Appropriately, welfare will itself have to become a feature of that more innovative, fluid and diverse society and economy which has been the most profound factor behind this week’s events.