One recurring theme of this website is Consequences, and where they come from.

My main conclusion after scrutinising the human race for some 50 years is that this is the hardest of all ideas to understand: (i) that, yes, there was a time when what you see around you did not exist, and (ii) that the way things look now reflects choices made for more or less good reasons many years, sometimes even centuries, previously. 

Another way to put it is that we ascribe to things existing now raisons’ d’être reflecting our own prejudices or hopes, neglecting the true reason why they are the way they are.

Take, say, democracy. Or the mass media. Both have the current form they have for reasons of information management going back deep into the past.

It was never technically possible to consult voters quickly and measure their opinions, so it is no surprise that centralised top-down, elites-know-best principles and institutions emerged as the way to make democracy work, much to the satisfaction of both Left and Right. Does that still make sense in a world where voters can be consulted in real time? Not really. Not that flash-mob rule is any better. But the current supposed balance between rulers and voters is inherently unstable.

Likewise as it became possible to centralise expensively huge printing-presses, what happened? Huge centralised newspapers displaced ad hoc pamphleteering. TV technology was difficult and expensive to run, so at first only a very few TV channels emerged.

These oligopolistic advantage having prevailed over many decades, we all start to see them as somehow ‘normal’. Whereas they are only an unmissable target waiting to be toppled as IT empowers citizens and erodes their fat unwieldy structures.

Adapting to change presents large organisations with horrendous problems, especially where IT is concerned. How to stop mid-stream what the organisation is doing and change processes and goals to what is now possible, as opposed to what fr years has seemed essential and successful? The Foreign Office, by most bureaucratic standards a fairly small if highly dispersed organisation, has suffered mightily with managing Change of this sort.

Nowhere is this dilemma more acute than in banks and healthcare. Banks and hospitals literally have to keep going day and night lest terrific damage be done to key customers. So the temptation is to do only what is capable of being done simply and safely to improve processes behind the scenes, hoping for the best that somehow the operational contradictions and missed opportunities that this improvisatory policy represents will be containable.

That often works well enough in the short term. But as time passes the internal difficulties of running improvised systems start to compound up, and the whole structure becomes more and more precariously unbalanced.

All of which is a rambling introduction to this truly magnificent piece by Frances Coppola about the legacy systems problems at RBS bank:

The existence of ancient “legacy” systems within the modern banking systems architecture is not necessarily to do with lack of investment, as Alastair Winter suggested, though fast growth and acquisitions complicate IT architectures and can make systems vulnerable. I shall return to the likely effect of RBS’s aggressive expansion strategy shortly.

But the real problem is the size, complexity and criticality of these old systems – plus the fact that many of them are written in progamming languages that are not widely used now, so there are skills shortages among IT staff. Many of these systems are also poorly documented (comments in code were something of a rarity when these systems were written) and their functions are poorly understood. Replacing them without affecting functionality is therefore not an easy task.

Even replacing a single program can have adverse effects if the program is not properly understood, as I discovered when my team replaced a start-of-day batch program in a systems upgrade on one occasion: the old program was complex and poorly documented, we (perhaps inevitably) failed to understand exactly what it did and we therefore subtly changed its functionality without realising it.

Fortunately the changes we made didn’t cause major problems, and it wasn’t a major retail banking system anyway. But imagine that, scaled up to an entire suite of retail banking applications running millions of bank accounts, with trillions of transactions going through every day? No wonder banks have shied away from replacing these systems. The risks, and the associated costs, are terrifying

And so on. Do read it all, and marvel at the way she lays out these issues so deftly.

In other words, when we stand aghast at the ‘problems of the NHS’ or the ‘problems caused by banks’ we tend to complain only about what we see and what we can more or less understand. Those things indeed may be bad enough.

But underneath them are indeed staggering technical issues of command and control and information management that may not be properly understood by anyone at all.

The issue is not ‘too big to fail’. It’s ‘too big and too slow to be able to manage change’.

And fail they do. Where ‘they’ equals banks, healthcare systems, USSR, nation states, EU, one-party rule in the Middle East, newspapers, and a very long list of other hitherto solid-seeming phenomena.