The monoliths and the scholars

Choose Innovative Technology

by Jon Pither

Published 2017-06-06

The Singularity

People talk about when the Technological Singularity will happen - when the software finally takes over and plots it own course - but I think it already has happened at the large IT institutions. Systems above a certain size last for decades and will nearly always outlast their developers who will rotate out in shorter time-spans. Developers become a source of food; supplying the new bolts and rivets of functionality that the big software beasts require for growth.

The type of developer needed to service these amorphous monolithic entities is of a particular breed. It's not the high octane keyboard warrior of bygone lore, rather, the mould that fits is that of the patient developer, the archaeologist and the scholar.

Developers need to first understand where a system sits in the complex ecology of other systems and of what ideals drove the original design. They need to submerge themselves deep into the code and to surface back up, repeating this ritual many times until they can see where the fault lines lie between thoughtful architectural design and hastened business pragmatism.

The scholars are the ones keeping the large IT intuitions running. They are ones with a grasp on the past, present and future.

The Fragile Scholar

Ten years ago it wasn't irregular for highly esteemed programmers to use Windows as their primary OS. Now in the present day, it's much more of a rare occurrence, save for the large IT institutions.

Large IT shops are not a natural attraction for top tier developers. Developers who want to be productive and to solve problems, do not want to spend their time traversing a huge bazaar of old tech with random scatterings of workarounds, hacks and repairs on display. No one savours getting dragged into a bike-shedding discussion over which version control system to use, when this decision was settled in the world outside long ago.

However, for the scholar who can remain unpickled by the institutional brine 1, for whom they have the necessary freedoms and political air-cover, then their work could be immensely satisfying. They would be the ones solving the truly difficult problems and making a real difference.

The challenge posed to the institution is to keep hold of them.

Innovation

'Surely' - the manager might say - 'it is all about having the right people and the right processes in place. To make our company an attractive place to work by having good compensation and career opportunities'.

Viewing technology choice as a detail - an afterthought - is a fallacy. It is technology that will be necessary to attract the calibre of people who will be able to make a difference. It is technology that will help to retain the existing scholars and employees who might otherwise wander. It is emerging technologies that will enable the scholars to disrupt the monoliths, be it Microservices, React front ends, event sourced architectures, infrastructure-as-code devops or GraphQL 2.

Innovation of technology can also help relatively successful projects to avoid an eventual heat-death, where the good people slowly dissipate and the tech starts to looks old. Innovation keeps projects fresh, and keeps the scholars interested.

The Challenge

The challenge we face as industry is to grow the frontier of senior managers who recognise the virtuous cycle of employing both good people and innovative technology.

At large institutions there will always be limitations and compromises, but there will also be the huge challenges that a certain creed of developer will want to scale.

The best managers recognise this, and they champion investing in using innovative technology choices to start solving the large, seemingly irretractable problems. They desire to bring the dynamic 'professional hacker culture' directly into their own institution to shake things up and to inspire.

Given support by capable and positive management, the freedom and steer towards using innovation technology, the scholars wil be able to have their way against the monolith.

  1. Gerald M Weinberg's The Secrets of Consulting
  2. See Paul Graham's beating the averages
submit to reddit