For IT Project Success, Consider The Wider Competition, not Just Today’s Fixture...
"One reason we have become so short-term focused is Agile and how easy it makes it to spin projects up in the cloud..."
With the Women’s Football World Cup just finished and the Rugby World Cup now underway, I’m often struck by the constant need to meet the current game’s objectives while also keeping your overall tournament campaign on track.
After all, as the coaches, managers, and fans, learn over time, winning the battle is not the same as winning the war: ‘That was a great tackle, but we’re still 10 points down and 60 metres from the try line!’
It’s also clear that in your first match you can get away with a simpler approach, and maybe take a risk or two. This isn’t something you can get away with for those crunch matches later in the tournament when you need to consider injuries and conserving energy.
Basically, it all boils down to short-term gain versus long- (or at least, mid-) term (potential) pain! Similarly in database architectural design, we should see we have a similar trend.
Agile! Multicloud! Microservices!
In essence, in the short term we’re committed to great things like Agile, we’re fantastic at strategy, we’re going multi-cloud and geolocated, and on the tactical side we can spin up a whole new set of microservices to solve problems. Many of us can also choose to substitute in a cloud database (often a version of PostgreSQL) to make delivering a great new app to the business quick and easy too.
But I don’t think we’re as consistent at thinking about the full ‘World Cup’ campaign as we should be. We can win the immediate match and solve our immediate problem, but may have had our star player red-carded out of the rest of the competition… By which I mean, (with my final egg-shaped ball analogy) we could end up relying on a new app but find in six months that we’re locked into a database that needlessly limits us. As a result - we don’t bring home the silverware!
Agile is great—but has it made us short-sighted when it comes to long-term problem solving?
With busy release schedules I worry that we’ve lost the past IT habits of mid-term scoping and forward thinking. I think it’s time we started bringing it back.
One reason we have become so short-term focused is Agile and how easy it makes it to spin projects up in the cloud. Don’t get me wrong, I don’t want to go back to an outdated Waterfall-based software development ‘methodology.’ We were right to move to Agile as sprints, scrums, DevOps approaches, and all the rest are so useful. And let’s be honest, pre-Agile IT had badly blotted its copybook in the eyes of business users by requiring absurdly long development cycles and a default ‘no’ to new ideas.
Now, we (almost) always complete projects on time and within budget. Modern developmental approaches have greatly increased success rates in software development, improved quality and speed to market, and boosted the motivation and productivity of IT teams.
Nonetheless, it’s still easy to rush into a project without serious consideration given to what happens (or could happen) if it is successful.
I mentioned PostgreSQL earlier, as it is often the tool of choice here, and for good reason. Over the past few years, PostgreSQL has steadily increased market share. People adopting a new relational database are more likely to pick PostgreSQL than anything else. PostgreSQL currently has a 17.4% market share in relational databases, and is seen as a safe choice for a new build.
And it is a great choice. But, despite it being open source and a fantastic alternative to proprietary SQL databases, or indeed other technologies that don’t work half as well as PostgreSQL does in the cloud, there are many flavours. If you choose the vanilla option, there may be consequences.
You don’t think about this when your team runs out and your priority is meeting that immediate project need (the first game of the season). Quite rightly, you’re being tactical; talking to the business, identifying its needs, breaking it up into small chunks that you can deliver quickly and independently as microservices, each one adding a little bit of value.
And all these tiny services don’t need huge databases… at the start. But as a good business app becomes more important and a focus of interest from the business, they will want to get as much value as they can.
We work with a major US retailer where exactly this happened, the team started building microservices and were very successful in delivering and building a richer and richer portfolio. Before long there was a huge ecosystem of little microservices, some of which needed to talk to each other and share data, some which were dependent on others.
Soon, one team was finding that to do their bit it had to integrate with other microservices, and so on. It all got very complicated, especially as the business demanded more services and additional features that delivered the same data to more customers.
When the business starts buying into you more and more, you need to rapidly figure out how to scale all this to keep everyone happy.
If you failed to plan ahead and only thought about winning this game (“We have to deliver this amazing new ecommerce platform,”) or the next one (“We need a service to de-dupe copies in the catalogue on the screen the user sees”) but not about how to get to the Final (“This could really take off. Let’s make sure we aren’t tied into a database that will start creaking in six months’ time”), as then you’re going to end up with a lot of tricky technical debt.
This means you’ll have to work much harder than you planned to keep your system patched up and (just about) working, rather than efficiently and organically scaling on demand.
This is more than a user story; think of the bigger epic it could become
Luckily, Agile is our friend here. Used correctly, it absolutely supports operational IT design. You should be thinking beyond the user stories, to the bigger issue—where your app could end up in the mid to far future (as in the epic approach).
So, my advice is to never neglect the bigger story your user stories are part of and always look at the wider campaign, not just the next game. Best practice suggests this will ensure your wider strategy is fully and effectively delivered.
To sum up, don’t rest on your laurels if you and your team have won one or two opening IT games. Instead, keep your overall campaign objectives in sight and do some longer-term planning to ensure the silverware is displayed in your cabinet come the end of the season!
Meanwhile, I’m ready for an excellent few weeks of first-class Rugby, though being an England fan I’m also prepared for the proverbial early bath too.
