The surprising link between Formula One and enterprise PostgreSQL optimisation
There’s a key reason only a few FI drivers become great, says Yugabyte EMEA Field CTO David Walker; it’s because they keep learning from their peers. And there’s nothing to stop you following their example for your data engines!
If, like me, you struggle to see a spark of humanity in Max Verstappen, think Daniel Ricciardo is amusing as a character in Drive To Survive but less talented on the actual track, and Pierre Gasly is rather, well, ghastly, it’s Lewis Hamilton you’d probably end up rooting for as the 2024 F1 Season picks up the pace.
Although Verstappen’s dominance seems almost as locked-in as the lad from Stevenage was in the 2010s, I still hope Lewis can reclaim at least one more World Championship. And he’ll do the same way he’s always progressed, by closely studying what his (currently) more successful peers are doing.
His father’s support was critical in those early days, but Hamilton also achieved Karting results so impressive that Ron Dennis of McLaren noticed him. At just 22 he was racing in F1, driving the fastest car on the track.
He earned his stellar start, and subsequent successful career, by learning fast. Players from other industries do this too; in our world, it’s enterprise IT teams at brands that want to push the technological envelope.
This is because IT shares one very useful thing with racing: common standards. If you look at the development of drivers like Lewis Hamilton, from Karting up to Formula One, they keep both competing and collaborating with their peers. By always working against a common defining standard, they naturally work to improve within that standard. This means there’s a lot of communication, a lot of development, and everyone learns from each other.
You need to see the latest way to get ‘database downshift’
I see a similar pattern in my area of focus --making applications and services based on scaling PostgreSQL. As more people adopt the Postgres API, more teams are having to find the best way to make it work for their business. There are multiple techniques for doing this.
But, getting it right in practice can be tricky. Most developers think scaling/optimising Postgres is as easy as just taking one of the (many) available options, then scaling it, deploying it, and comparing this with the performance they were previously seeing to find out if they now get better results (or how fast you can take advantage of downshift in an F1 machine).
This quickly reveals that to achieve full optimisation you need to get the sharding right, and understand the topology of the network, such as where you're going to put your servers, etc.
So, your team must do some book-learning to make the changes required to accommodate that scalability; otherwise, you're not going to achieve any meaningful, business-impacting change.
But where do you get that information?
If it's a big and mature product, like the Oracle databases of this world, there's lots of people out there that you can speak to (or hire). If you have a proprietary or niche product, you're reliant on the small engineering team at your partner, and how much information they've put out.
Somewhere in the middle is the open source world—which, luckily, happens to be where most people who work with scalable PostgreSQL hang out! This is where you—the figurative Lewis Hamilton of my analogy—will go to learn, just as he did in his F1 apprenticeship.
There may not be masses of documentation available—but you will be able to talk to an active community who use Postgres (or whatever it may be) and have experience running the same tests and benchmarks you want to do.
You may be surprised how openly people will share data on how many transactions per second they can get with what size database, how many nodes you will need—and most critical of all, what are the ‘gotchas’ that you need to avoid.
Soon, you’ll be able to pick out two important things: What is unique about our problem that we need to test, and then, what can we do better than everyone else?
To understand the new stuff, hang out with the kids building it
In open source, like every level of motor racing, we are in a world where when someone innovates, people immediately see resultant performance improvements. You may not see everything, but you observe enough to give you an indicator of where you should go.
My advice then, is to explore the roots of the open source platform you want to exploit. Don’t ignore the vendors who are there to help, but explore what your fellow ‘racing car’ drivers are up to, and discover what the smart kids are doing to get the most out of their kit.
Follow this example and it shouldn't be long before you too find yourself competing for the top spot on the leaderboard!
