Can open source survive the cloud?

Open source plays a critical role in most enterprises. DB-Engines’ top ten databases list shows that seven of the top ten are open source, and the likes of Apache Spark, Apache Kafka and Presto are vital for analytics, streaming and machine learning. Although these projects attract users and communities, are they being developed in a way that can be supported and maintained?

Open source relies on contributors — individuals willing to support and maintain a technology. Without the community giving input and insight , projects wither and die. The link between open source and this support system has been severed by the growth in cloud services. Today, you can enter your credit card and instantly access cloud services built on a range of open source projects. These include services from companies directly involved in the community, and from the cloud providers themselves. These services seem like an easy way to take the pain out of running data projects yourself.

Additionally, when developers make decisions, they tend to take the easiest route. Their focus is on developing and bringing applications to market more efficiently, and anything that lets them concentrate on software — rather than infrastructure or management — will be the preferred choice. Where developers go, the money follows. The result of this is that revenue which used to go to open source companies and community members for support now goes to cloud companies.

The first point to be made here is that open source is not a business model — it is a licensing model for software. There is no de facto direct link between a company supporting open source and its ability (or inclination) to monetize that open source software. Some companies have looked at changing their licenses to prevent competition from cloud providers using their open source versions, but this goes against the ethos of the open source approach.

Is open source itself the real problem?

Instead, we should encourage greater understanding around the value chain that exists around the cloud, open source and software. In order to provide something that customers want and are willing to pay for, open source companies need to demonstrate their value.

For example, new launches that offer Database as a Service (DBaaS) for developers can be incredibly useful, particularly if they support microservices, containerization and compartmentalization in application design. Soon, every application component will have its own accompanying database, tracking activity and logging requests. For database companies, this is a good move as it increases demand for their DBaaS services.

DBaaS can automate some of the process for developers by using container orchestration and Kubernetes. However, what happens when something goes wrong? The ability to restart instances in Kubernetes can come in handy, but this does not automatically fix all of the problems applications can encounter with database deployments. While the cloud makes things easier, it does not make them invincible from issues.

The value of open source comes from insight and expertise

DBaaS services are built to manage the majority of common use cases. They do not offer the same level of control, tuning or performance management that your own implementations give you. We might like to think that ‘one size fits all’ services like DBaaS can meet every need, but this is not the case.

Additionally, DBaaS services are more likely to be proprietary and tied to a specific cloud provider or version. You might be able to look at MySQL or PostgreSQL in the cloud, for example, but a DBaaS version of that database may not be completely compatible with the open source version. This is because it is a fork, built on and developed with features that aren't compatible with the original open source software. Understanding your options at the start can help you ensure your implementation is ‘fully open’ and can be moved easily in the future, rather than this coming as an unwelcome surprise.

From a developer perspective, acquiring database expertise may seem like an unnecessary overhead, not worth the initial investment. But, it can be essential for longer term application performance and stability. A full understanding of the issues around databases will still be required, even if you hand over some of the management side to the cloud. Knowing the right approach to use for your application will pay dividends over time, as it can help ensure that your costs don’t massively scale up due to prematurely upgrading instances. This insight comes from the open source community, not the cloud providers.

For open source to survive, we must ensure that developers understand exactly what the term means and how these projects remain sustainable. We also have to be sure that our collective efforts to develop open source communities are focused on the right things. By concentrating on the value side, we all benefit regardless of what takes place around the cloud. We can improve the market for everyone by encouraging users to ask better questions — from developers and architects through to software companies and the cloud providers themselves.

About Peter Zaitsev

Peter Zaitsev is the co-founder and CEO of Percona. An early employee of MySQL A -- one of the largest open source companies in the world, which was acquired by Sun Microsystems in 2008 -- he is a leading global expert on MySQL strategy and optimisation, as well as open source databases.