The 3 paradoxes of cloud native platform engineering

Cloud native computing has come to dominate enterprise IT for every organization that requires dynamic, flexible infrastructure at scale. 

Kubernetes and the rest of the cloud native ecosystem Kubernetes enable companies to build and deploy applications faster, more reliably and at a lower cost but can be complex and difficult to master. The promises of speed and scale may seem out of reach when development teams get caught up in the details. To build such software, developers require all the help they can get. 

DevOps practices and tools are essential but are merely the starting point. To succeed with DevOps in cloud native environments, organizations hope to build out internal developer platforms (IDPs) following still-nascent platform engineering best practices.

The goal of platform engineering is to improve developer productivity. And if development teams are productive, then they should be able to deploy software at the velocity and scale that the business requires.

Even with IDPs in place, however, the goals of cloud native development set a high bar for the organizations that seek its benefits. For every step forward, they seem to take one step back. 

An excessive focus on developer productivity can backfire. Platform engineering can counterintuitively drive developers away. And the more complex cloud native deployments become, the more difficult they are to manage.

This article – the first of a series of three – will explore these paradoxes. How can organizations achieve the business goals for their software efforts when so many pitfalls threaten their success?

Everybody agrees that developer productivity is a good thing. The more productive developers are, the more software they can deliver for every dollar their organizations pay them.

The problem with developer productivity, however, is how to measure it.

McKinsey tackled this question in a recent controversial article, “Yes, you can measure software developer productivity,” in which it theorized that such measurement can improve software development outcomes.

In particular, McKinsey posited that when developers focus their activities on building, coding and testing software, they are being productive. Other “soft” activities, including planning, thinking, mentoring and architecting, suck away developer productivity.

Given this perspective, it’s clear why developers are pushing back. Any productivity measurement approach that follows McKinsey’s recommendations will entirely miss the fact that developers who focus their efforts on the “soft” part of their jobs are in truth more productive and valuable to their organization overall as compared to their colleagues who spend all their time with their hands on their keyboards.

Complex software initiatives like cloud native only exacerbate this disconnect over developer productivity. In the cloud native mindset, the most productive developers are the ones that focus on what makes for dynamic and scalable software that meets the business need, independent of how much time they spend coding.

Developers don’t like to be measured with vanity metrics like number of pull requests. To optimize productivity, it’s best to let the developers themselves decide which activities they should focus on.

Given the plethora of DevOps tools on the market, assembling the optimal toolchain can slow everyone down and lead to inconsistent results.

The solution: ensure platform engineering teams build an IDP that includes the best set of tools for the tasks at hand. The goal of such a platform is to provide a “golden path” for developers to follow, essentially a recommended set of tools and processes for getting their work done. 

However, this golden path can become a straitjacket. When this golden path is overly normative, developers will move away from it to get their jobs done, defeating its purpose. 

As with measuring their productivity, developers want to be able to make their own choices regarding how they go about crafting software. 

As a result, platform engineers must be especially careful when building IDPs for cloud native development. Jumping to the conclusion that tools and practices that were suitable for other architectural approaches are also appropriate for cloud native can be a big mistake.

For example, observability tooling is an important component of any IDP, but most observability tools are a poor fit for the needs of cloud native developers. Instead, an observability tool like Chronosphere running on Google Cloud will give cloud native developers the insights they need to do their jobs, even in complex and dynamic environments.

This cloud native mindset brings an architectural focus to the work of developers. 

Cloud native developers build individual microservices but must also keep tabs on the behavior of pods in clusters and clusters in fleets of clusters. In other words, they must focus on the forest while simultaneously focusing on the trees.

Developer productivity depends upon this dual focus. Any successful platform engineering effort does as well.

How, then, to resolve this forest vs. trees paradox? The answer lies in the data. The only way to maintain an appropriate focus on the big picture while developing microservices is to have a grasp on all relevant data about the performance of the cloud native infrastructure.

Too much data, however, is worse than too little, which is why cloud native observability from tools coupled with the cloud native mindset that Google Cloud brings to the table are essential for achieving the business goals of cloud native computing.

Resolving these paradoxes depends upon achieving the right balance.

Too much focus on developer productivity is counterproductive, but too little will leave areas for improvement unexplored. The answer: Listen to your developers to strike the right balance.

An overly constrained IDP will lead to developers finding ways around it, defeating its purpose. Building an IDP that follows the cloud native mindset will help to achieve the proper balance.

Encouraging developers to grasp the big picture of cloud native computing at scale is essential for achieving the goals of the software effort, but without the proper observability data, they will never find the balance between this big picture and their day-to-day work of building and deploying microservices.

The entire notion of a cloud native mindset, therefore, comprises all these balancing acts. Listen to your developers, give them the right data and let them find the balance.

Copyright © Intellyx BV. Chronosphere is an Intellyx customer. None of the other organizations mentioned in this article is an Intellyx customer. Intellyx retains final editorial control of this article. No AI was used to write this article.