Why platform engineering leads to a better experience

This blog: 

• Discusses challenges and limitations that can arise with traditional DevOps practices
• Explains why shifting to platform engineering can provide significant benefits
• Picks up where where we left off in the previous blog in this series on Effective Platform Engineering: What is platform engineering?

Now let’s dive in again.

By traditional DevOps we mean: a development approach that emphasizes collaboration and communication between development (Dev) and operations (Ops) teams, and that values automating the configuration of infrastructure. This is not a bad value statement. But better collaboration and communication has always been a visible goal (or challenge).

Platform engineering vs DevOps

What changed was the increased emphasis on using software to manage infrastructure. And as with software development in general, it is the way in which those communication and collaboration boundaries function that will define the issues and possible solutions.

Understanding these issues and solutions is important because it demonstrates how platform engineering can enhance efficiency, standardize processes, and improve overall software delivery by providing self-service tools and automated guardrails for development teams.

DevOps efficiency

DevOps as a practice, has revolutionized software delivery by giving teams the cross functional skill sets needed to manage the creation and delivery of their software end-to-end. However, we are now seeing issues with the practice that require a new way of thinking, and you may be seeing some of these issues in your organizations.

• As DevOps is adopted, the increased autonomy to deploy can lead to significantly increased costs as more cloud resources are provisioned for each team’s software lifecycle environments.
• Required skill sets in cloud technologies, infrastructure as code, and networking requirements (among others) can be hard to find and staff.
• The growth in complexity and the difficulty in finding the right skills results in DevOps Teams beginning to appear responsible for ensuring infrastructure is provisioned when requested (in a compliant way), accounts are set up, permissions for access are set appropriately, and so on, switch requests queueing up much the same as traditional IT.
• Whether called an Ops, DevOps, or Platform team, quickly this team can become overwhelmed and unable to keep up with the volume of tickets sent in, resulting in lengthy fulfillment times that slow down onboarding, innovation, and, ultimately, releases to production.

As DevOps teams expand in companies, another common issue arises: technology becomes messy. Figure 1.4 demonstrates how the DevOps teams are evolving over time.

Each team can choose how they set up their software (especially in the cloud), which leads to a need for more standardization. Some teams use serverless tech, others prefer Kubernetes, and some mix things up. This makes it hard to share knowledge across the company and slows the start of new teams because they have to decide on their tech setup from scratch.

Solutions like architecture review boards or central architecture teams can help, but in the midst of sprawl the effect is still more delay. Security and compliance rules become significant concerns as teams become more independent and release software more frequently.

To mitigate this, security teams make lists to ensure everyone is careful when setting up their software. This leads to new processes like Change Advisory Boards or security reviews, creating delays.

One way to tackle these issues is to task the very open-ended DevOps cultural goals with the very specific product experience goals of platform engineering (See Figure 1.3 in our last blog). This experience is a measure both of the time required, and equally importantly, the actions required.

An effective product does require interactions with the builder of the product in order to use. This means creating self-service tools that allow teams to handle problems instead of waiting for help. By giving teams access to standard patterns, tools and autonomous ways to use them, we can start dealing with the mess of tech problems that have built up over time. This also lets us focus more on making things better for developers so they can work on what matters most: building great products.

In a very real way, platform engineering is another way of saying product engineering. It’s not enough to ask, are we engineering good infrastructure implementations, but rather are we creating:

• A good consumer experience
• A good product experience for the developer using our infrastructure platform

By providing an efficient path to production and allowing teams to start a new service without needing to make infrastructure architecture, security and governance decisions, they can get started while being able to deploy to production quickly, even on day one.

As an engineering team grows and requires capabilities for internal team management, the ability to deploy and manage new infrastructure, and add additional capabilities for continuous integration and delivery (CI/CD), providing self-service APIs can make these functions available for immediate use with no waiting time for another team to execute.

With platform engineering, automated guardrails can be provided as part of the paved road, with security and governance teams moving from manual compliance verification to developing automation that can do the same verification with every deployment. The organization can quickly gain confidence that all processes are followed, and teams can become more efficient by focusing only on what is needed to make their new features work. For all of these reasons, more and more organizations are seeing the benefits of platform engineering practices.

We recommend introducing platform engineering concepts and the ensuing team very early in an organization’s journey. This might mean different things to different organizations.

For an established organization without explicit platform engineering thinking (product thinking), the challenges will be greater such as:

• Internal organizational structures can present barriers
• The difficulty of change can be harder to predict
• Time needed before experiencing the benefits is likely to be longer

Enterprises

For large enterprises, the time to begin platform engineering is after they first identify the most strategically valuable software development initiatives. It is often the case that Enterprises struggle to know which development efforts are returning value, which have the greatest potential for value, versus those that are merely routine maintenance.

It is not unusual for a large enterprise to first create a digital division — sometimes even as a whole new company — in which to gather the strategically valuable work and invest there in PE.

Digital native startups

For a digital native startup, organizations that started their journey using digital techniques to build the products as opposed to adopting them later on in their lifecycle, platform engineering may not be top of mind as you work on translating some critical concepts into products.

However, we strongly recommend that the strategy around platform engineering is considered part of your overall business and technology strategy, even if you cannot establish a dedicated team to do the work.

In organizations like this, we see this as an evolutionary path where the first step will be to build the most valuable technical capabilities as part of your domain-aligned team and then find your inflection point closely associated with your need for scaleups.

For teams that are deploying and operating software in a similar way this may include:

• Reusable modules for common CI/CD pipeline tasks
• Reusable infrastructure definitions
• Or alignment on tooling to use across the delivery ecosystem

In reality, most organizations have a standalone DevOps team that might execute software delivery functions as an activity that could be more cohesive from the developers building the product capabilities. For these organizations, platform engineering techniques will provide the maximum benefit.

Chapter 3 will discuss in detail setting up the Key Performance Indicators (KPIs) first and determining the absolute need for building Engineering Platforms in a platform engineering paradigm. (Zoom to chapter 3 by downloading the full Manning Book: Effective Platform Engineering)

As you learn about the concepts of platform engineering, we ask you to consider them as principles that can be applied in your work, whether or not you have an Engineering Platform at your organization or an initiative to create one. One may at first think that “Platform” implies large-scale or big organizations. Still, the concepts and principles of well-built engineering platforms and platform engineering practices apply to any organization or team of any size. For example, consider that self-service principles may be less valuable to a tiny organization or team.

Still, the principles of CI/CD, decoupling lifecycles, and creating repeatedly deployable environments are practices from which any organization of any size can benefit.

If you work at a large organization and are considering building a platform engineering practice, the Manning book, Effective Platform Engineering is for you. If you work at a tiny organization or startup, there are still lessons to be learned here even if the investment required in a PE team may not make sense until the company reaches a larger scale.

When you’re just starting a business and working on your first products, don’t stress about creating an engineering platform. It’s when your startup grows bigger that you’ll start thinking about ways to make things more efficient and standardize common tasks thereby bringing about economies of scale.

That’s when you’ll realize you need an engineering platform. In the Manning book, Effective Platform Engineering, you will still find a subset of principles to adopt to help your company grow and experience less overall scaling pain as your organization and customer base increase.

Next time we’ll talk about Foundational Concepts in Platform Engineering. Stay tuned or go ahead and download the Manning book, Effective Platform Engineering to keep reading.