Talking observability data and costs with Data Protection Gumbo

On December 6th, I shared my experiences on the Data Protection Gumbo podcast with its host Demetrius Malbrough. We had a nice chat about my journey into cloud native observability (o11y) since joining Chronosphere.

In episode 171, “Your First Steps in Cloud Native Observability,” we covered a bunch of questions around my O11y Guide article series. These articles take you along on a journey of my learnings as I go from an application development background into the world of cloud native o11y.

We covered several observability topics in the 25-minute podcast, which you can listen and subscribe to on all major platforms.

A podcast is always interesting, as you join a virtual call on video, then record the session. In this case the video was used for a small clip to promote the episode going live and was shared on social media channels.

The host, Demetrius, had stumbled upon my blog series called the O11y Guide and wanted to have a chat about my experiences around the observability world and how that applies to the data angle of his podcast.

For the full conversation, I’ll leave it up to you to jump over to the podcast and give it a listen!

We started off examining what cloud native observability means to the listeners, and I gave a bit of background sharing my journey from application development to observability.

Next, we transition to the 3 pillars of observability and talk about if they are really relevant to modern cloud native observability. I talked about how you can’t just hold on to the technology views in monitoring and expect to be able to massively scale in a cloud native world.

This requires a business view of how to achieve your goals without focusing on the technology used. We covered the 3 phases of cloud native observability; know, triage, and understand which involves the process used by observability teams across the globe to solve problems in their cloud native environment and focuses on outcomes rather than underlying data formats (logs, traces, and metrics).

Demetrius then asked how data connects to cloud native observability. I shared some thoughts on the massive amounts of data that cloud native platforms generate at scale and how that’s becoming the biggest roadblock to organizations succeeding in the cloud.

To answer, I pointed out a very shocking fact: That organizations are paying more for their observability data than for their income generating business application and customer generated data.

Next, we covered what a sample application in a cloud native environment would generate as an observability tooling target. This led to a discussion on average data retention rates and the cost of unused data that’s being collected and stored in the cloud.

He then asked me if companies are paying to retain all their observability data and if so, what can be done to reduce some of those costs?

I shared an architectural overview of Chronosphere’s product, which includes a control plane that gives users the ability to control the amount of data after ingestion, but before storage to save on unused data collection; our customers are saving around 50% on their data storage using this control plane.

We then cover the players and teams that Chronosphere interacts with in organizations such as SREs or observability teams. To finish out, we touch on how organizations start out on open source observability tooling — where open standards compatibility is very important — but then find out that such tools are very complicated to maintain and scale in the cloud native world.

If you have something interesting to talk about in the data world, reach out to Demetrius and he might have a slot for you to join his podcast!

To learn more about how you can use Chronosphere to save costs, request a demo.