Ian Smith and theCUBE: Effective observability is more than software 

At KubeCon and CloudNativeCon 2023, Chronosphere Field CTO Ian Smith sat down with theCUBE hosts John Furrier and Savannah Peterson to discuss what customers are looking for in observability today, important considerations to have when developing an observability strategy, and what Chronosphere hopes to achieve going forward.

If you don’t have time to watch the video, you can find the full transcript below.

Savannah: Good afternoon cloud natives, and welcome back to KubeCon and CloudNativeCon. We’re here in the Windy City. My name’s Savannah Peterson, joined by my co-host, John Furrier. John, how’s Chicago treating you?

John: Doing great; I love this place. Love the weather here. It’s beautiful outside … It was stunning when I came in this morning — a little rain yesterday. Little wind.

Savannah: Shocking, I wonder where that came from. I know my hair’s holding up. I was worried about that here in the Windy City. It actually held up in that fantastic Daft Punk helmet that you may have seen. We’ve got Ian [Smith], the CTO of Chronosphere joining us – not wearing the Daft Punk helmet. How are you doing, Ian?

Ian: Great. It’s great to be back.

Savannah: Good. How’s the show been for you?

Ian: It’s been really interesting seeing everyone come back, and seeing the new crop of startups and all kinds of interesting perspectives. I definitely think in-person conferences are back with a mighty force.

Savannah: It’s a nice feeling, isn’t it? I think we all missed each other, and there’s nothing like doing business in-person. I know you had a lot of customer conversations — tell us some of the themes.

Ian: I definitely think there’s a lot of interest, obviously in OpenTelemetry, but it’s tipped over from the interest perspective to the: “We have to implement this now.” And [for] organizations, we’ve been waiting a little bit more enthusiastically with some of those 1.0 announcements, which are really important. Stability is key, right? But then, the mass adoption across open source technology and backends as well as vendors. But, there’s definitely a lot of: “How are we going to do this? How are we going into this brave new era?” We’ve been talking about cloud native for a while. Let’s start really digging into it.

Savannah: I feel like there’s a theme of maturity to a degree, and evolution both within the projects, as well as all the different companies. Observability, very hot topic. Do you think that customers are still nervous about Kubernetes?

Ian: I don’t think so, I think it’s become: “This is stable. This is something that we can derive potentially a lot of really great business impact from.” I even see excitement from what you might call larger enterprises or legacy businesses of: “This could be transformative, right? We can start attracting like this really next key batch of talent and start innovating and disrupting in ways that maybe were constrained when we’re running our own data center and all these other things.” So, microservices, architectures, on top of Kubernetes and all the interesting new technologies — definitely a lot of interest from smaller startups all the way up to the spectrum.

John: Ian, talk about the change with [artificial intelligence (AI)] now on the landscape. Everyone’s talking about their AI observability, its key to security, container security data coming out. You’re in the crosshairs of the AI wave. What are you seeing? What’s the real story there? What’s real, and what’s kind of hype at this point?

Ian: I think there are two parts to it. One is from an observability perspective. A lot of customers themselves are experimenting with observability and Generative AI, even building their own large language models. And so, they want to observe those workloads. I liken it to say, for example, when we adopt a GraphQL, [there is] a massive explosion of data needed to sift through it. It’s not just: “Oh, I need to store it.” But: “I need to make sense of it. I need to be able to highlight what matters.”

And people weren’t used to GraphQL at all. Like: “Well, I don’t actually know how to debug these things. So, there’s a learning curve of the vendors themselves helping to identify, but also the engineering organizations themselves. Like: “How do I optimize? What should I be looking for? And so, I think the same pattern is going to repeat itself again with a lot of this AI stuff. But then, there’s sort of a subsection, which are those organizations who are relying on a third party provider to do a lot of the heavy lifting for them.

Obviously, you know there are some pretty key names in the room. But, like anything, it’s an external dependency. You need to be clear about what’s happening in that environment. And then, if I do need to go raise a ticket or urgently call the CEO of my AI vendor or workload vendor, I need to be able to express exactly what’s happening and what I’m seeing from our end and sharing, right? Like, context and sharing so we can all work together. The second piece is that, for a long time, just with the complexity of observability, we’ve obviously had AIOps for a while. This new surge has been: “What else can we do with AI here? Will it supplant AIOps? Will it support these things? As things like query languages and data sets have gotten more complex, can they be more consumable? Can my inputs and my outputs be there?”

One of the things from a consumer perspective, particularly large enterprises is, and you mentioned security, there’s a concern of:

• How are these things implemented on the vendor side?
• Are we pulling things together too quickly?
• Are my queries being sent to a third party vendor?

So, there’s some concerns there. But I think in general, on the vendor side of things, trying to make sure that those concerns are well met and experimenting, not just rushing forward into that.

John: We’ve been seeing a lot of people say you have to have observability, and instrument things like software supply chain without observability. That’s kind of a standalone problem that’s not fully solved. And all these other questions come up I have to ask you, because the observability space was super overpopulated with vendors, and companies funded, and customers are clearly looking and voting with their wallets right now and choices. How does a customer figure out who’s a player and a pretender in the observability space? What are some of the key things that are table stakes, but also advantages for customers who are looking at figuring out what their observability play should be, right?

Ian: It’s a double-edged sword, right? Wide selection of options, but having to pick what’s right for you. I think that in past years, and even at companies that I worked for before, a lot of the mantra from the vendors, and the industry as a whole, was essentially that your observability strategy can just be tool selection. If I pick a vendor and they tell me the data that I should be generating, and they give me the pipeline, storage, dashboards and alerts, then that’s all taken care of. But things are getting more complex, right? Like all the things we talked about, Kubernetes, large language models; I would argue that observability strategy can no longer be just tool selection. There are a lot of choices out there. I don’t think there are some hard and fast rules, but you really have to look at it through the lens of: “How does my tool selection feed into my observability strategy?”

And by observability strategy, I even call out things like OpenTelemetry, right? Why is OpenTelemetry so exciting to people? It’s not just because it’s “cool” technology, but it’s the purpose that it serves. It allows you to own your data and do things that you couldn’t do when you were sort of a single vendor before. You might be able to take a whole bunch of your data, and put it in cold storage for compliance.

Observability companies generally aren’t super focused on compliance, right? You can go put some of it into one vendor, put it into another vendor, tie those things together, and now you have choice. And if I decide to move from one vendor to another one, my migration friction is a lot less. And so, that feeds into observability strategy. I can mitigate my risks. I can look past this next vendor selection and I can make sure that I’m setting myself up for the future. But all of this has to tie into: “What’s my technology and product strategy? What is my business strategy?” To come back to your question, what you really need to be thinking is: “What do I need from a vendor to support that strategy, and ultimately the business outcomes?” If I’m getting hard pitches about features and functions, maybe that vendor doesn’t have my bigger interest in mind.

John: The complex system now that is the enterprise, is distributed computing paradigms. The instrumentation has to be flexible and have optionality for the customer.

Savannah: Flexibility is such a big thing this week. One of the cornerstones of Chronosphere’s business has been cost control and value. How imperative is that right now?

Ian: Obviously in the current economic conditions, that is key. In a way, that’s something that’s been very consistent for us. And the market has really said: “Oh, hold on, that is important.” And to link it back to what I said before, for observability strategy, everyone’s probably heard some stories about a certain vendor, and a certain crypto company with a 60 plus million dollar bill. Is that a good example of an observability strategy? You might have had the money, but is it reasonable to think we will always have that money to be able to spend on observability? Maybe [it’s] not the most robust way.

Tying that back into all the things that we’re doing around cost control, cost control and value is a really important aspect of the strategy, which is: “What are we expecting out of observability and what do we need to trade off?” You can’t just spend money infinitely, and you can’t have every single feature and function and deal with every stakeholder. And that’s important because stakeholders can range nowadays from FinOps to the individual developers to the executive audience.

Savannah: You’ve talked a lot about observability as a strategy, but I’m curious, do you find that most companies have a robust observability strategy? Or are you also educating them on what that strategy should look like?

Ian: In the conversation I’ve had, particularly over the last six to nine months, I don’t hear the words “observability strategy”, but I start to hear things like:

• Could we have a conversation about the bigger picture?
• I like what you’re showing me from a product perspective, but what about data that you don’t take?
• What about data that can’t leave my environment? How should I be treating that?
• What about things before it touches your service? Are there ways that we could better optimize or utilize what we are doing?

Ian: In my role, I’m fortunate enough to be able to have those great conversations up and down the spectrum. And again, it feeds into, say for example, larger, older organizations who may be in the position of: “Hey, we have to move out of our data center. What do you need from observability to help support that?”

Ultimately, if you think from a top-down perspective, what you’re trying to accomplish as a business, it needs to feed in. There’s not a giant gap of business outcome tool selection — because you can’t go to a vendor and say: “Well, that’s all.” You need to become informed, come with a strong opinion, and challenge your vendor, for example. So, that strategy needs to feed into: “What am I doing with instrumentation? Who are my stakeholders? What do I want from these things? And what am I willing to trade off a little bit in the pursuit of maybe better economics? Or faster developer velocity, or the ability to better exceed my customer expectations?”

John: Ian, as a Field CTO, you’re out there talking to customers: Share with the audience what you’ve seen this past year. What are some of the notable things you guys have done? What’s the coolest thing you’ve worked on this year that you could share as the environment’s changing? What are some of the highlights?

Ian: I think what’s interesting to me is, tying back to some of the things we said before, the transformations of these larger, complex organizations really trying to centralize and get behind a very specific point of view. Things like: “We want to own our instrumentation and our data. We want to be in control of these things and the costs. So, elevating away from point solutions and maybe an assortment of open source things.

And it’s very interesting as well, with the current pressures, seeing companies who are famous for building some of their own in-house technology around observability going: “It’s not a differentiator anymore. The market is caught up, and we can align to open source standards and collectively take advantage of the work that’s been going on across the industry.”

I think there’s a convergence on things like: “We want one place to start looking at our problems. We might store the data in a bunch of different places, but there’s also an understanding that if we do this, we can’t be reliant on the most senior engineers who also are the bottleneck on innovation, to be the bottleneck on our reliability and delighting our customers from a performance perspective.

John: So, leveraging the talent and not wasting talent on what could be managed by getting more from everything.

Savannah: Developer productivity, and letting the humans actually get creative and innovate rather than worrying about what’s going wrong — I think it really does matter. We look forward to having you on the show, hopefully in Paris as well.

Savannah: What do you hope that you can say in Paris that we haven’t been able to say yet at this show?

Ian: I think we would love to see that we’ve gotten to the point where things like observability strategy are common for us to be talking about openly, right? And getting a little away from who got to Generative AI first. And also, being able to deliver those key values.

I definitely want to see a lot of the convergence of OpenTelemetry — metrics traces and now logs together, and seeing what you can do with those data sets together. And a common point — can we cross-link these things? Can we get into a more intelligent place where we’re less: “Hey, how do I query my logs directly? Can I derive those insights that I need so that every developer in my organization can be on-call successfully?” So that they can be empowered and our limited resources can be used more effectively.

Savannah: I love it. Well, we look forward to having that conversation there. I agree with you. I think the observability strategy will be good. We’ll keep our ears out for it on the show and see who’s talking about it. Ian, thank you so much for being here. Really appreciate it.

To learn more about Chronosphere and cloud native observability, schedule a demo.