Off-Call with Paige Cruz: Is Mobile Observability A Misnomer? Part 1

Ever had a mobile app freeze or crash? How about weird intermittent interaction bugs? Well, like me, you might assume mobile engineers have a wealth of telemetry data at their disposal, but the reality is a little bit different. As I found out in talking with today’s guest, Hanson Ho of Embrace.

In part one, he shares what makes mobile on call so different from other rotations. And how his time at Twitter and Salesforce shaped his perspective on mobile observability today, and whether or not mobile observability is a misnomer.

Hanson’s Recommended Resources

• DroidconSF 2023: Combating sampling bias in production
• OTel Community Day 24: OpenTelemetry for Mobile Apps
• O11ycast: Ep. #72, Mobile Observability with Hanson Ho
• Test Case Scenario: Mobile App Testing Automation

Paige Cruz: Welcome to the podcast where we meet the people behind the pagers. I’m your host Paige Cruz, and today I am joined by Hanson Ho of Embrace. Hanson is a mobile observability and performance enthusiast with a focus on Android, OpenTelemetry, and linking app performance to user impact. And if you haven’t already guessed, today’s episode is all about mobile.

Now, despite being in tech for almost a decade now, and according to my screen time, I’m seemingly always on my phone…there’s a lot I have to learn about the challenges that mobile engineers uniquely face, and if anyone actually reads those crash reports. Let’s find out after learning a little bit more about Hanson.

Welcome! Do you want to share a little bit about how many years you were on call? Let’s dig right in.

Hanson Ho: Hi Paige. My life on call, 10-15 years. I think I was first on-call at Salesforce. That was only a little bit and at Twitter, I was on-call, for many years.

Paige Cruz: Were you there in the fail whale days?

Hanson Ho: So it was post and I don’t think the fail whale graphic ever was in the Android app as far as I was aware. The fail whales wouldn’t really appear in native UI. It would just be a request that went “the app handles it probably”. Because everybody tests with 2G connections and with the WiFi off, right? I mean, everybody does that, right? And it always works.

Paige Cruz: Between Salesforce on-call and Twitter on-call, I have to imagine you have seen some things. You have been answering pages at maybe all hours because, one, Salesforce has such a huge population of users across a bunch of different time zones and obviously Twitter’s global.

Hanson Ho: The good thing about working on the front end is our deploys aren’t that frequent. So even at Salesforce, being on-call is really when we rolled out releases back then, this is 2010, once every 4 months. At Twitter the problematic instances were generally when we rolled out a new app version and that is done gradually. With dogfood releases, alpha and beta releases, to make sure that things are of reasonable quality. 

When we actually do all of that, generally when really bad things happen, we know immediately, or relatively immediately. We haven’t gotten that many legitimate weird pages in weird hours. Some in interesting places, but we can go more on that later.

Paige Cruz:  It sounds like if there was an issue, it was likely bundled in with all of the changes going out with the new release. So there was a little bit of, we’re doing a release, everyone’s got their guard up, we’re being alert, we’re looking for things, we’re prepared to handle when things go sideways.

Hanson Ho: When the issue is with the app, that is definitely true. But sometimes the issue isn’t with the app, but surfaces in the app simply because the backend may not have instrumentation on the client side in order to tell them, “hey, the request did not make it to the server”. So the server doesn’t know about it.

Paige Cruz: Oh!

Hanson Ho: Generally when a mobile on-call is pinged, it’s because of two reasons.

1. Somebody flipped the feature flag, and basically effectively changed how the workflows worked in the app, and something bad happened. 
2. When there is effectively a server side issue, that is not seen in the server side metrics and only bubbles up. Because on the client side, certain workflows were not succeeding.

A weird edge case is when some 3rd party data that is not being sanitized, ends up causing interesting issues, on the client side. So, deserialization, failures. Things that basically contract violations between the API and the app. That is unexpected. That happens too. But that’s rare. That’s like when somebody says, “Oh, I’m going to start an ad campaign”, but you don’t generally do it at 2am, unless they’re not in North America, in which case I’m sleeping, but Japan is not.

Paige Cruz: Interesting. What got you to this space of not only frontend, but also mobile?

Hanson Ho: Mobile, I had the opportunity to work on at Twitter, on the Android app and it sounded great. I know Java, so of course, Android, why not? Of course, you get into it and you’re like, “oh no, it’s slightly different!” It’s Java in appearances, but under the hood it’s not. For performance and things like that, it definitely matters of the implementation of the ArrayList or whatever.

Paige Cruz: Totally.

Hanson Ho: But the platform has improved in the years, in a sense. For me getting into the mobile I found it a really interesting problem, which is I was hired to do, emerging market improvements.

Paige Cruz: Oh, the real greenfield stuff.

Hanson Ho: Yeah. Well, it’s green and sometimes pretty brown because, emerging markets. This usually implies devices that are not the fastest, networks that are not the best, and making sure that Twitter at that point worked well in those places. So folks that are not used to having really fast phones and really high networks, they are used to basically being pissed off at their apps.

I learned a lot about the Android ecosystem and how fragmented it is, how networks are an untrustable ally that is necessary. But you have to always assume that it’s not going to work or it’s going to work in ways that you didn’t think it was. So learning about and getting thrown into that, I was working on like web stuff prior to that.

Going to Twitter and going into Android, it opened my eyes to a whole set of interesting issues of devices and low performing devices. If you think having a browser on a desktop computer, even an older version with the older rendering engine, maybe that’s slow – try using a phone with 1GB of RAM using an optimized version of Android that is designed to run on really low powered devices. Sometimes you might not expect things to finish, in what you thought was a reasonable time.

Paige Cruz: Reasonable is relative. Interesting! So that’s what got you hooked. I think we get a lot of flack in the tech industry for only serving users with the newest devices, the highest power, the best networks, I think that’s really gratifying to be able to have the skills to say, “no, I can help bring this app to people with varying device functionality and power and networking.” I think that’s pretty cool.

Hanson Ho: It’s a bit grandiose and self congratulatory. But being able to enable access for folks outside of the Western world to have access to information, good information, bad information, whatever. It was good. People were using phones as their first computing devices to be able to talk to their family and friends via Twitter or via messaging apps. This is important. Not everybody has a computer, but everybody can have a phone. Or a phone that their grandkid gave them because they don’t normally need to use it.

To say that, “Oh that phone’s five years old, or that phone supports an older Android version. I don’t want to support it because it makes maintenance hard.”Well, you’re cutting off a whole segment of people who would otherwise be using your device or your app. 

Depending who you are, certainly if you’re talking about games that need really high quality CPUs and stuff like that it doesn’t make sense to make it work for crap phones. If you’re a grocery delivery app and you just want to order your chips and salsa, and have it delivered, why have that heavy burden of, “oh yeah, I’m sorry, you have to use Android 12 and the phone that is at least $800.”

It doesn’t make sense.

Part of it is the mission. Part of it is just, it’s a really interesting problem to me because it doesn’t seem like it’s fully solvable, but I can get closer and closer to it.

Paige Cruz: I love it! That’s the perfect blend of a role. I think something that you’re intrigued by that gives you a lot of runway. It’s great to be able to be proud of the work that you do. 

I’m so excited about digging into mobile today because, as a heavy mobile user, the companies I’ve worked with, there’s either been no mobile team, because not a lot of monitoring companies offer mobile apps. Or the places I worked at had a mobile app, it was kind of 1 team kind of off to the side. They had a different workflow than everybody on the backend and the website. It was kind of like, they take care of themselves. They do their own infrastructure, their own pipeline management, because it’s so different. 

Is that something that sounds familiar? Unless the main product that you’re offering is a mobile app, that sort of siloing, in the eng org?

Hanson Ho: Yeah, I think the tools that we use are different. As you said, the workflows of deployment are different. Testing in prod means very different things when you’re talking about mobile apps versus a backend.

The siloing is a huge problem, especially when it comes to observability because the data – They [backend & mobile teams] have to talk to each other for it to actually be linked up. Usually the tools that mobile developers use to monitor their apps in production are not the same ones that are used by the back end folks.

Things are changing. I think we are at the precipice of some pretty big improvements. Traditionally data isn’t even accessible as data. We get dashboards that are pre-aggregated, with data about app startup, and various things. 

The notion of creating spans and traces for workflows just is not something that folks are used to in production. When people do things like tracing locally they get some pretty detailed traces and profiles. But having this done in production is not a common workflow. We work with crashes. That’s when you talk about production data. It really is crashes. 

On Android, you go on Android vitals and what’s the p50 in the last 24 hours? Great. 

Tell me more. 

Nah, you can get it for the whole month or the week. Here’s p75. 

I can cut it by version maybe but anything beyond that is impossible. I mean, I don’t even know if Google has the data. They may do really quick aggregations and summaries or the data might just be gone.

Paige Cruz: Wow. That sounds very limiting and frustrating because, we talk a lot on the back end about the overwhelming amount of data that we have. We have a data problem. There’s too much. Not every piece of telemetry is valuable, but y’all are on the total opposite end of, “Oh my God, please give me more! I need more detail!”

At a certain point, you can’t figure out what is going wrong, where, or why, if you can’t ask these deeper questions. 

Do you have time for a quick, modern history of mobile observability? Did it start open source? Did we move to proprietary agents? Educate us. What have you all been suffering through for mobile observability?

Hanson Ho: First of all, one of the things I’m trying to workshop is, mobile observability is a misnomer. Because what we have is dashboards of aggregates, and maybe some crash listings that have some granularity, but everything else, we don’t have it as an industry. Certainly big companies have found the need to do this and create proprietary systems to do effectively what OpenTelemetry does for traces for the client and have measurements like that. After that, you basically don’t even have anybody that offers anything free. 

For a long time, back to the history question, we struggled with whatever the platforms gave us, whatever Apple and Google deemed worthy, here you go.

And then crash SDKs, start coming up to basically instrument, crashes because there are multiple ways of crashing on Android , and then several different ways of crashing iOS as well. Entire startups got created based on crash logging. Crashlytics is one – they were independent then bought by Twitter and then sold to Google and now is part of Firebase.

Paige Cruz: Interesting that Twitter was like, “I’m just going to buy the tool I need and bring it in house.” Dang. I mean, that speaks to the problem.

Hanson Ho: Yeah. People generally worked with crashes because, first thing you care about is if the app crashed, that’s bad. And then more sophisticated SDKs came up, Embrace, one of them. Obviously some of the other players have mobile solutions that do similar things. Some are more optimized for mobile and some just did the basics. Depending on what you want and who pays the bills, you use whatever you use.

Paige Cruz: My tagline that I’m talking about is there’s so many organizations that say, or leaders that say “we want the single pane of glass” and they mean I want one tool, one place. 

I think today, as long as you’re owning your data, that pipeline, that journey, that path, really, your best bet is to choose tools that are best in their domain. 

So like Chronosphere, great for app and infra visibility. We use Kentik for network monitoring because Kentik’s the best in class for that. And Embrace, y’all think about mobile day in and day out. 

I see some platforms want to check all the boxes, and like you say, they do the basics. I would so much rather get my mobile monitoring, data, and analytics from people that that’s all they’re thinking about. They can be a strong partner, the support’s gonna be great, I think we do best when we choose to partner with vendors who are already experts to level us up. 

I think like a single pane of glass, tie all your data together? Yes. But choose the tools that are best for what you’re looking at.

Hanson Ho: This is why I’m so bullish on OpenTelemetry because at the end of the day, how you capture the data is less important than how the data is, formatted and structured.

As long as your data is basically the same in terms of format, you can link them together. And whether you do it yourself, whether it’s some third party instrumentation, OpenTelemetry is OpenTelemetry. If things can be linked together, it’s good.

So what I said, mobile observability is a misnomer – that is the past. In the future, observability is going to be de rigueur. Everybody’s going to want it and everybody’s going to realize the need to have it. Instrumentation is only the start.

We are bootstrapping instrumentation because we prepared previously at Embrace – we had our own proprietary data format. The things that we capture, it’s nothing fancy. We capture data, we shove it to the other side and we show you some cool dashboards. We explored OpenTelemetry and stuff, but now we fully natively capture OpenTelemetry. You could do a direct export from the SDK to your backend collectors, as well as have Embrace get the same data, with additional context that we have. You can do both. You don’t have to choose between a single pane of glass and tie yourself to a particular platform that’s very expensive, and you can’t get off of. No names.

Paige Cruz: [that expensive platform] it makes headlines, every quarter.

Hanson Ho: Your bills, your finance department will tell you. You don’t necessarily choose between that and then some open source software solution that you have to patch together. As long as your solutions can talk, it’s a federation of data, and it is important for us to support that.

Right now we spend our time in the SDK, we spend time on instrumentation. Like the backend ecosystem where instrumentation happens at the library level and there are separate packages for that, we want this to happen for mobile as well. I don’t want to go and instrument IO or whatever popular DI framework that you use — I want it to be there using the APIs that are common. So you can have that instrumentation use an instance of the Embrace logger provider, OTel logger provider then have that go into our sessions and co-mingle and work like that. The pieces and the bones of true observability for mobile are there – we just have to build it out. We’re in the process of doing that. Sometimes we want things to go faster and it doesn’t go as fast as we want, but it’s okay. We’re going to get there. As long as we maintain the North Star. Folks are doing great work; important areas in the ecosystem are being filled out. Whatever we had in terms of concerns about OpenTelemetry, it’ll get there, things that we need, it’ll get there.

Paige Cruz: And it’s all happening in the open. For folks that have criticisms of OTel (which I don’t think any project is above being looked at from all the angles) I do think, I would rather have the spec, the schema, everything be done out in the open and be able to contribute. Versus a company who can make decisions one way or the other, or switch a pricing format, or change the fidelity to make things better on their end. It is gonna take a long time to build up what the New Relics, the Datadogs, what those companies built up for their proprietary offerings, We’re recreating that in OTel totally from scratch and of course other companies have helped accelerate by donating their code and their work. To me, open is good, and I’m really excited when we get to the place of instrument once, observe anywhere. 

Instrumentation, while important, is not always the funnest to write. There’s a lot of other things we could be doing with our time, so I’m happy that you and Embrace, and all our friends in OTel are working on this. 

Now, I want to talk really quickly about where you picked up your knowledge in monitoring and observability, especially as you were coming up to speed with mobile. How did you learn? Were you given a great onboarding course? Did you have a good shadow rotation? The laugh makes me think you were kind of thrown to the wolves, but, you tell me.

Hanson Ho: We had on-call at Twitter, we had run books, we had shadows. But the problem is on mobile, the code base is different. You’re not just on call for your portion of the code you understand, you’re on-call for the entire app, and that generally means code that you don’t know, for the most part. You can start the investigation, you can do triaging. But at the end of the day, you have to understand the code. and that’s not a very good place to be. for mobile, it’s reasonable because the app at the end of the day isn’t that big and it’s all generally a monorepo. Even if it’s broken down to modules. There are different modules. It’s like they’re separated,

With mobile on-call, it’s almost here’s a problem, figure out who owns it so you can triage it. Here’s a stack trace, I’m going to go and look at “Oh, this is called here. Oh, it goes through this other layer.” 

You learn a lot by going through that process. But let’s just say it’s not the best when an incident is happening. There’s a crash that is occurring 1000s of times a second or a minute or whatever it is. You don’t want to be under time pressure and especially on mobile. 

A lot of times you can’t really fix it quickly. You have to build the app, make sure it goes, and it gets approved in the various stores and then it gets rolled out and then you just have to install it.

So it’s a whole process.

Paige Cruz: They have to approve it? They’ve [App Stores] have already said, “We’re letting you in the store.”?! 

I’m thinking of the time that I caused an incident, it was a misconfiguration. I was able to fix it just by a quick rollback. We had the GitOps thing so I had to merge a PR, and the system went from denying all traffic into the cluster to everything’s hunky dory in under a minute. 

….Wait, so they could say no to a fix that you’re putting out?? That blows my mind. Why do they need to approve everything? They think you’re gonna sneak something sneaky in there? I’m sure it’s happened, but wow.

Hanson Ho: Oh, it definitely has happened. Basically every deploy to the Play Store and so my experience of the Play Store with Android, but, Apple has a similar process, which, I’ve been told is more, inexplicable sometimes in terms of failure, but basically we submit a new change, even if it’s like literally no code change.

Even if it’s just like an asset that we switch out. It doesn’t matter. It goes through the same review process. Or at least appears to us as it goes through the same review process. 

It basically says, “Oh, waiting to be approved”. And then eventually you’ll get approved. Sometimes it’s pretty fast. Certain bigger apps have priority, in terms of getting reviewed and things like that.

Paige Cruz: So you’ll get a decision within five minutes

Hanson Ho: Minutes? Minutes? People have been waiting 24 hours for their hotfix to be deployed.

Which is why on mobile, well, first of all, rollback is literally not a thing. You can never roll a version back. You could stop a percentage rollout, at a certain percent. So you roll forward – basically build a binary such that it is a rollback, but it is a roll forward, it is a different version. So getting database migrations are depending on versions.

That’s why we unfortunately both have to test in production just because of the chaotic environment of mobile apps running. But we also have to be super careful when we deploy. That’s why different companies will do it differently.

The best way to do it is to have a dogfood app that you use internally. At Twitter, it’s pretty easy because a lot of people at Twitter use Twitter. Contrary to popular belief, a lot of people do use it. Used to, at least.

Paige Cruz: Anymore, who knows?

Hanson Ho:  Dogfood would have like 20, 30, 40 users. Definitely more for iOS than Android. But it’s sufficient that if there’s instant crashes or things that affect a lot of people, right away, we find it and we fix it. 

We have alphas, which is outside, but smaller, and betas, which is outside, but like, tens of thousands or something like that.

Then we do a 1% release. After going through real users, real devices, distributed globally and with whoever decides to sign up with whatever device. At least the population is –it’s not truly randomized, but it’s a fairly decent representation of at least early adopters.

Then we do the full release, which is 1%, and we give it like a 24 hour bake time to say, “Hey, this has been released to 1% in 24 hours.” 

Now, not everybody updates right away.

Paige Cruz: Thinking of all the times I’m like, “No, excuse me. I had something I wanted to do. I’m not trying to update and stop the world right now.”

Hanson Ho: There’s auto updates that you turn on that will do it at more convenient times. So with 100% auto-update, we do get a fairly representative sample out there. Generally when it’s gone through the 24 hours and things generally look good, we release it and things are good.

Then we monitor, mainly crashes at that point because crashes are easy to count. They’re definitive. With workflows and the duration, who knows how it’s implemented or who knows who’s actually using it and you can’t get representative data with just a 1% rollout.

You’re not going to be able to have any regressions checked with production data with 1%. You basically have to do some synthetic testing beforehand to make sure that your key workflows don’t regress. That’s why there’s a huge demand on mobile because most mobile perf and stability testing is done prior to release.

Paige Cruz: Oh my gosh. Which like, yes, great, good to have that confidence, good to catch stuff early, but that does not sound like enough. 

I’m thinking, “oh, I’m going to be on call for this mobile app of which I know a slice of it really well, and there is a lot that is a mystery to me, and I have to go on a hunt to figure out who even owns the problem before we can even start fixing it, all the while crash reports are like rolling in.”

That sounds pretty stressful. I know mobile engineers that are smiling and positive, like, how??

Hanson Ho: Which is why we have the pre-release process. The betas of the 1%, generally all bad issues are caught there for crashes. The more subtle insidious issues are performance things.

Things are 30% slower. How do you know? Well, you don’t really know until you have production data because your sample affects it. Maybe it only affects devices with more than or less than 4 cores, because you’re just doing a ton of parallelization. When you have more than enough cores to handle it, yeah, things are good. But if you don’t, then you get back pressure, and your expected things are not finishing in time. You’re not going to know unless you’re testing the real device. 

Production data is, in my opinion, crucial and people don’t pay attention because they have enough problems to deal with already. The aforementioned crashes, supporting new OS versions, supporting interesting device form factors. It doesn’t come about often, but you know, sometimes it does and it’s everything.

The idea of looking for more issues for performance. There isn’t a groundswell, unless you work for one of the big tech companies, when you have specialists who literally focuses on performance on mobile. Like myself, or at least used to be. That’s why we had to build proprietary systems to work with the data in our own kind of ecosystem or the proprietary dashboards and stuff like that. Things are changing now, people are going to have more time, because the ecosystem is improving, the platforms are getting better.

In fact, people are demanding that, “Hey, my internal dashboards aren’t showing that customers are pissed off, but they’re pissed off. How do I tell?” 

Well, do you actually know what’s happening on the mobile app? 

Well, no.

What about mobile testing? 

Oh, we have synthetic testing!

 Great. What kind of device–

We use a Pixel 7 to test it. 

Oh, great.

Even choosing 10 devices—

Paige Cruz: The network, how many other apps I’m running, am I just leaving my house going off WiFi.  I could think of so many —. Our phones go everywhere with us! That is a very hostile environment, I feel like, to be operating in.

Hanson Ho: It becomes a bit of a self fulfilling prophecy, when you don’t test with crap devices, and your data shows that nobody with a crap device uses your app? You’re like, no one uses it. Why am I even bothering? Hey, he says he can’t run it. That sounds like it’s 35 seconds. and every time I try to press a button it freezes, so of course I’m going to uninstall right away. It’s not that they don’t want to use your app, the demand is there, it’s just not possible.

I always imagine performance as basically, a cliff edge. You create a level of performance such that anybody who is worse than that will not use your app. if you can improve performance, you’re not making people who use the app happier, you’re allowing new people to use your app, perhaps unsatisfactorily, because it’s slow, but still fast enough that they can’t actually use it.

Paige Cruz: They’re not abandoning.

Hanson Ho: Exactly, you’re extending your total market that you can address. if you don’t want to support Android 10 and below, okay, cool. But what if I have an Android 9 phone? What if grandma has an Android 9 phone? Oh, sorry, you can’t. I don’t know what grandmas do – many things, cool things. Listen to cool music from-

Paige Cruz: Spotify, podcasts, sending silly photos with filters.

Hanson Ho: Exactly.

Paige Cruz: My grandma’s great at emoji. She learned emojis before me and got in a little texting and driving fender bender. It was not serious, but I was like, grandma, if anyone in this family was gonna get in a texting and driving little bump, I would think it would be the teenager at the time.

Hanson Ho: Let me just back up. I don’t want to spread ageism around. However, usually people who buy new phones are young-ish people who are interested in this. They give their phones to their parents and their grandparents who may not care.

“I use this to play fantasy football. As long as the Yachting Fantasy Football app works, I’m okay. “ Well, it does work. Until it doesn’t. Until, they decide, I want to only support API 29 and above so sorry. You’re just not going to use it.

Paige Cruz: I was doing a little bit of looking at what Nielsen was saying, the big marketing company about mobile experiences. If you have a bad mobile experience, people are 62 percent less likely to ever come back and buy something if it happens to be an e-commerce sort of thing. I think whoa, we have really high expectations and we have a really low tolerance for bad experiences. They specifically were talking about things that have to be optimized for mobile. 

I can think of last week I had to buy a plane ticket in a rush. I didn’t want to have the mobile app. I’m so app fatigued. I don’t want to download anything else. So I was going through my mobile web browser and the site was clearly for desktop because I had two pop ups covering the whole thing of what I was trying to buy. Every screen that I changed, the pop ups came back and I was like, so infuriated.

The purchase went through, thank god, but I was so angry. Why can’t we have nice things on mobile web and mobile apps?

Hanson Ho: Imagine if you were so frustrated, you decide to use a competitor’s website instead of using one food delivery app or website, use another one because it works better. You’re not going to know if you lost the business because you never had the transaction completed. So do your SLOs measure, client, starting a transaction, starting an order and finishing? It should.

Paige Cruz: That sounds so much like business analytics. It seems like mobile is so close to the customer or the user and the business experience, we’ve got to infuse what are typically seen as business metrics: abandonment, cart crashing, like all of that sort of stuff with the data that y’all see.

Hanson Ho: That is what pays the bill. I hate having instrumentation that’s “oh, you can go see this in Amplitude or whatever.” How does that data work with— No, it doesn’t work with the performance data. It’s completely separate.

Paige Cruz: Why would you want that?!

Hanson Ho: The ultimate thing for this app is to let people buy stuff. If you’re not tracking buying stuff successfully as a first class SLO on your mobile app what are you even doing? I don’t care how fast your app loads. I don’t care how fast app startup is. if you buy the thing, that is good. That means it’s good enough at least to get you to buy.

So you may not know how far off you are from losing that person, but hey, at least you knew that person bought. Knowing what your screen jank is that’s an indirect metric.

Paige Cruz: Y’all have been dealing with a lack of data!

Today Hansen gave us a fascinating look at the world of mobile engineering. How seemingly small decisions can impact millions of users and why high quality telemetry is critical to solve mobile performance challenges. If you’ve ever wondered how observability is evolving to meet the demands of mobile, you won’t want to miss part two. 

And now a word from our sponsor Chronosphere. Let’s talk logs. Like most folks, you’ve probably got logs flying in from every direction, in every format imaginable, and that need to get routed to more and more platforms than ever. before. And that list is growing.  If that sounds familiar then you already know that managing that mess can be a big headache, but what you might not know is how Chronosphere Telemetry Pipeline can help.

With Chronosphere Telemetry Pipeline you can collect logs from anywhere. process and enrich them in real time then route them exactly where they need to go. Whether that’s a cloud storage bucket, a security platform, or even another observability vendor.  

Built by the creators of Fluent Bit, Chronosphere Telemetry Pipeline is Kubernetes native and can scale effortlessly to handle whatever your data demands. If you’re curious to see this in action. Head over to chronosphere.io or click the link in the show notes. 

Thank you Chronosphere!