Experience faster dashboards with Query Accelerator

You’re an on-call engineer, and you are woken up at  3 AM to address a production issue. You review the situation and follow a link to a dashboard, but it’s slow to load. Some of the queries that power it are taking a long time. You identify a potential source of the issue but, as you change variables to dig deeper, those changes are again slow to load. You’re tired, and your thought process keeps getting interrupted as you wait for dashboards to load to see if your changes were successful, over and over again.

This happens all the time, so you’re used to it, but it’s still frustrating. And even though every engineer in this situation says that they’ll work on making the dashboards faster, it rarely gets done.

Why? Because writing correct and performant queries against observability data is difficult, especially in cloud native environments. Doing so requires possessing a significant amount of knowledge about many things, including:

• A query language (such as PromQL)
• The architecture and scale of the environment
• The underlying data model of the observability solution
• How a query written in Dev/Test will perform in production

And writing a query is not a one-time thing. Queries need a lot of care and feeding to stay performant. They require human intervention to adapt to underlying system changes such as more services (and data) being added, dashboard display option updates, and management-driven events that roll out without your involvement due to the dynamic and asynchronous nature of a cloud-native environment.

Software engineers moving to DevOps often don’t receive specific training on any of this, so they are forced to reference public documentation, blog posts (maybe including our own!), and/or recommendations to, “meet with Jane, our query expert, she can show you what to do!” in order to learn to work with observability data.

Or imagine that you’re an SRE or observability team manager who has spent hours working to build a dashboard to show performance metrics across your system. You’ve tried what feels like hundreds of versions of your PromQL query, but so far none of your changes get the dashboard panel to load with the data you want. You eventually realize you’ve run up against the limitations of the browser, which cannot retrieve more than a certain number of data points without crashing.

While slow-loading dashboards are inconvenient and waste valuable engineering time, they also increase the time it takes to remediate incidents, which can create frustration for customers and have a negative impact on the business.

At Chronosphere, our mission is to build an observability solution designed for the way cloud native engineers need and want to work, not one that creates more work. Most observability solutions aren’t capable of addressing slow-loading dashboards without asking for more manual and client-side work from their customers, so it became accepted as the way it always would be – until today. Using the unique aggregation capabilities built into Chronosphere, we are happy to announce a new enhancement to our platform called Query Accelerator.

Query Accelerator ensures every possible dashboard across your fleet is fast and performant – no manual optimizations needed. Engineers no longer have to be proficient at writing “perfect queries” that work across environments and data scale. They can instead focus on creating a query that returns the data they need, and Query Accelerator will ensure that it performs optimally.

Key benefits include:

Query Accelerator identifies slow-loading dashboard queries and, where possible, pre-writes the aggregated data point that the dashboard was trying to return. It then looks for similar queries across all of a customer’s dashboard and sends the faster, pre-aggregated time series to them as well.

By pre-writing the aggregated data point that the users were trying to retrieve, Query Accelerator is able to load the requested data much faster and for larger slices of time without timing out or losing the ability to retrieve any data at all. By using this pre-aggregated series to fulfill query requests from all eligible dashboards, Query Accelerator ensures that users get the maximum performance from the system without requiring them to take the manual steps of identifying similar queries by hand and updating each of them to use the new pre-aggregated time series individually. What used to take one engineer or one team hours to accomplish, or even be so daunting that it discouraged anyone from beginning to optimize dashboard queries, is now accomplished continuously and automatically by Query Accelerator.

Win-Win for everyone

For the 60% of engineers responding to off-hours alerts once a week or more, this will be a welcome change. Being woken up at 3 AM and having your dashboard taking three minutes to load feels like an eternity. With Query Accelerator we hope that all of these dashboards load almost invisibly faster, as if they were never slow to begin with.

For the observability team, if you want to know how Query Accelerator is helping, you’ll be able to visualize it in its own dashboard. This means you can see over time how many queries are getting accelerated and how much time is being saved as a result.

And for anyone in the organization creating queries and/or using dashboards, this will be a better user experience that helps free up valuable engineering time, speed up troubleshooting, and shorten time to remediation.