The business impact of observability

This excerpt from the recent EMA Research Report “Observability: Challenges, Priorities, Adoption Patterns, and Solutions” lists the top ten reasons your observability costs are on the rise and why they are impacting your overall business. 

1. Overpaying due to lack of granularity: Cloud native apps often consume infrastructure on a temporary basis, often only for  minutes. If the observability platform vendor charges for longer increments, observability cost can quickly exceed the cost of the  resource. 
2. Unpredictable monthly bills due to complexity: There are many factors that can influence observability cost. These factors  can differ by cloud and observability platform. Data ingestions, query complexity, data retention, number and level of users,  features subscribed to, number and type of API calls, data export, multicloud and hybrid cloud integrations, overage fees and  premium support plans are some of these factors. 

1. Continuous risk of cost escalation due to lack of controls: Misconfiguring a Kubernetes cluster to log and poll telemetry data at a detailed level and in very short intervals can lead to negative surprises when the monthly observability bill arrives. Combine this with a lack of data retention policies; detailed monitoring for Amazon EC2, RDS and other public cloud services; a  large number of custom metrics and the detailed logging of serverless invocations, and you may receive a bill that you will have  to explain to your boss. 
2. Vendor lock-in increases long-term costs: Being tied to a specific observability platform due to contractual obligations or technical dependencies can make it difficult to switch to a more cost-effective solution, leading to higher long-term costs. 
3. Hidden costs in “free” or “freemium” plans: Some observability platforms offer free or freemium plans that seem cost effective initially, but come with limitations that can result in substantial costs when exceeded. 
4. Inefficient use of premium features: Subscribing to premium features that are underutilized can contribute to unnecessary costs. Teams may enable these features without fully understanding their utility or cost implications. 
5. Costs of compliance and data security: Ensuring compliance with industry regulations and data security standards can  add extra layers of complexity and cost to your observability strategy. 
6. Geographical data storage costs: Storing data in multiple geographical locations for compliance or latency reasons can  increase storage costs significantly.
7. Real-Time monitoring and alerting costs: Setting up numerous real-time alerts and monitors can seem like a good idea for immediate issue detection, but can add to the overall cost. 
8. Skill gap and training costs: The need for specialized skills to manage and optimize the observability platform can lead to  additional costs in training and possibly hiring specialized personnel. 

The pitfalls of overpaying stem from a lack of granularity since cloud native apps temporarily use infrastructure, yet observability platforms may charge for longer durations. The complexity of various factors, such as data ingestions, query complexity  and multicloud integrations contribute to unpredictable monthly bills. 

Furthermore, the risk of cost escalation is prevalent due  to potential misconfigurations, like overly detailed logging in Kubernetes clusters or insufficient data retention policies. Vendor  lock-in scenarios pose a threat to cost-effectiveness in the long run, limiting the transition to more budget-friendly solutions. “Free” or “Freemium” plans, though initially appealing, may conceal costs that manifest when usage limits are surpassed.  

Similarly, the underutilization of subscribed premium features, compliance and data security costs, geographical data storage  costs, real-time monitoring and the necessity for specialized skills and training further inflate the overall expenditure. These myriad factors elucidate the necessity for a well-strategized, granular approach toward managing observability platforms to avoid a financial quagmire.

About EMA  

Founded in 1996, Enterprise Management Associates (EMA) is a leading IT analyst research firm that specializes in going “beyond the surface” to provide deep insight across the full spectrum  of IT management technologies.