CNCF TAG Observability: A year in review

The Cloud Native Computing Foundation (CNCF) has many different projects (with over 143k contributors at the time of writing), and each project falls under one of the following categories (check out the CNCF landscape to see the projects under each category).

• Application definition and development
• Orchestration and management
• Runtime
• Provisioning
• Special
• Platform
• Observability and analysis

The problem with these categories, however, is there are many sub-categories within each one, making it challenging when wanting to locate a project of a specific engineering domain or see how various projects overlap and interact across categories. This is why the CNCF created Technical Advisory Groups (TAGs) – formerly known as Special Interest Groups (SIGs) – to help provide technical guidance and expertise across projects pertaining to a specific domain – in particular around security, app delivery, storage, network, runtime, contributor strategy, and observability.

Each TAG has an active member base that meets on a regular basis to discuss new and existing projects, address any challenges or concerns from the community, establish best practices, and provide resources and expertise around the current state and future of the domain.

Chronosphere is a silver member of the CNCF, and has been involved in the TAG Observability for the past year. This blog provides an overview of the TAG’s overall charter, as well as a recap of what was accomplished this year.

The TAG Observability’s mission statement is to “focus on topics pertaining to the observation of cloud native workloads. Additionally, it produces supporting material and best practices for end-users and provides guidance and coordination for CNCF projects working within the TAG’s scope.” You can learn more about the TAG’s scope in their Github repository.

The TAG meets on the first Tuesday of every month, and is led by co-chairs Matt Young, Alotita Sharma, and Richard Hartmann. Some CNCF projects tied to the TAG include Cortex, OpenMetrics, Prometheus, Thanos, Fluentd, Jaeger, OpenTelemetry, OpenTracing, Chaos Mesh, and Litmus.

The TAG Observability has had a busy and productive year. In addition to helping perform due diligence for the incubation of OpenTelemetry, Cortex, Thanos, and OpenMetrics into the CNCF, the group published a whitepaper, established new working groups, and presented at KubeCon Europe and North America. See the below recaps for more information.

The TAG published a whitepaper earlier this year with the aim of helping community members get started quickly with different approaches to observability within a cloud-native world.

The paper has sections for metrics, logs, traces, and profiles, as well as an explanation around how these observability signals are correlated and should be handled. It lists different methods that companies have used when tackling common observability issues or challenges, presents tools that fall under the observability scope and where they should fit into your observability stack, and finally, shares some of the commonly known gaps in the observability market.

Note: The whitepaper is still a work in progress. If interested in participating or providing feedback, join the #tag-observability channel on the CNCF’s slack.

TAGs can have various Working Groups (WGs) focused on addressing specific problems or areas to help improve or progress the TAG’s overall mission. This year, several WGs were created with a goal of creating more resources for new and existing members of the TAG, as well as to help grow membership and participation. The WGs are broken out as the following initiatives:

• Create and curate a YouTube channel for future and existing observability talks, webinars, etc.
• Generate a plan for how the TAG should reach out to new projects within the TAG’s scope
• Identify and develop key personas within the observability space
• Create a vendor list with mappings to their CNCF project contributions
• Foster and encourage in-person observability meetups

TAG leaders Bartlomiej Płotka, Richard Hartmann, and Simone Ferlin gave an update on the TAG at this year’s KubeCon Europe. A few highlights include:

• A recap of the due diligence process for incubation of Cortex, Thanos, OpenMetrics, and OpenTelemetry into the CNCF
• Discussion around how the three pillars – metrics, traces, and logs – can be used together as a starting point for your observability function
• An update on what’s upcoming for the TAG including the whitepaper, best practices documentation, and observability webinars

Co-chairs Matt Young and Alolita Sharma gave a session focused on the TAG at this year’s KubeCon North America. In addition to providing an update on the TAG – both on what’s been accomplished in 2021 and on what to expect in the upcoming years – they also discussed:

• What is observability, and how the evolution from monoliths to microservices has increased the scale and complexity of observability tooling
• How observability goes beyond the “three pillars” of metrics, traces, and logs with new signals such as continuous profiling, crash dumps, and chaos testing
• Why observability matters for developer experience, incident response, and cost management

See the full session recordings for KubeCon Europe and KubeCon North America for more insight into what was discussed.

If interested in learning more about or joining the TAG, feel free to join the #tag-observability channel within CNCF’s slack or subscribe to the mailing list. The group also holds monthly meetings which can be found on the CNCF’s events calendar, and are open to any and all interested participants.