As a new engineer, understanding cloud-native observability is important for several reasons.
First, it is a key component of cloud-native application development. It uses monitoring and observability tools native to the cloud, designed to work seamlessly with cloud-native applications. This helps build, deploy, and manage applications more effectively.
Second, cloud-native observability provides better visibility and troubleshooting. It provides a comprehensive view of the application and its underlying infrastructure, helping to identify and troubleshoot issues quickly.
Third, it promotes better collaboration and DevOps culture. By providing insights into the performance and behavior of the application, it becomes easier for different teams and developers to work together.
Fourth, it allows for better security. By using monitoring and observability tools native to the cloud, it enables quick detection and response to security threats.
In summary, understanding cloud-native observability is important for building and deploying applications in a cloud environment. It is a key component of cloud-native application development, provides better visibility and troubleshooting, promotes better collaboration and DevOps culture, and allows for better security. It is essential for any engineer working in the field today.
Learning Materials
Here’s a list to get you started learning about cloud-native observability. Note that some of these links may not be free and may require a subscription or payment. I receive no affiliate payments for these links.
Beginner:
- “What is Cloud Native Observability” by new relic: https://newrelic.com/blog/best-practices/what-is-cloud-native-observability
- “Cloud monitoring guide: Observability” by Microsoft: https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/manage/monitor/observability
- “What is Observability? ” by splunk: https://www.splunk.com/en_us/data-insider/what-is-observability.html
Intermediate:
- “Cloud-Native Observability with AWS” by AWS: https://aws.amazon.com/cloudops/monitoring-and-observability/?whats-new-cards.sort-by=item.additionalFields.postDateTime&whats-new-cards.sort-order=desc&blog-posts-cards.sort-by=item.additionalFields.createdDate&blog-posts-cards.sort-order=desc
- “Cloud monitoring guide: observability” by Microsoft: https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/manage/monitor/observability
- “Implementing observability for always-on Ecommerce experience” by Google: https://cloud.google.com/blog/products/management-tools/observability-on-google-cloud
Advanced:
- “The Definitive Guide to Observability in Kubernetes” by observeinc: https://www.pluralsight.com/courses/advanced-cloud-native-observability
- “Kubernetes Observability 101: Tools, Best Practices, and More” by Cloudzero:
Videos to Watch
LF Live Webinar: Kubernetes Observability with OpenTelemetry and Beyond
Martin Fuentes and Cedric, product managers at Instantana, discussed Kubernetes resource management and how to observe Kubernetes workloads. They explained how CPU and memory are the most important resources managed by Kubernetes, and how requests and limits can be configured for containers. They also discussed the different types of scaling, such as horizontal and vertical scaling, and how HPA (Horizontal Pod Autoscaling) can be used to automatically scale containers.
A Possible Learning Path
Hands-on experience: Start by setting up a simple Kubernetes cluster and experimenting with different observability tools such as Prometheus, Grafana, and Elasticsearch. This can be done by following tutorials and guides, and deploying these tools on a cloud platform like AWS, Azure, or GCP.
Theoretical learning: Once you have a basic understanding of observability, you can begin to explore the underlying concepts and technologies such as Kubernetes metrics, logging, and tracing. This can be done through online resources such as tutorials, courses, and documentation provided by Kubernetes, as well as books and blogs on the topic.
Understanding the principles and best practices: Observability is an important aspect of a microservices architecture, so it’s important to understand the key principles and best practices of observability, such as metrics collection, log aggregation, and tracing.
Joining a community: Joining a community of Kubernetes enthusiasts will help you connect with other people who are learning and working with observability for Kubernetes. This can be done through online forums, meetups, and social media groups.
Practice, practice, practice: As with any new technology, the best way to learn is by doing. The more you practice deploying and using observability tools in a Kubernetes cluster, the more comfortable and proficient you will become with the technology.
A Note from the Architect
Ok, let’s talk about observability. It’s a pretty important concept to understand, especially if you’re working in a DevOps culture. Even if you’re not, it’s a great practice to start using when you think about building out your solutions.
So, what is observability? Essentially, it’s the ability to understand the internal state of a system by observing its external behavior. This means that you can use metrics, traces, and logs to understand what’s happening inside your system, even if you don’t have direct access to the internal state.
There are three main principles of observability:
- Measurability: The ability to collect and aggregate data from your system. This includes metrics, traces, and logs.
- Understandability: The ability to understand the data that you collect. This includes using visualization tools and dashboards to make sense of the data.
- Explainability: The ability to explain the data that you collect. This includes using tracing and logging to understand the cause of issues.
So, how does observability fit into a DevOps culture? Well, in a DevOps culture, you’re focused on smooth, but rapid development and deployment. This means that you need to be able to quickly understand and fix issues that arise in your system. Observability gives you the tools to do that, by providing a way to understand the internal state of your system.
Now, let’s talk about some tools that can help with observability on Kubernetes. Here are a couple of popular options:
- Prometheus: Prometheus is an open-source metrics collection and monitoring tool. It’s great for use cases where you need to collect and aggregate metrics from your system.
- Jaeger: Jaeger is an open-source tracing tool. It’s great for use cases where you need to understand the cause of issues in your system.
Both of these options are great choices for different types of observability, and they’re both built specifically to work with Kubernetes.
So, what are the consequences of not practicing observability? Well, if you’re not practicing observability, you’ll have a hard time understanding what’s happening inside your system. This means that you won’t be able to quickly fix issues that arise, which can lead to downtime and lost revenue. Additionally, you won’t be able to understand the performance of your system, which can lead to poor performance and a bad user experience. So, observability is a crucial practice to ensure the smooth running of your system.
So, that’s the basics of observability. It’s a powerful tool that can help you build robust and scalable solutions on Kubernetes. If you have any more questions, feel free to ask!
