Tag: DevOps

Posted on by — Leave a comment

Understanding Containerization: A Guide for New Engineers

As a new engineer, understanding the concept of containerization is important for several reasons.

First, containerization is a key component of cloud native application development. Containers are a lightweight and portable way to package software, making it easy to run and manage applications on cloud infrastructure. By understanding how containers work, you will be able to build, deploy, and manage cloud-native applications more effectively.

Second, containerization allows for greater consistency and portability across different environments. With containers, you can package an application and its dependencies together, ensuring that it will run the same way regardless of where it is deployed. This eliminates the “works on my machine” problem and makes it easier to move applications between different environments.

Third, containerization allows for greater scalability and resource efficiency. Containers use less resources than traditional virtual machines, and can be easily scaled up or down as needed. This makes it easier to handle the increasing demand for more computing power and storage.

Fourth, containerization also allows for better collaboration and DevOps culture, as containers can be easily shared and reused, making it easier for different teams and developers to work together on the same application.

In summary, as a new engineer, understanding the concept of containerization is important because it is a key component of cloud native application development, allows for greater consistency and portability across different environments, enables greater scalability and resource efficiency, and promotes collaboration and DevOps culture. It is a powerful tool for building and deploying applications in a cloud environment and is essential for any engineer working in the field today.

Learning Materials

Here’s a list to get you started learning about containerization. Note that some of these links may not be free and may require a subscription or payment. I receive no affiliate payments for these links.

Beginner:

Intermediate:

Advanced:

Videos to Watch

Kubernetes Crash Course for Absolute Beginners

Kubernetes is an open source container orchestration framework designed to manage applications made up of hundreds or thousands of containers across multiple environments. It offers features such as high availability, scalability, and disaster recovery, as well as a virtual network that enables communication between pods. This video provides an overview of the Kubernetes architecture and components, and a use case of a web application with a database to illustrate how it works.

Possible Learning Path

Hands-on experience: Start by installing Docker and Kubernetes on your local machine or in a virtual environment. This can be done by following the official documentation provided by Docker and Kubernetes. After that, you can follow tutorials and guides to build and deploy simple applications in containers using Docker and Kubernetes.

Theoretical learning: Once you have a basic understanding of Docker and Kubernetes, you can begin to explore the underlying concepts and technologies. This can be done through online resources such as tutorials, courses, and documentation provided by Docker and Kubernetes, as well as books and blogs on the topic.

Joining a community: Joining a community of Docker and Kubernetes enthusiasts will help you connect with other people who are learning and working with these technologies. This can be done through online forums, meetups, and social media groups.

Practice, practice, practice: As with any new technology, the best way to learn is by doing. The more you practice building and deploying containerized applications using Docker and Kubernetes, the more comfortable and proficient you will become with these technologies.

Specialize in a specific use case: Docker and Kubernetes can be used in a wide variety of scenarios and use cases, so it is beneficial to specialize in one or two that align with your business or career goals.

A Note from the Architect

I’m trying to think of the first time I worked with a Docker container. I believe it was almost four years ago, which in technology timeframes was forever. I was trying to decide if I wanted to use Angular or React to create components for a plugin framework running on SharePoint. It wasn’t the type of development I was used to doing at the time, but I knew the industry was heading away from Angular and more toward React. So, I installed Docker on my laptop, learned how to check out the images, and eventually started learning the basics of containers. I was hooked. Here was a great way to build apps, host them locally, and get the same experience locally that I could expect in production. No more, “It worked on my machine.”

In the last couple of years, I’ve used the remote connection capabilities of VS Code to run pretty much all of my development in containers. It gives me the freedom to try out different languages, frameworks, and libraries without ever needing to install those on my local operating system. I’m proud to say that I never get bugged for Java or .Net updates now. I just get the latest images and add a volume that connects to a local folder where I manage my Git repositories. It’s made my life as a developer much easier.

If you’re wondering, “What’s the big deal with containers? I just want to write code. Why do I need to use containers?” I’ll try to answer that question. Because we don’t just write code anymore. As developers and as operations engineers, we’re beginning to move into a phase where we are sharing the overall solution. This means that when I create something or have a hand in creating something, I have ownership over that thing. I’m responsible for it.

Now, you might work for an enterprise that’s still a bit behind the times. You may write code, and some other team might test that code, and then some other team might try to deploy that code. In that situation, you probably aren’t using containers or anything that looks like modern DevOps. And in that situation, the team between you and the customers who will derive value from your code is a bottleneck. If you rely on a QA team, they will find bugs, because that’s what they are incentivized to do. It’s their job to compare your code against some form of requirements and fail it if it doesn’t meet those requirements. Operations in this type of environment is incentivized to keep systems running smoothly, so they’ll look for any excuse to deny your code entry into production—that usually looks like a set of meetings designed to make sure you met all the criteria needed for something to go into production.

That is the old way of developing software. Let me tell you, if you’re working some place like that, get out. No. Seriously. Leave. Find a better job.

I believe this is the way software should be developed:

The Most Powerful Software Development Process Is The Easiest

In an ideal software development process, the only work done is understanding the problem, writing code to solve it, testing to confirm it is solved, and making progress in small steps to retain the ability to change the system when needed. The goal is to minimize work and maximize learning, allowing for changes to be made easily and with confidence.

Containers make this process easier. Your code remains modular, making it easier to version and manage libraries and dependencies. You can even build out the needed infrastructure for a container management system, such as Kubernetes, without involving operations in some cases.

As a developer and an architect, I have found that containers have improved the quality of my development life. They have allowed me to have more control over the solutions I deliver. I believe that if you start working regularly with containers, you will feel the same way too.

Connect with Shawn
Connect with Shawn

Connect with me on LinkedIn. It’s where I’m most active, and it’s the easiest way to connect with me.

Shawn’s LinkedIn
Posted on by — Leave a comment

AI Edge Engineering at University of Oxford

This week I’m wrapping up a continuing education course from Oxford focused on the approaches and tools of the AI Edge Engineering discipline. I’m overwhelmed with how much I’ve learned and how much I still have to learn. Without a doubt, this course will help set the direction of my career moving forward.

I wasn’t new to the concept of AI Edge Engineering. Before entering the course I had already passed my AZ-220 and AI-100. Two certifications I saw as fundamental to practicing AEE on the Azure platform. These certifications focus on Azure’s IoT tools with the AZ-220 and Azure’s AI services and Machine Learning tools with AI-100. It’s important to note that the AI-100 primarily focuses on applying existing AI in solutions. It only covers Data Science from a high level; you might be expected to understand what clustering, categorizing, and regression are, but you aren’t expected to build them from scratch or use any DS tools to build them for the certification test. That’s appropriate for a 100 level certification. Despite having these certifications, I wasn’t ready for the depth we would take on in the course in such a short period.

Luckily, the course used cohort learning as a mechanism to complete some of the more challenging projects. Our group efforts afforded us the opportunity to trade opinions, approaches, and skills to achieve project deliverables. This is also a skill in the AEE field. Few people will have all the skills needed to apply artificial intelligence at the edge. This means that organizations who wish to use AEE will need to team members who have varied specialized skills and knowledge of the bigger picture of AEE. Our projects made this very clear to me.

I won’t go into detail on what we learned and how we learned it, because much of that is the IP of the course and Oxford, but I will say that we dived deep in the following general areas:

  • The basic concepts of delivering AI to the edge using IoT and Edge platforms
  • Cloud- all the clouds (Azure, AWS, GCS)
  • Cloud concepts like PaaS, SaaS, and Cloud Native Services
  • All the big pieces of Machine Learning and Machine Learning concepts
  • 5G networks
  • Device design and development
  • DevOps, MLOPS, and even AIOPs

That doesn’t even cover all the guest lectures and insights into AI and AEE application demonstrated. And without the fantastic instruction of course director Ajit Jaokar and his amazing team of tutors and instructors, we wouldn’t have been able to learn so much in such a short period of time. Ajit’s passion for this specialty was clear in every class, which made it a joy to attend. It was definitely worth waking up at 3 AM to attend a class remotely just to spend the time with others who have such a strong appreciation for this burgeoning discipline. This course succeeds because of the people behind it. I have to include the choice of students as well in that success. My study group was full of passionate, knowledgeable, curious, and delightful professionals. We plan to stay in contact well after the course ends. I expect to see amazing accomplishments from their careers.

We wrap up the course on Tuesday and submit all our final homework projects over the next couple of months. I won’t be officially done with the course until May. However, I won’t ever consider myself officially done with AEE, from a learning perspective. I’m taking what I’ve learned from Oxford and building a continuing learning track to master most of the concepts covered in the course.

It’s even clearer to me now that the engineering skills of delivering greater levels of intelligence closer to the source of events and data, so that they may act upon those events and data is a skillset that will be in higher demand in the coming years. I believe this course has helped me to begin building a better roadmap toward mastery of AI Edge Engineering.