DevOps Tools: The Glue Holding 2023 Together

Walter Day

January 9, 2023

︎DevOps

My top DevOps tools for 2023

Well, what a year it has been in the DevOps space! From my dramatic move away from GitHub Actions to the rise of Tekton Pipelines and Flux for CI. But as we reflect on the past year, let's take a moment to appreciate the twists and turns shaping the 2023 automation landscape. Whether you're a fan of the big players or a fan of the underdogs, there's no denying that the CI/CD is constantly evolving and always full of surprises. Let's pour ourselves a cup of hot cocoa (or your beverage of choice) and walk down memory lane while reviewing the highlights (and maybe a few lowlights) of the past year's automation tools.

CI

Are you tired of constantly merging code changes and manually building and testing your software projects? Look no further because continuous Integration is here to save the day! Continuous Integration, or CI for short, is a software development practice where developers regularly merge their code changes into a central repository and automatically build and test their projects. Continuous Integration allows teams to detect and fix errors early on, resulting in higher-quality code and faster development cycles. So say goodbye to manual builds and hello to continuous Integration!

OUT: GitHub Actions

This past year I've left GitHub actions. I needed an engine with a more robust framework that was Kubernetes native. This led me away from GitHub Actions, not because of Github Actionsaweful but because it is limited to action-based workflows that cannot perform mitigations or integrate with monitoring to enhance the SRE layer of Kubernetes. Not to mention the value of supporting a Secure Supply chain for your deliverables.

IN: Tekton

Tekton's Event listeners have become the bread in butter of my CI/CD Kubernetes life. The ability to onboard new tenants to the cluster while still being able to post diffs and allow instant feedback to developers at scale has become something that my team can't live without. Also, building out a secure signing model is much easier with a CI engine with an integrated Supply chain.

CD

Are you ready to revolutionize your software development process with continuous delivery and GitOps? Continuous delivery is a software engineering practice in which code changes are automatically built, tested, and deployed to production. GitOps implements continuous delivery by using Git as a single source of truth for declarative infrastructure and applications. All infrastructure and application code are stored in Git repositories, and any changes to those repositories are automatically reflected in the live environment. Using GitOps with continuous delivery, you can achieve rapid and reliable software updates and ensure your code is always production ready. So why wait? Start using continuous delivery and GitOps today and take your software development to the next level.

OUT: Fleet

Fleets is an excellent Git Ops tool; while I was working with Rancher, it made the most sense as it came out of the box with Rancher and made automating Cluster API deployments super simple. The downside to Fleet, however, is the need for advanced security protocols and advanced configurations features I needed for the new Secure Software Factory I'm building.

IN: Flux

Flux checked most security boxes regarding securely automating tenant syncs and configurations. Flux's impersonation mandate for its controllers is the best in the industry. The Integration with Kyverno out of the box also allows for added security layers with CoSign and advanced image ingestion logic. Lastly, The dev team's chat space in Git Hub is super helpful, and you can usually find friendly users there. I've enjoyed the project and community and would like to start contributing to flux this year.

Infrastructure ( IAC)

Are you tired of manually configuring your infrastructure and worrying about inconsistencies and errors? Well, fear not because infrastructure as code is here to help! Infrastructure as code, or IaC, treats infrastructure in a software development environment like any other code. IaC means using version control, testing, and deployment automation to manage and provision your infrastructure, just like you would for your application code. With IaC, you can easily version, track, and automate your infrastructure, resulting in faster deployment and more consistent environments.

IN: Terraform

Tekton is an open-source project that provides tools for managing infrastructure as code (IaC) using Terraform. With Tekton, you can define your infrastructure in reusable Terraform modules and then use Tekton's pipelines to automate the provisioning and manage that infrastructure. Tekton's pipelines are written in a simple, declarative language called YAML, which makes it easy to specify the steps required to provision and manage your infrastructure. Tekton integrates seamlessly with other tools, such as version control systems and cloud providers, so you can easily manage your IaC across various environments. Whether deploying a simple application or managing a complex, multi-tier infrastructure, Tekton and Terraform can help you automate and streamline your IaC management process.

Monitoring and Metrics

Monitoring and metrics is the process of collecting and analyzing data about the performance and behavior of your software and systems. You can continuously monitor and gather metrics to identify issues and trends, optimize performance, and make informed decisions about your software and infrastructure. Tools like Prometheus and Alert manager make it easy to define events that Alert Manager can send to Slack, or use in conjunction with an automation engine like Tekton to take instant remediation against a known issue in the environment.

IN: Prometheus | Loki | Grafana

Are you ready to embark on a journey through the land of Kubernetes cluster monitoring? Grab your dice and armor, because we have a powerful trio to help us on our quest! Prometheus, Loki, and Grafana are the brave heroes we will rely on to keep our cluster running smoothly. Prometheus is the trusty warrior, constantly looking for threats to the cluster's performance and alerting us when issues arise. Loki is the cunning rogue, sneaking through the logs to gather valuable information on the inner workings of our applications. And finally, Grafana is the wise wizard, using its magical powers to conjure clear and visually appealing dashboards to help us understand the data collected by Prometheus and Loki. Together, this dynamic team will help us conquer any challenges that come our way and emerge victorious in Kubernetes cluster monitoring!

Cloud Provider

IN: GCP

Google Cloud's GKE (Google Kubernetes Engine) is a highly reliable and scalable platform for deploying and managing containerized applications. It allows for easy deployment and management of containerized applications and automatic scaling and self-healing capabilities. In addition, GKE integrates seamlessly with other Google Cloud services, such as BigQuery and Cloud Functions, making it a powerful and flexible choice for cloud computing in 2023. With my primary focus being on securing Kubernetes supply chains and software factories, GCP is hands down the best choice when picking a Kubernetes platform.

Productivity

Productivity is all about effectively using your time and resources to achieve your goals. One way to increase productivity is by leveraging great software as a service (SaaS) tools. These tools can streamline and automate various tasks, freeing your time to focus on more important things. Some examples of SaaS tools that can boost productivity include project management software, time-tracking software, and collaboration tools. Using these types of tools allows you to stay organized, keep track of your progress, and easily communicate with your team, all of which can help you be more effective in your work.

ALLSTARS

Excalidaraw

Excalidraw is the best whiteboard tool in 2023 because it is incredibly user-friendly and intuitive. It has a simple interface that allows users to quickly and easily create diagrams and visualizations. Additionally, it is a cloud-based tool that can be accessed from any device with an internet connection, making it highly convenient for teams working remotely. Excalidraw also has a wide range of features, including the ability to import and export images, collaborate with team members in real time, and customize the appearance of diagrams with different colors and fonts. Overall, Excalidraw is an excellent tool for creating professional-looking whiteboard diagrams and collaborating with team members seamlessly and efficiently. I'm looking forward to adding Excalideck (PowerPoint in Excalidraw fashion) in 2023

Copilot

I was skeptical about paying for Copilot at the start of 2023. I certainly didn't see myself as the person to pay for an AI to help me write code. Copilot boosted my productivity to that of a 10x engineer and helped me do what I wanted to do, ship valuable code that impacted my customers rather than fight with semantics and language grammar. Still, after a year of using Copilot for everything from generating Terraform module templates to asking for examples for that weird bash try-catch block, I always need to remember.

my bash profile

Shameless plug here, I update my bash profile every year with the latest functions and aliases I used last year to help me build containers and manage Kubernetes. Like my vim hotkeys, I'd be dead in the water without my functions.

Grammarly

I Still need to improve at editing written content for consumption on the internet. I need to run this through the Grammarly editor to post anything. Setting tone and emotions is extremely useful, and I trust Grammarly to help my thoughts in a way that makes even me sound less like a 5th grader.

New Contenders

Kubernetes Security tools

Are you tired of being a one-person show regarding securing your Kubernetes clusters? Well, it's time to bring in some heavy hitters to help get the job done! When it comes to Kubernetes security, you need the right tools to get the job done. And let's face it, using just a wrench and a stapler isn't going to cut it. It would be best to have the hammer of network policies, the screwdriver of role-based access control, and the drill of container runtime security. With these tools in your toolbelt, you'll be well-equipped to secure your clusters and keep your applications running smoothly. So don't be afraid to bring in some reinforcements – your clusters (and your sanity) will thank you!

Kyverno

Kyverno has proven to be a powerhouse this past year. In conjunction with Flux, this security webhook tool can provide extra layers of protection to prevent Kubernetes resources from doing things they shouldn't be able to. Kyverno will also auto-reject misconfigured YAML, which helps keep harmful code out of the cluster. Lastly, the feature I love most about Kyverno is the ability to autogenerate Cluster Policies to enforce Network Policies or service limitations on CRDs and objects in the cluster.

SigStore's CoSign

Secure Supply Chain revolved around ensuring that only those allowed to edit code have touched it. CoSign can be used to sign containers and create SBOMS. In conjunction with Kyverno, this tool makes a powerful utility to reject containers the developers haven't marked on your team. Cosign enabled me to build a Software Factory to refuse unsigned containers effectively.

Utilities

Validation tools like CEL, kubeconform, and pre-commit are essential for ensuring your code is high quality and meets all necessary standards. So make sure to take advantage of validation - incorporate these helpful tools into your workflow! CEL is a programming language that allows you to write custom rules for validating data. kubeconform is a tool that checks that your Kubernetes configuration files follow best practices. Pre-commit is a framework for managing and maintaining multiple pre-commit hooks for your code repositories. These tools can help you catch errors and issues before they become more significant problems, saving you time and effort in the long run.

CEL programming languages

I use CEL to validate the API endpoints I've created for the Software Factory I'm building. This tool allows custom logic to trigger particular Tekton Pipelines based on GitHub events. For example, if a PR is opened against a branch matching a CEL filter, a diff pipeline can automatically be run to provide feedback to my developers.

Kubeconform

I use Kubeconform with the openAPI standard to pull and validate large MonoRepos managed by Flux. In doing so, I can ensure that only code ( and YAML) pass all CRD checks and schemas before pushing to a repo.

pre-commits library of tests

You got to have pre-flight checks; developers and consumers (tenants) of your CI/CD engine need effective ways to self-lint and test their code before it becomes live in the system. Pre-commit provides an easy-to-use framework for working on my current mono repo.

Take Aways

Tekton and Flux are the dynamic duo I've been looking for to run my DevOps stacks. As for tools I'm working with right now, anything helping to shift security to the left is invaluable in this bear market. Teams need to grow more right now; solving problems with software is the right call.

Building large-scale mono repos require tools to automate testing and lint code effectively. Pre-commit helps with client-side validation before Tekton picks up the code to run validation before merging into Dev/Stage/Prod.