DevOps

Debian packaging can be quite mysterious and hard to figure out. In this guide I'll provide a simple introduction into the Debian packaging process and its most important concepts.

This is not a full guide into all aspects of packaging. Instead, I'll cover just enough of the basics to help you develop a mental model of what Debian packaging is about, and to be able to produce useful results.

I'll cover:

• What a Debian package is.
• The anatomy of a package.
• How to inspect a package.
• How to create a package.
• What APT repositories are.
• How to create an APT repository.

What's the difference between command and command | cat? There shouldn't be any, right? The first prints the output directly, and the latter prints output via cat, but they should have the same effect. Not so: the latter can cause the command to get stuck indefinitely. This has given a particular CI pipeline of mine quite some headache.

To learn why this happens, and how we can mitigate this problem, we need to dive into the arcane magic that is Unix process management. Join me on this journey.

The development and deployment of modern apps is very different from how it used to be. Networked applications are now very common. Development at scale is now common: hundreds of developers working on a system, which may be split into many microservices. Deployment at scale is now also common: apps serving millions of users. Various practices emerged, or became popular, to deal with these new challenges. I've learned many practices over the past decade, but resources that describe these practices appear to be scattered. This post is an attempt to inventorize the various practices that are used for the development, deployment and management of modern apps.

I have been researching how the Kubernetes Ingress system works. My use case is to setup an autoscaled Nginx cluster that reverse proxies to Pods in multiple Deployments. It wasn't immediately obvious how to do this. By default, Pods in Kubernetes are not supposed to be reachable from outside the cluster. One makes them reachable either by associating those pods with a Service of the right type (i.e. either NodePort or LoadBalancer), or by defining an Ingress. But what is an Ingress? How do I put Nginx in between an Ingress and a set of Pods? This post describes my journey through the jargon-loaded Kubernetes documentation which does not hold any hands, as well as my journey through the Kubernetes source code, all in a quest to find answers.

This post a bit long, so if you just want a summary then you can skip straight to the conclusion at the bottom.

Even though I rejected Prometheus as a choice in my last blog post about Netdata, I actually appreciate Prometheus' engineering quality. From its documentation it is apparent that the authors are very experienced on the subject and have thought through things.

This post reviews some of the things that demonstrate that, namely their responses to the push vs pull debacle, the way they limit Prometheus' scope, the way their alerting system is designed and documented, and the way they treat storage.

I have been looking for an easy-to-use monitoring solution for Phusion's servers. One that does not require a lot of setup and that provides a reasonable interface without too much work. Such a solution has to display a bunch of graphs at the very least. (Email) alerting is considered a bonus. The solution also has to be open source, not only because of the cost factor but because I want to own my data. So solutions like New Relic and Datadog are out.

In this blog post I will describe the solutions that I've checked out – Ganglia, Monit, Munin, Prometheus, Grafana – and why I didn't like them. Then I will explain why I think Netdata is a good choice and review its pros and cons.