Memory leak in a Python project

I have recently analyzed and fixed an interesting issue. One of the Metal3 components, Ironic Inspector, started showing a linear growth of consumed memory. Other containers in the Metal3 pod do not exhibit similar issues. The memory usage of the service that normally requires 100-200 MiB exceeds 1 GiB within a month of being mostly idle.

This blog post describes what I did to debug the problem, and how an unorthodox approach to resource clean-up can cause unexpected troubles down the road.

Read more …

Metal3 and Cluster API demo

This is the demo that I was supposed to present on StackConf.EU 2022, but unfortunately fell victim to The Plague (aka COVID-19).

It starts with show-casing various Metal3 components in their natural habitat, then highlights the related cluster API components. The demo ends with provisionion a new worker using a previously registered host.

Watch it …

Random thoughts on camping equipment

Knowledge sharing is a critical pillar of the civilization. After I've been asked several times to share my camping experience, I decided to write this post. It is by no means a complete guide - I'm not qualified for that. Rather a few random thoughts based on my recent experience.

Note: Contains links to web sites and online shops. They are here only for illustration purposes, I'm by no means affiliated with any of these organizations.

Read more …

How to propose the best project for Outreachy?

The project submission deadline for Outreachy is rapidly approaching: prospective mentors only have until March 4 to come up with projects for the summer cohort (May to August 2022). For those who don't know:

Outreachy (previously the Free and Open Source Software Outreach Program for Women) is a program that organizes three-month paid internships with free and open-source software projects for people who are typically underrepresented in those projects.

[from Wikipedia]

As a past mentor and one of the current coordinators of the OpenStack community in Outreachy, I would like to give a few tips for the mentors on how to get (and give!) the most of the program by picking a right project.

Read more …

Integrating coreos-installer with Ironic

In OpenShift bare metal IPI [1] we're using the Metal3 and Ironic projects for provisioning bare metal machines that later become a part of the cluster. Historically, we were using binaries from the Red Hat OpenStack, including a deploy ramdisk [2] based on Red Hat Enterprise Linux 8. However, OpenShift itself uses CoreOS - a RHEL-based distribution built specifically for container deployments.

Several months ago I was tasked with migrating from the RHEL-based to a CoreOS-based ramdisk for Metal3, as well as using coreos-installer for the installation process.

This post explains how I created a CoreOS-based ramdisk and a custom deploy process for Ironic using a new hardware manager [3]. I will not cover the complete solution, but will rather concentrate on how the extensibility and flexibility of Ironic can be used to fulfill similar tasks.

Prior reading:

Read more …

Bare Metal + Kubernetes = ♡

In this blog post, I'm talking about Metal3 (pronounced "metal-kubed"): a Kubernetes API and a cluster API provider for bare metal. I'll explain how and why it uses Ironic, and how you can use it to provision bare metal machines.

I assume that you are familiar with Kubernetes and the concept of Custom Resource Definitions (CRD). I also won't spend time explaining what cluster API is, there are enough good resources on this topic. I will only say that it's a way for Kubernetes to manage itself, in this case by provisioning (and deprovisioning) bare metal machines.

Read more …

Deploy steps tutorial

In this tutorial I'm showing how to create a custom deploy step for Ironic, how to build a ramdisk with in and how to use it when deploying a node.

Deploy steps are an answer to question "how do I run non-standard actions during deployment". Out-of-band steps run from your control plane and can talk to the BMC. More interesting for us are in-band steps that run from within the machine and offer nearly infinite opportunity for customization.

Today we'll create a solution for the following story:

As an operator, I would like to inject small files into the root partition of the final instance through the bare metal API.

There are, of course, numerous way of implementing it, cloud-init being probably the most popular. But we will concentrate on using a deploy step. The complete source code for this tutorial can be found here: https://github.com/dtantsur/ironic-inject-files/

Read more …

Ephemeral workloads with Ironic

In this post I'm presenting the ramdisk deploy interface, explaining how to use it to run ephemeral workloads and how to provide configuration data for them.

Ironic has been actively explored by the scientific community as a way to automate running calculations without incurring the costs of virtualization. This sort of workloads does not necessarily require installing anything on the machine's hard drive, which may instead be used for caching, swap or not used at all. The results are posted back via HTTP(s) or stored on a network share.

Read more …