Today I'm presenting the miniscript project and discussion the future of the ironic-inspector feature called introspection rules. If the ironic business is of no interest for you, scroll right to Enter MiniScript and stop there.

Introspection Rules

As far as I can remember the events from year 2015, introspection rules appeared in response to two requests:

A way to assign nodes to profiles, TripleO's predecessors to resource classes.
An alternative to the dreaded IPMI credentials setting

The last case was based on the (usually correct) assumption that the operators have their BMC credentials somewhere. However, they may not have an easy way to enroll all their fleet by hand. They need some sort of automation - and here is where introspection rules come into play.

As you can learn from the introspection rules documentation, ironic-inspector provides an API to create small scripts that run on the data, received from the ramdisk, and modify it or execute other pre-defined actions. For example, an operator can write the following logic:

if vendor is Dell
    and ipmi_password is not set
    set ipmi_username=root, ipmi_password=calvin

Combined with site-specific actions (written in Python), this API can cover a large variety of cases. It can not, of course, set random credentials for you (the actions are run on control plane), but I'm still not convinced anybody needs that.

The Ugly

Introspection rules solved a lot of customization problems for years. However, as people (most notably, people from CERN) started writing more rules, the ugly side of the API became apparent. It is the syntax.

You see, the initial idea for the rules was quite simple: if data matches something, set something on a node. The rule format is structured accordingly: it has conditions to check, then actions to execute. The documentation provides a typical example (I'm converting it to YAML for fair comparison later):

- description: "Set IPMI driver_info if no credentials"
  actions:
    - action: set-attribute
      path: "driver"
      value: "ipmi"
    - action: set-attribute
      path: "driver/ipmi_username"
      value: "username"
    - action: set-attribute
      path: "driver/ipmi_password"
      value: "password"
  conditions:
    - op: is-empty
      field: "node://driver_info.ipmi_username"
    - op: is-empty
      field: "node://driver_info.ipmi_password"

Some downsides are already visible, some became apparent later:

It's ugly.
It requires hand-written checks for typical Python operations.
It has a special syntax for fetching node attributes in conditions,
… which does not even match the syntax of setting attributes,
… and also does not match the syntax of fetching attributes in actions!
Conditions are joined with AND, no other options were available initially. A patch has been proposed to make the joining configurable, but it arguably makes the ugly feature even uglier.
It has no support for loops, except for a custom flag to handle multiple values in a condition.
IT. IS. UGLY!!

In my defense I can only say that I was trying to keep the implementation as simple as possible. I think it serves as a good example that the simplest is not always the best (looking at you, systemd haters). Actually, while writing this post I had to check the documentation (that I've written) to remember how exactly this feature (that I've written!) works!

The upside? It works and works reasonably well. However, I haven't stopped thinking about ways to improve/rewrite introspection rules.

Homecoming

One of the reasons I keep returning to this topic are discussions about merging some parts of ironic-inspector back into ironic. For historical context, ironic-inspector started as a series of patches to ironic back in summer 2014 (introspection rules were not a part of them), but back then the community was not fond of the idea. So I started ironic-discoverd, which later got accepted under the ironic project umbrella as ironic-inspector.

What I'm trying to say is that the separation between ironic and ironic-inspector is quite artificial and routinely confuses newcomers.

Interestingly, it is introspection rules that caused this discussion to happen. An ability to make decisions based on the node's properties is definitely useful for a wide variety of cases beyond node auto-discovery. Applying introspection rules to all nodes in ironic, including ones using out-of-band inspection methods was initially blocked by two things:

A common format of hardware inventory to process. Nowadays, the ironic-python-agent inventory format is well-documented and can be used.
Moving the introspection rules engine to ironic from its current home in ironic-inspector. Since doing that would involve deprecating one API and creating a similar in a new place, it presents a good chance to rethink the rules format.

Crazy Ideas

One idea that never left my mind is how a Lisp-like language could be expressed using JSON objects. Removing brackets and adding indents (i.e. using YAML for readability), I came up with this prototype:

---
- description: Rule 1
  actions:
    - if:
        and:
          - eq:
              42: 42
          - ~eq:
              1: 2
      then:
        set-attribute:
          foo: bar

Hacking on it in my spare time, I realized that many of the downsides are still here:

Still custom checks.
No new ideas for fetching attributes.
Still ugly and alien-looking.

Years passed by, and while working on Bifrost I became increasingly exposed to Ansible.

Enter MiniScript

I like Ansible. Well, often enough I hate it, but mostly I like it. Last weekend I got curious: how much will it take to create an embedded scripting library using Ansible-like syntax? This is how MiniScript came into life. Ansible syntax:

Uses standard Python for conditions.
Uses Jinja2 templating for accessing attributes.
Has loops and blocks.
Still a bit ugly, but at least it has a flat structure, which is arguably more readable.

MiniScript is exactly this: a scripting engine with a syntax that is very similar to Ansible, but tailored for data processing rather than remote execution. To give you an idea, here is a meaningless example from the documentation:

---
- name: only accept positive integers
  fail: "{{ item }} must be positive"
  when: item <= 0
  loop: "{{ values }}"

- name: add the provided values
  add: "{{ values }}"
  register: result

- name: log the result
  log:
    info: "The sum is {{ result.sum }}"

- name: return the result
  return: "{{ result.sum }}"

It uses the custom add action defined as:

class AddTask(miniscript.Task):
    """A task implementing addition."""

    required_params = {'values': list}
    """One required parameter - a list of values."""
    singleton_param = 'values'
    """Can be supplied without an explicit "values"."""

    def validate(self, params, context):
        """Validate the parameters."""
        super().validate(params, context)
        for item in params['values']:
            int(item)

    def execute(self, params, context):
        """Execute the action, return a "sum" value."""
        return {"sum": sum(params['values'])}

Backslash horror

Not everything went smoothly. Imagine we're trying to use the regex_replace filter with a substitution, equivalent to the following Python code:

import re
print(re.sub('with_(.*)', 'without_\\1', 'with_test'))
# -> without_test

How many backslash signs will an equivalent script have in YAML format? Let's try!

---
- return: "{{ 'with_test' | regex_replace('with_(.*)', 'without_\\1') }}"

Correct? Yes, but not in the first naive implementation I ended up with! Actually, you are supposed to use 4 slashes, this is why:

First, YAML processes backslashes, treating \\ as a single backslash.
Then Jinja sees \1 inside a string, which is not a valid escape sequence!

Apparently, Ansible folks have been aware (issue 11891) of this behavior and created a work-around. I took the same approach of pre-processing all templates and replacing 2 backslashes with 4 in string constants.

All to make the reasonably readable snippet above actually work.

Quotation horror

Let me give you a funnier one! What if we try to surround the resulting string in quotation marks?

---
- return: "{{ 'with_test' | regex_replace('with_(.*)', '\"without_\\1\"') }}"

Looks reasonable, right? We added quotation marks, gave them one backslash (per Backslash horror), so the result is "without_test".

Wrong.

Let us follow the calculation (and ignore the regular expression because I only added it to confuse you):

YAML takes "{{ ... }}" and yields a string {{ ... }}.
Jinja takes {{ ... }} and evaluates "without_test"
which is a Python string, its content is without_test.

The result is without_test. This is because we're not operating on strings, we're operating on Python values. However, for convenience Jinja defaults to assuming strings, so if you remove one of the quotation marks, the example will work as expected:

---
- return: "{{ 'with_test' | regex_replace('with_(.*)', '\"without_\\1') }}"
  # -> "without_test

I'm very annoyed by this behavior, but it seems that there is no way around it.

Rules 2.0

If we recreate introspection rules based on MiniScript, this is how the example above may end up looking:

- name: "Set IPMI driver_info if no credentials"
  node.set:
    driver: "ipmi"
    driver_info/ipmi_username: "username"
    driver_info/ipmi_password: "password"
  when:
    - node.driver_info.ipmi_username is undefined
    - node.driver_info.ipmi_password is undefined

Better? Well, it looks much better to me. And a person familiar with Ansible and Python can likely understand it without deep knowledge of MiniScript or ironic. You can use the full power of Python for conditions and the full power of Jinja2 for templating, e.g.

- vars:
    ipmi_address: "{{ data.inventory.bmc_v6address or data.inventory.bmc_address }}"

- node.set:
    driver_info/ipmi_address: "{{ ipmi_address }}"
  when:
    - node.driver_info.ipmi_address is undefined
    - ipmi_address != '0.0.0.0'
    - ipmi_address != '::/0'

- fail: BMC address is not provided
  when: node.driver_info.ipmi_address is undefined

(I don't want to imagine the equivalent version using the old rules).

On top of that, it may be possible to automatically convert the old rules into the new format:

Each rule becomes a block with its actions.
Each condition - a when clause on the block.
node://field.subitem values become {{ node.field.subitem }}.
{python_formatting} becomes {{ jinja_formatting }}.

Looks great, when?

Here is bad news: I'm unlikely to find substantial time to continue this work. Creating MiniScript was probably my biggest contribution, but I'll be very open to mentoring anyone who wants to drive it to completion.

Note that it probably does not make sense to change the existing API. Instead, I would like to see it revived in ironic proper, with ironic-inspector providing raw data for consumption. This way two implementations can co-exist for long time and operators can opt into one or the other.

Stop by #openstack-ironic IRC channel on OFTC if you're interested to help!