This short note explains how to use OpenStackSDK to make raw HTTP queries against OpenStack services (including Ironic) in an easy and convenient way. It is very handy when testing new API features or trying to understand how API works.

Enter OpenStackSDK

Surprisingly to no one, OpenStackSDK is the official Python SDK for OpenStack. It features support for many OpenStack projects including Ironic. Through the keystoneauth library it supports all possible authentication methods, including standalone authentication (HTTP basic and none).

More importantly, it supports the modern way for providing authentication and client configuration: clouds.yaml files. Such files can be written by hand, but many installation tools, including Bifrost, generate them.

Here is ~/.config/openstack/clouds.yaml on one of my testing labs:

# WARNING: This file is managed by bifrost.
clouds:
  bifrost:
    auth_type: "http_basic"
    auth:
      username: "bifrost_user"
      password: "R7fugYZSHwtAE,BtHm"
      endpoint: http://192.168.122.1:6385
    baremetal_introspection_endpoint_override: http://192.168.122.1:5050
  bifrost-admin:
    auth_type: "http_basic"
    auth:
      username: "admin"
      password: "JmZsE7B,tWiNqjA4O5"
      endpoint: http://192.168.122.1:6385
    baremetal_introspection_endpoint_override: http://192.168.122.1:5050

It contains both authentication credentials, as well as endpoint configuration. More configuration is possible, for example, you can force a certain API version.

Now all you need is to select a cloud, which can be done e.g. with a shell variable

export OS_CLOUD=bifrost

And any code that is based on OpenStackSDK will work! Let us try:

$ python
Python 3.6.8 (default, Aug 24 2020, 17:57:11)
[GCC 8.3.1 20191121 (Red Hat 8.3.1-5)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import openstack
>>> conn = openstack.connect()
>>>

Proxies and Adapters

OpenStackSDK has three levels of functionality. Directly on the connection object you have high-level methods that often operate on several API entities and even services. We won't talk about them today.

Existing on the connection object are also attributes corresponding to service types, for example:

>>> bm = conn.baremetal
>>> insp = conn.baremetal_introspection

The resulting objects, called proxies, represent two remaining layers of functionality:

They contain low-level methods that more or less directly corresponds to API calls. They return smart objects and handle microversions automatically. For example:
```
>>> for node in bm.nodes(provision_state='active'):
...     print(node.id, node.name)
...
878c3113-0035-5033-9f99-46520b89b56d testvm2
```
The proxies are pretty well documented in OpenStackSDK, e.g. bare metal proxy and introspection proxy.
They are also keystoneauth Adapters preconfigured to use the specific service (in this case - Ironic). This functionality is often overlooked, that's why I want to talk about it today.

Wait, microversions?

Aha, I knew you would notice! You see, some OpenStack services have versioned API, Ironic is among them.

The very first thing you need to know about microversions is that the prefix micro- does not mean anything to you. It does mean something to us, but applying common sense to the word micro will lead you in the opposite direction from the truth. More precisely, it does not imply that the changes between versions are small. They can be as tiny as one parameter addition or as large as a new API endpoint family (or even a complete API rewrite, which fortunately has never happened in reality). I find the name pretty horrible, so I'll be saying API version from now on.

The second thing you need to know is that API versions are independent and isolated. Which means, if you're using API version 1.42, you cannot use anything introduced after it, e.g. in API version 1.44, even if the server supports it! By using an API version you're limiting yourself to any functionality introduced up to it.

What if no API version is provided? The third rule: no API version means the lowest API version. This is very important for us today since keystoneauth does not provide any API version by default. Higher-level OpenStackSDK API does try its best to hide the existence of API versions from its consumers.

The fourth and final rule: API versions are sequential. Each service supports a continuous range, gaps are not allowed. The practical consequence of it is that we only need to learn the minimum and the maximum supported version.

Why are we doing all this versioning? There is a pretty long document explaining the reasons. Although I'm personally not really fond of some aspects of it, these are the rules we live by.

The practical aspect of this is that you need to know which API version to use with each request, keeping in mind that API may behave quite differently between versions. For Ironic, check the API version history, other services maintain similar documents.

Looking around

Empowered by the new knowledge, we can look into the adapter object. Let us start with the basics:

>>> bm.get_endpoint()
'http://192.168.122.1:6385'
>>> bm.get_api_major_version()
(1, 0)

The adapter knows how to reach Ironic and its major API version. Nice! What else can we learn?

>>> ep_data = bm.get_endpoint_data()
>>> ep_data.max_microversion
(1, 69)
>>> ep_data.min_microversion
(1, 1)

Aha, that's these dreaded microversions. Now we know that our Ironic supports API versions from 1.1 to 1.69. What else?

>>> ep_data.url
'http://192.168.122.1:6385/v1/'

That's the final required bit: the versioned API URL for the current major version. We needn't use it directly, it's enough that the adapter knows it. API versions, on the other hand, are very useful: they're telling us which features are available!

Get it!

What else do we need to know? Obviously what the API looks like! OpenStack API guides exist for most services, we need the bare metal API reference. Using it let us try our first query:

>>> bm.get("/v1/nodes")
<Response [200]>

Hmm, that was.. informative? Sure, we got a requests library Response object, we need to extract JSON out of it:

>>> bm.get("/v1/nodes").json()
{'nodes': [{'uuid': '4e41df61-84b1-5856-bfb6-6b5f2cd3dd11', 'instance_uuid': None, 'maintenance': False, 'power_state': 'power off', 'provision_state': 'manageable', 'links': [{'href': 'http://192.168.122.1:6385/v1/nodes/4e41df61-84b1-5856-bfb6-6b5f2cd3dd11', 'rel': 'self'}, {'href': 'http://192.168.122.1:6385/nodes/4e41df61-84b1-5856-bfb6-6b5f2cd3dd11', 'rel': 'bookmark'}]}, {'uuid': '878c3113-0035-5033-9f99-46520b89b56d', 'instance_uuid': None, 'maintenance': False, 'power_state': 'power on', 'provision_state': 'active', 'links': [{'href': 'http://192.168.122.1:6385/v1/nodes/878c3113-0035-5033-9f99-46520b89b56d', 'rel': 'self'}, {'href': 'http://192.168.122.1:6385/nodes/878c3113-0035-5033-9f99-46520b89b56d', 'rel': 'bookmark'}]}]}

This is great, but we can do even better! Since v1 is the only major bare metal API version supported (or existing), we can omit the prefix:

>>> bm.get("/nodes").json()
{'nodes': [{'uuid': '4e41df61-84b1-5856-bfb6-6b5f2cd3dd11', 'instance_uuid': None, 'maintenance': False, 'power_state': 'power off', 'provision_state': 'manageable', 'links': [{'href': 'http://192.168.122.1:6385/v1/nodes/4e41df61-84b1-5856-bfb6-6b5f2cd3dd11', 'rel': 'self'}, {'href': 'http://192.168.122.1:6385/nodes/4e41df61-84b1-5856-bfb6-6b5f2cd3dd11', 'rel': 'bookmark'}]}, {'uuid': '878c3113-0035-5033-9f99-46520b89b56d', 'instance_uuid': None, 'maintenance': False, 'power_state': 'power on', 'provision_state': 'active', 'links': [{'href': 'http://192.168.122.1:6385/v1/nodes/878c3113-0035-5033-9f99-46520b89b56d', 'rel': 'self'}, {'href': 'http://192.168.122.1:6385/nodes/878c3113-0035-5033-9f99-46520b89b56d', 'rel': 'bookmark'}]}]}

A quiz: what API version (aka microversion) are we using here?

3...

2...

1...

A careful reader already knows that the lowest API version, 1.1, is used by default.

With versions

API versions are passed through headers, but needn't worry about it: keystoneauth already supports microversions! Let us compare:

>>> list(bm.get("/nodes/878c3113-0035-5033-9f99-46520b89b56d").json().keys())
['uuid', 'created_at', 'updated_at', 'console_enabled', 'driver', 'driver_info', 'extra', 'instance_info', 'instance_uuid', 'last_error', 'maintenance', 'maintenance_reason', 'power_state', 'properties', 'provision_state', 'provision_updated_at', 'reservation', 'target_power_state', 'target_provision_state', 'links', 'chassis_uuid', 'ports']
>>> list(bm.get("/nodes/878c3113-0035-5033-9f99-46520b89b56d", microversion="1.69").json().keys())
['uuid', 'created_at', 'updated_at', 'automated_clean', 'bios_interface', 'boot_interface', 'clean_step', 'conductor_group', 'console_enabled', 'console_interface', 'deploy_interface', 'deploy_step', 'description', 'driver', 'driver_info', 'driver_internal_info', 'extra', 'fault', 'inspection_finished_at', 'inspection_started_at', 'inspect_interface', 'instance_info', 'instance_uuid', 'last_error', 'lessee', 'maintenance', 'maintenance_reason', 'management_interface', 'name', 'network_data', 'network_interface', 'owner', 'power_interface', 'power_state', 'properties', 'protected', 'protected_reason', 'provision_state', 'provision_updated_at', 'raid_config', 'raid_interface', 'rescue_interface', 'reservation', 'resource_class', 'retired', 'retired_reason', 'storage_interface', 'target_power_state', 'target_provision_state', 'target_raid_config', 'vendor_interface', 'links', 'traits', 'conductor', 'allocation_uuid', 'chassis_uuid', 'ports', 'states', 'portgroups', 'volume']

Wow, such a difference! Well, 1.1 appeared in the Kilo release cycle around 2015 :) Now you can test things that are not yet implemented in the official CLI, e.g. the new feature I've been developing recently:

>>> bm.put("/nodes/testvm1/states/provision", microversion='1.70', json={'target': 'clean', 'disable_ramdisk': True, 'clean_steps': [{'step': 'print_node', 'interface': 'deploy'}]})

(yes, 1.70 is not released at the moment of writing, hence 1.69 above). Or you can play with the private API that our ramdisk uses:

>>> [port['address'] for port in bm.get("/ports").json()['ports']]
['52:54:00:9e:b1:16', '52:54:00:cd:6f:46']
>>> bm.get("/lookup?addresses=52:54:00:9e:b1:16", microversion='1.22').json()
{'node': {'uuid': '4e41df61-84b1-5856-bfb6-6b5f2cd3dd11', 'driver_internal_info': {'agent_erase_devices_iterations': 1, 'agent_erase_devices_zeroize': True, 'agent_continue_if_ata_erase_failed': False, 'agent_enable_ata_secure_erase': True, 'disk_erasure_concurrency': 1, 'agent_erase_skip_read_only': False, 'last_power_state_change': '2021-03-11T17:00:31.557062', 'agent_version': '6.6.1.dev8', 'agent_last_heartbeat': '2021-02-25T17:30:26.263005', 'hardware_manager_version': {'generic_hardware_manager': '1.1'}, 'agent_cached_clean_steps_refreshed': '2021-02-16 16:44:04.143865', 'clean_steps': None, 'deploy_steps': None, 'agent_cached_deploy_steps_refreshed': '2021-02-25 17:27:53.998316'}, 'instance_info': {}, 'properties': {'cpu_arch': 'x86_64', 'disk_gb': '24'}, 'links': [{'href': 'http://192.168.122.1:6385/v1/nodes/4e41df61-84b1-5856-bfb6-6b5f2cd3dd11', 'rel': 'self'}, {'href': 'http://192.168.122.1:6385/nodes/4e41df61-84b1-5856-bfb6-6b5f2cd3dd11', 'rel': 'bookmark'}]}, 'config': {'metrics': {'backend': 'noop', 'prepend_host': False, 'prepend_uuid': False, 'prepend_host_reverse': True, 'global_prefix': None}, 'metrics_statsd': {'statsd_host': 'localhost', 'statsd_port': 8125}, 'heartbeat_timeout': 300, 'agent_token': None, 'agent_token_required': True}}

Conclusion

OpenStack overall and Ironic specifically come with great SDK and command-line tools. When they're not enough, you don't necessarily have to leap to the good old curl: OpenStackSDK adapters are also very handy for dealing with OpenStack API!