This post was originally published on the
XLAB Steampunk blog.
It has been a while since our last hardcore technical post. So we decided to put steamy cloud posts aside for a moment and get down and dirty with one of the new features of Ansible Core 2.11: argument specification for Ansible roles.
We will start with a short description of how we can parameterize Ansible roles and what problems we can expect when using them. Next, we will give a high-level overview of the argument specification for Ansible roles, describe its benefits to Ansible playbook and role authors, and go over a simple example.
And for those of you who will stick with us to the bitter end, we will even throw in a short story about an argument specification development.
Sounds like an (in)sane plan? Good ;)
Prerequisites
If you would like to follow along, you will need to install the ansible-core
Python package. To prevent making a mess out of your system, you should install
Ansible Core into a new virtual environment:
$ python3 -m venv venv
$ . venv/bin/activate
(venv) $ pip install --pre ansible-core
The last command installs a prerelease version of Ansible Core. That version contains a critical bug fix that we require and is not part of the latest stable release yet. Stick to the end if you are interested in gory details.
You will also need to have the Sensu Go Ansible Collection installed:
(venv) $ ansible-galaxy collection install "sensu.sensu_go:<1.11"
With the installation out of the way, we can start talking about Ansible roles and their arguments.
Role arguments
Strictly speaking, Ansible roles do not have arguments. But we can parameterize them with variables. Typically, Ansible playbooks look something like this:
---
- name: Install Sensu components
hosts: all
become: true
tasks:
- name: Install backend
include_role:
name: sensu.sensu_go.install
vars:
components: [ sensu-go-agents ]
Of course, we do not have to set the variables on the task itself. Variable values can come from a wide variety of sources, but we defined them close to the include task for the sake of simplicity.
The most common way of documenting Ansible role variables today is to describe them in a readme document. Why? Because Ansible Galaxy knows how to display it. The Apache Cassandra Ansible role contains a great example.
With the introduction of Ansible Collections, things became a bit more complex. Since Ansible Collections can contain more than one Ansible role, it is almost impossible to place all that information into a single readme file without making it obscenely long. For the Sensu Go Ansible Collection, we solved this issue by publishing a dedicated documentation site. Each Sensu Go role has a dedicated section there.
Documenting Ansible role variables does give all required information to Ansible playbook authors. However, there is still one major problem: we rely on playbook authors not to make typing and copy-paste mistakes. And to make matters even worse, such typos usually manifest themselves relatively late - when Ansible tries to access the variable value deep inside the Ansible role.
For example, if we run our Ansible playbook from the start of this section, we will see output similar to this:
PLAY [Install Sensu components] ******************************************
TASK [Gathering Facts] ***************************************************
ok: [default]
TASK [Install backend] ***************************************************
TASK [sensu.sensu_go.install : Prepare package repositories] *************
included: /home/tadej/.../tasks/repositories.yml for default
TASK [sensu.sensu_go.install : Prepare package repositories] *************
included: /home/tadej/.../tasks/dnf/prepare.yml for default
TASK [sensu.sensu_go.install : Include distro-specific vars (CentOS)] ****
ok: [default]
TASK [sensu.sensu_go.install : Add yum repository] ***********************
changed: [default]
TASK [sensu.sensu_go.install : Add yum source repository] ****************
changed: [default]
TASK [sensu.sensu_go.install : Install selected packages] ****************
included: /home/tadej/.../tasks/packages.yml for default
TASK [sensu.sensu_go.install : Install selected components (Linux)] ******
included: /home/tadej/.../tasks/dnf/install.yml for default
TASK [sensu.sensu_go.install : Install component] ************************
failed: [default] (item=sensu-go-agents) => {
"ansible_loop_var": "item",
"changed": false,
"failures": ["No package sensu-go-agents available."],
"item": "sensu-go-agents",
"msg": "Failed to install some of the specified packages",
"rc": 1,
"results": []
}
PLAY RECAP ***************************************************************
default : ok=8 changed=2 unreachable=0 failed=1
skipped=0 rescued=0 ignored=0
As you can see, Ansible executed quite a few tasks before one of them failed. And the error message is not especially helpful here because it depends on the module that failed. (Can you spot the error in our Ansible playbook?)
One way of making sure all required variables are set and contain acceptable
values is to add an ansible.builtin.assert task at the start of
each task file in the Ansible role. This way, Ansible can error-out at the
beginning of the Ansible role run.
Adding an assert task at the start of the task file has one major downside: Ansible role maintainers must keep assertions in sync with the documentation. But if we could get rid of that information duplication, we’d’ be golden. This is precisely what argument specification allows us to do.
Argument specification
Argument specification is, at its essence, machine-executable Ansible role documentation:
- It serves as a source from which
ansible-doccan produce reference documentation for an Ansible role. ansible-playbookuses it to validate variable values before the Ansible role gets executed.
If we now update our Sensu Go Ansible Collection to the latest stable version,
ansible-doc will produce the following output:
(venv) $ ansible-galaxy collection install "sensu.sensu_go:>=1.11"
(venv) $ ansible-doc --type role --list
sensu.sensu_go.agent configure Configure Sensu Go agent
sensu.sensu_go.agent start Start Sensu Go agent
sensu.sensu_go.agent main Install, configure, and start Sensu Go
sensu.sensu_go.backend configure Configure Sensu Go backend
sensu.sensu_go.backend start Start Sensu Go backend
sensu.sensu_go.backend main Install, configure, and start Sensu Go
sensu.sensu_go.install repositories Enable Sensu Go repos
sensu.sensu_go.install packages Install selected Sensu Go packages
sensu.sensu_go.install main Enable Sensu Go repos and install
Role names should look familiar to anyone who used the Sensu Go Ansible Collection before. But why are there three lines for each Sensu Go Ansible role, and what do the values in the second column mean?
All Sensu Go Ansible roles are modular. Each of them has three entry points
that ansible-doc lists in the second column. And if you never heard of entry
points: they are task files that Ansible playbook authors can import
individually from an Ansible role.
Most Ansible roles only have one entry point called main that Ansible
executes by default when we include an Ansible role. But Ansible playbook
authors can also select a different entry point by setting the tasks_from
parameter to a non-default value.
We can get the documentation for the main entry point that we are using in
our sample Ansible playbook by running the following command:
(venv) $ ansible-doc --type role --entry-point main sensu.sensu_go.install
> SENSU.SENSU_GO.INSTALL
(/home/tadej/.ansible/collections/ansible_collections/sensu/sensu_go)
ENTRY POINT: main - Enable Sensu Go repos and install selected packages
The main entry point just combines the repositories and
packages entry points.
OPTIONS (= is mandatory):
- build
Package build to install.
Can be any valid build string such as `8290' or a special
value latest.
If the `version' variable is set to latest, this variable is
ignored and the latest available build is installed.
[Default: latest]
type: str
- channel
Repository channel that serves as a source of packages.
Visit the packagecloud site to find all available channels.
[Default: stable]
type: str
- components
List of components to install.
(Choices: sensu-go-backend, sensu-go-agent, sensu-go-
cli)[Default: ['sensu-go-backend', 'sensu-go-agent', 'sensu-
go-cli']]
elements: str
type: list
- version
Package version to install.
Can be any valid version string such as `6.2.5' or special
value `latest'.
[Default: latest]
type: str
The output of the ansible-doc command more or less replicates the
documentation available on our documentation site.
If we now re-run the ansible-playbook command from before, we will also get a
different output:
PLAY [Install Sensu components] ******************************************
TASK [Gathering Facts] ***************************************************
ok: [default]
TASK [Install backend] ***************************************************
TASK [sensu.sensu_go.install : Validating arguments against arg ] ********
fatal: [default]: FAILED! => {
"argument_errors": [
"value of components must be one or more of: sensu-go-backend,
sensu-go-agent, sensu-go-cli. Got no match for: sensu-go-agents"
],
...
}
PLAY RECAP ***************************************************************
default : ok=1 changed=0 unreachable=0 failed=1
skipped=0 rescued=0 ignored=0
Because the sensu.sensu_go.install Ansible role in the latest Sensu Go
Ansible Collection version has an argument specification, Ansible automatically
inserted a validation task before it started executing tasks from the main
entry point. And the error message is clear now: we entered an invalid
component name. Awesome!
And now that we know how to use the argument specification let us take a stab at writing a simple one ourselves.
Writing argument specification
The argument specification we will dissect here will only have the main entry
point with a single components argument. So just enough to cover the use case
we were playing through the post. And without any further ado, here it is in
all of its glory:
argument_specs:
main:
short_description: Enable Sensu Go repos and install selected packages
description:
- The main entry point just combines the repositories and packages
entry points.
options:
components:
description:
- List of components to install.
type: list
elements: str
choices:
- sensu-go-backend
- sensu-go-agent
- sensu-go-cli
default:
- sensu-go-backend
- sensu-go-agent
- sensu-go-cli
The first six lines should be pretty self-explanatory: they attach some
human-readable information to the main entry point. Ansible adds this
information to the generated documentation.
The remainder of the specification is where the documentation meets validation.
In our case, the components variable must conform to the following rules:
- It must be a list of strings (
type: listandelements: str). - It can only contain sensu-go-backend, sensu-go-agent, and sensu-go-cli strings.
- By default, the variable will hold all valid package names.
Once we place the argument specification into the meta/argument_specs.yml
file, Ansible will automatically use it to validate variables before executing
an Ansible role.
For more information on writing argument specifications, we can visit the official documentation that contains more details and some additional samples. Or browse through the roles directory in the GitHub repository for the Sensu Go Ansible Collection.
And now for the best part of today’s post: storytime!
A tale where CI saves the day
We added an argument specification to all roles in the Sensu Go Ansible Collection right after the Ansible Core 2.11 saw its first stable release. Things went suspiciously smoothly: it took us about an hour to transform our documentation into argument specifications.
When we tested new functionality locally using Ansible Core 2.11.0, all checks were green. So we polished things a bit and created a pull request. Then all hell broke loose.
We run our integration tests against all supported Ansible versions. And it turned out that the introduction of argument specification broke backward compatibility with Ansible 2.9 and Ansible Base 2.10, which is a no-go for a certified collection.
A quick chat with the core developers confirmed that this is indeed a bug in Ansible Core. Fortunately for us, Ansible core developers are fantastic, and they prepared a fix for this in no time at all. The bugfix did not make it to the stable release yet, so this is why we are using the prerelease version of Ansible Core in this post.
What did we learn
If everything went according to plan, you now know more about Ansible role parameterization and how argument specification helps you make your automation setup more robust. And if you did not know about entry points before, well, now you do ;)
We also learned that the quality of your Ansible collection depends heavily on the quality of your test suite and that we should try to test new features before the latest stable version of Ansible is out. If you want to learn more about testing Ansible collections, we recommend watching the Intro to testing Ansible Collections webinar.
Also, if you need help creating or upgrading your Ansible content, feel free to contact us. We will do our best to help.
Cheers!
