Let us give Ansible a REST

19. junij 2019 - Avtor Tadej Borovšak

19. junij 2019
Avtor Tadej Borovšak

It is becoming increasingly difficult to find a home or office product that cannot be controlled using some kind of web application programming interface (API). Smart light bulbs, robotic lawn mowers, espresso machines - they all have it. Wouldn’t it be cool if we could use Ansible to control them?

Introducing PMS plugin

We will start small and create three ping my service (PMS) Ansible plugins today. And yes, they will all be just as useless as the real PMS is ;)

The only thing these plugins will do is send an authenticated GET request to a selected endpoint and check if the service responds. You know, just like regular ping command does, but with more sessions and Ansible.

Why three plugins? Because we can ;) And because we want to compare three different approaches to interfacing with the web APIs.

We will start with an ordinary Ansible module that we all know and love, continue on with the action plugin, and end with a combination of an Ansible module and a connection plugin.

Because messing around with our espresso machine is out of the question (you do not want to meet a non-caffeinated developer), let us introduce a simple mock service that we will be using to test our Ansible plugins.

Mock web API

The mock server that we will be using throughout this post is available in the GitHub repo. If you would like to follow along, now it would be a good time to clone the repo, start your terminal emulators, tame the snakes, and wake up your curl. Ready? Great.

The mock API is somewhat inspired by the Redfish API, which means that we should create a session before performing any other requests and delete it after we are done.

We will begin the test by executing the following command:

$ python3 server.py

This will start the server and wait for the incoming connections. Now we need to open a new terminal window where the curl magic will happen. First, we need to log in:

$ curl \
    -v \
    -X POST \
    -H "content-type: application/json" \
    -d '{"username": "user", "password": "pass"}' \
    localhost:8000/tokens 2>&1 \
  | grep x-auth-token

If nothing went awry, the previous command printed the x-auth-token header that the API returned and that we should add to our actual work requests like this:

$ curl -H "x-auth-token: 123" localhost:8000/test/me

When we are done playing, we should destroy the session. A simple DELETE /tokens/123 request will delete the session and release the resources on the server:

$ curl -X DELETE -H "x-auth-token: 123" localhost:8000/tokens/123

And this is all there is to it. If we have not screwed up anything majorly, the server should respond to requests that target /test* endpoints.

Time to turn the heat up and start teaching Ansible about our newest toy.

Ansible module

We will start with an Ansible playbook that replicates the actions that we did with curl: create a new session, ping the selected endpoint, and log out. By the way, all the code that we are showing and discussing here is available in the module directory of the accompanying git repo.

- name: Ping it like you mean it
  hosts: localhost
  gather_facts: no
  tasks:
    - name: Check if the service is available
      pms:
        auth:
          address: http://localhost:8000
          username: user
          password: pass
        endpoint: /test/me

If you have seen at least one Ansible playbook before, you should feel right at home. The pms Ansible module code is stored in library/pms.py file and can be logically split into two parts:

the connector part, which is the component that talks to the service, and
the executor part that controls the connector.

If we look at the relevant parts of the actual code, we find this:

class Connection:
    # Connector implementation here

def main():
    # Parameter parsing here
    conn = Connection(module.params["auth"]["address"])
    conn.login(module.params["auth"]["username"],
               module.params["auth"]["password"])
    status, _, data = conn.get(module.params["endpoint"])
    conn.logout()
    module.exit_json(changed=True, status=status, response=str(data))

Squinting a bit at this code, we can find our curl calls from the introductory section hidden in this code under the conn.login(), conn.get() and conn.logout() names.

When we run the ansible-playbook command, this file is bundled up with some other Ansible parts, copied to the localhost and executed. We can check this if we run Ansible in verbose mode:

$ ansible-playbook -vvv play.yaml

Is the output of this command not familiar to you? Maybe you should check our post about Ansible internals. We will wait ;)

Next on our list: the action plugin implementation.

Action plugin

We already learned in our post about Ansible internals that action plugin is a control node part of the Ansible module and is responsible for packaging up the module code that is then transmitted to the host. But in use-cases like ours, where sending the module code around is not desired (we are specifying the localhost as our host, remember?), we can use the action plugin to perform the actual work and sidestep the sending part altogether.

Transforming the regular module into an action plugin is really straightforward: we just move the actual code from the library/pms.py into action_plugins/pms.py and do some minor modifications around the parameter parsing.

class Connection:
    # Connector implementation here

class ActionModule(ActionBase):
    # Constructor code here
    def run(self, tmp=None, task_vars=None):
        # Parameter parsing here
        conn = Connection(self._task.args["auth"]["address"])
        conn.login(self._task.args["auth"]["username"],
                   self._task.args["auth"]["password"])
        status, _, data = conn.get(self._task.args["endpoint"])
        conn.logout()

        result.update(status=status, response=str(data))
        return result

If you are interested in the nitty-gritty details, we have the action plugin implemented in the action directory inside the git repo.

We do not have to change the playbook at all since the parameters have not changed. But running the Ansible in talk-to-me-about-everything mode reveals that we now skip the bundle and transmit stages and get straight to executing the code.

We are almost done. One more variation and we are off the hook.

Connection plugin

Up until now, we kept the connector and the code controlling it together in the same file, be it regular module or action plugin. But in this section, we will place the connector part into a custom connection plugin. This has several implications that we will address now.

The first thing we can do is remove the auth part of the playbook since module no longer needs them. The modified playbook looks like this:

- name: Ping it like you mean it
  hosts: all
  gather_facts: no
  tasks:
    - name: Check if the service is available
      pms:
        endpoint: /test/me

Because the connection plugin will handle the authentication and talking to the service for us now, we can simplify our Ansible module into just a few lines:

def main():
    # Parameter parsing here
    conn = Connection(module._socket_path)
    status, _, data = conn.get(module.params["endpoint"])
    module.exit_json(changed=True, status=status, response=str(data))

The authentication data we removed from the playbook should be placed in the inventory file where the connection plugin can get hold of it. In our case, the inventory looks like this:

all:
  hosts:
    my_host:
      ansible_connection: pms
      ansible_pms_address: http://localhost:8000
      ansible_pms_username: user
      ansible_pms_password: pass

The ansible_connection part tells Ansible to use our custom pms connection plugin and the ansible_pms_* fields hold the actual authentication data.

Just like before, the complete plugin sources are available from the connection directory in the git repo.

Now we are ready to run the Ansible playbook. Because we placed the authentication data in the inventory file, we need to specify it on the command line:

$ ansible-playbook -vvv -i inventory.yaml play.yaml

Make sure you are using Ansible 2.9.0 or newer to run this command. Older Ansible releases use different persistent connection API that is not compatible with the newer one.

All that is left for us to do is compare the three implementations and try to learn something from this exercise in reverse-engineering the Ansible internals.

And the bestestestest plugin is …

… well, it depends on the use-case we are trying to solve. And yes, we need to have a reasonably well-defined problem that we are trying to solve. Otherwise, we are better off with no plugin at all. So, let us come up with a few different use cases.

One of the possible scenarios could be: we want to check one of our endpoints every few hours. In this case, an action plugin suffices. And because this kind of plugin is the simplest to debug, we just made our future life much more comfortable.

Note that since action plugins are always executed on the Ansible control node, we would need to switch to the regular Ansible module if we would like to ping our service from a different host. Thankfully, this change does not complicate the plugin’s code much, and we can still debug it with relative ease.

Let us now assume that we hit the jackpot, and millions of users started using our product on a daily basis. Because we had to scale our service, we need to ping several different endpoints.

Now, changing the playbook is straightforward: we add a loop over our endpoints, and we are done. The modified playbook would look something like this:

- name: Ping it like you mean it
  hosts: all
  gather_facts: no
  tasks:
    - name: Check if service is available
      pms:
        endpoint: /test/{{ item }}
      loop: "{{ range(0, 3) | list }}"

If we execute this playbook when using Ansible module or action plugin, the mock server will log the following requests:

127.0.0.1 - - [10/Jun/2019 10:40:25] "POST /tokens HTTP/1.1" 201 -
127.0.0.1 - - [10/Jun/2019 10:40:25] "GET /test/0 HTTP/1.1" 200 -
127.0.0.1 - - [10/Jun/2019 10:40:26] "DELETE /tokens/123 HTTP/1.1" 204 -
127.0.0.1 - - [10/Jun/2019 10:40:26] "POST /tokens HTTP/1.1" 201 -
127.0.0.1 - - [10/Jun/2019 10:40:27] "GET /test/1 HTTP/1.1" 200 -
127.0.0.1 - - [10/Jun/2019 10:40:27] "DELETE /tokens/123 HTTP/1.1" 204 -
127.0.0.1 - - [10/Jun/2019 10:40:28] "POST /tokens HTTP/1.1" 201 -
127.0.0.1 - - [10/Jun/2019 10:40:28] "GET /test/2 HTTP/1.1" 200 -
127.0.0.1 - - [10/Jun/2019 10:40:29] "DELETE /tokens/123 HTTP/1.1" 204 -

Each GET request that checks if the endpoint is available is clamped between POST and DELETE request. And this makes sense because we log in and log out in the code every time the module is executed. Usually, authentication is not the cheapest operation to perform, and this means we have a problem on our hands that we better solve before we have more than three endpoints to check.

This is where our connection plugin starts to shine. If we execute the playbook that uses our custom connection plugin, we get the following output from the mock server:

127.0.0.1 - - [10/Jun/2019 10:41:05] "POST /tokens HTTP/1.1" 201 -
127.0.0.1 - - [10/Jun/2019 10:41:06] "GET /test/0 HTTP/1.1" 200 -
127.0.0.1 - - [10/Jun/2019 10:41:07] "GET /test/1 HTTP/1.1" 200 -
127.0.0.1 - - [10/Jun/2019 10:41:08] "GET /test/2 HTTP/1.1" 200 -
127.0.0.1 - - [10/Jun/2019 10:41:39] "DELETE /tokens/123 HTTP/1.1" 204 -

We can clearly see that by using a custom connection plugin, we managed to reduce the number of login and logout API requests to two per playbook. Take that regular module and action plugin ;)

If we were to expand our scenario once more and wanted to test endpoints on different services, we would find out that for connection plugin this simply means adding more entries to the inventory. Clean and efficient. Regular module and action plugin, on the other hand, would need some help from the product filter in the loop statement, which is something we should avoid if possible.

Summary

If we compress our findings into a list, this is what we get:

Ansible module can ping our service from hosts other than localhost, but we pay for this privilege with packing and unpacking the module code on each task execution.
Action plugin eliminates the packaging but also removes the ability to ping our services from hosts other than localhost.
Custom connection plugin can only ping services from localhost and still does the packaging dance but can use a single connection for the whole playbook.

So, if your plugin requirements are quite modest (you only use a few different tasks that interact with the service), we would suggest that you create an action plugin or module since they are a bit easier to write and debug compared to the connection plugin. For more complex scenarios, taking some time to learn about the custom connection plugins might make it worth its while.

Oh, and if you are wondering how our custom connection plugin is doing all this funky stuff with service sessions, you might consider joining us next time when we will look at the persistent connections in greater detail. But until then, you can get hold of us on Reddit or Twitter.

Cheers!