Ansible/Hosts/Dynamic Inventory

What is a dynamic inventory script

In order to remotely connect to machines and run commands to configure the machines, Ansible needs information about how to connect to each remote machine.

The There are two ways to provide this information:

• a static inventory file, in which details are given by hand (this can be streamlined using groups and variables)
• a dynamic inventory file, in which a script provides Ansible with details about nodes using a dynamic resource (API, database, etc.)

How to use dynamic inventory script

The dynamic inventory script must accept to command line flags, for the two ways Ansible will call this dynamic inventory script:

--host=<hostname>   show host details
--list              list groups

For example, Ansible will call the inventory script like so:

$ ./dynamic.py --host=vagrant2

Example executable dynamic inventory script

Let's walk through an example script with Vagrant.

This example script is written in Python. It takes two input arguments, --host=X and --list, and interacts with Vagrant to return the requested information.

Vagrant

Getting List of Hosts

Start with how to get a list of hosts for the --list flag.

Vagrant provides a command to show info about all running vagrant machines:

vagrant status

This info can be used to create the list of hosts for the dynamic inventory file. A more convenient format for parsing is:

vagrant status --machine-readable

which prints everything in CSV format. The code:

def list_running_hosts():
    cmd = "vagrant status --machine-readable"
    status = subprocess.check_output(cmd.split()).rstrip().decode('utf-8')
    hosts = []
    for line in status.split('\n'):
        (_, host, key, value) = line.split(',')[:4]
        if key == 'state' and value == 'running':
            hosts.append(host)
    return hosts

Getting Machine Configuration Details

Now we need to get host-specific details for the --host=X flag.

Next we can use the vagrant ssh-config command to get info about the machine-specific configuration details of each machine, which is necessary for Ansible to connect to them. To parse SSH config files, we can use a Python library called pamiko.

def get_host_details(host):
    cmd = "vagrant ssh-config {}".format(host)
    p = subprocess.Popen(cmd.split(), stdout=subprocess.PIPE)
    config = paramiko.SSHConfig()
    config.parse(p.stdout)
    c = config.lookup(host)
    return {'ansible_host': c['hostname'],
            'ansible_port': c['port'],
            'ansible_user': c['user'],
            'ansible_private_key_file': c['identityfile'][0]}

Parsing user arguments

A quick argparser tutorial for the flags we're interested in parsing:

def parse_args():
    parser = argparse.ArgumentParser(description="Vagrant inventory script")
    group = parser.add_mutually_exclusive_group(required=True)
    group.add_argument('--list', action='store_true')
    group.add_argument('--host')
    return parser.parse_args()

Now we can call this function and use args.list or args.host to access the values of the flags.

Putting it all together

Here is the final dynamic inventory script:

dynamic_vagrant.py:

#!/usr/bin/env python
# Adapted from Mark Mandel's implementation
# https://github.com/ansible/ansible/blob/stable-2.1/contrib/inventory/vagrant.py
# License: GNU General Public License, Version 3 <http://www.gnu.org/licenses/>
import argparse
import json
import paramiko
import subprocess
import sys


def parse_args():
    parser = argparse.ArgumentParser(description="Vagrant inventory script")
    group = parser.add_mutually_exclusive_group(required=True)
    group.add_argument('--list', action='store_true')
    group.add_argument('--host')
    return parser.parse_args()


def list_running_hosts():
    cmd = "vagrant status --machine-readable"
    status = subprocess.check_output(cmd.split()).rstrip().decode('utf-8')
    hosts = []
    for line in status.split('\n'):
        (_, host, key, value) = line.split(',')[:4]
        if key == 'state' and value == 'running':
            hosts.append(host)
    return hosts


def get_host_details(host):
    cmd = "vagrant ssh-config {}".format(host)
    p = subprocess.Popen(cmd.split(), stdout=subprocess.PIPE)
    config = paramiko.SSHConfig()
    config.parse(p.stdout)
    c = config.lookup(host)
    return {'ansible_host': c['hostname'],
            'ansible_port': c['port'],
            'ansible_user': c['user'],
            'ansible_private_key_file': c['identityfile'][0]}


def main():
    args = parse_args()
    if args.list:
        hosts = list_running_hosts()
        json.dump({'vagrant': hosts}, sys.stdout)
    else:
        details = get_host_details(args.host)
        json.dump(details, sys.stdout)

if __name__ == '__main__':
    main()

Running the dynamic inventory script

Once we've written the dynamic inventory script and we're ready to run Ansible using it, we can pass the inventory script using the -i flag:

ansible -i dynamic_vagrant.py -u ubuntu myvagrantbox -m ping

EC2

There is a very thorough EC2 dynamic inventory script in the Ansible Github repository: https://raw.githubusercontent.com/ansible/ansible/devel/contrib/inventory/ec2.py

This script also has an .ini configuration file associated with it: https://raw.githubusercontent.com/ansible/ansible/devel/contrib/inventory/ec2.ini