How to use Ansible hosts file with AWS

The hosts inventory file is treated as static when managing our own infrastructure, but with AWS this information will become out of date quickly. Ansible can get information via the AWS API, but has to know to do that.

Ansible can be put in dynamic inventory mode by making the inventory file (playbooks/hosts executable):

chmod +x playbooks/hosts

The script must accept two command line flags:

--host=<hostname>   show host details
--list              list groups

For example, Ansible will call the inventory script like so:

$ ./dynamic.py --host=vagrant2

Example executable dynamic inventory script

Let's walk through an example.

List of hosts:

For vagrant machines, vagrant status gives output about which machines are running. This is information we want to provide to the dynamic inventory file to create a list of hosts.

vagrant status --machine-readable

which is a lot of info in CSV format that can be parsed.

Next, we can use the vagrant ssh-config command, which outputs information that is formatted for an SSH config file, and parse it using pamiko (a Python library for doing SSH-related things - in this case, parsing an SSH config file and turning it into a python dictionary).

#!/usr/bin/env python
# Adapted from Mark Mandel's implementation
# https://github.com/ansible/ansible/blob/stable-2.1/contrib/inventory/vagrant.py
# License: GNU General Public License, Version 3 <http://www.gnu.org/licenses/>
import argparse
import json
import paramiko
import subprocess
import sys


def parse_args():
    parser = argparse.ArgumentParser(description="Vagrant inventory script")
    group = parser.add_mutually_exclusive_group(required=True)
    group.add_argument('--list', action='store_true')
    group.add_argument('--host')
    return parser.parse_args()


def list_running_hosts():
    cmd = "vagrant status --machine-readable"
    status = subprocess.check_output(cmd.split()).rstrip()
    hosts = []
    for line in status.split('\n'):
        (_, host, key, value) = line.split(',')[:4]
        if key == 'state' and value == 'running':
            hosts.append(host)
    return hosts


def get_host_details(host):
    cmd = "vagrant ssh-config {}".format(host)
    p = subprocess.Popen(cmd.split(), stdout=subprocess.PIPE)
    config = paramiko.SSHConfig()
    config.parse(p.stdout)
    c = config.lookup(host)
    return {'ansible_host': c['hostname'],
            'ansible_port': c['port'],
            'ansible_user': c['user'],
            'ansible_private_key_file': c['identityfile'][0]}


def main():
    args = parse_args()
    if args.list:
        hosts = list_running_hosts()
        json.dump({'vagrant': hosts}, sys.stdout)
    else:
        details = get_host_details(args.host)
        json.dump(details, sys.stdout)

if __name__ == '__main__':
    main()

How to manage static and dynamic inventory

To have a regular static inventory file and a dynamic inventory script, or any combination of the above, put them all in a directory, and tell Ansible to use this directory for inventory in the Ansible configuration file or on the command line.

If our directory structure is:

playbooks/inventory/hosts
playbooks/inventory/vagrant.py

we would have an ansible.cfg with the contents:

[defaults]
inventory = inventory

How to add and configure hosts in a playbook

It is important to note that the dynamic inventory script is executed at the beginning of a playbook.

If a playbook creates new hosts, the dynamic inventory script will not pick up the new hosts.

Therefore, need to add tasks using the add_host module.

add_host name=hostname groups=web,staging myvar=myval

Example playbook adding a host

Here is an example playbook that uses the add_host command:

- name: Provision a vagrant machine
  hosts: localhost
  vars:
    box: xenial64
  tasks:
    - name: create a Vagrantfile
      command: vagrant init {{ box }} creates=Vagrantfile

    - name: Bring up a vagrant machine
      command: vagrant up

    - name: add the vagrant machine to the inventory
      add_host: >
            name=vagrant
            ansible_host=127.0.0.1
            ansible_port=2222
            ansible_user=vagrant
            ansible_private_key_file=/home/username/.vagrant.d/
            insecure_private_key

- name: Do something to the vagrant machine
  hosts: vagrant
  become: yes
  tasks:
    # The list of tasks would go here
    - ...

NOTE: This is a good pattern to use.

Play number 1 runs against localhost - it configures and provisions the hosts.

Play number 2 configures the hosts.

Because we use the creates=Vagrantfile, it tells Ansible to only run the first play once (or, if our Vagrant boxes have been destroyed).

Flags

Ansible/Hosts/AWS

From charlesreid1

Contents