Ansible/Hosts/Dynamic Inventory: Difference between revisions

From charlesreid1

(Created page with "==Example executable dynamic inventory script== Let's walk through an example. '''List of hosts:''' For vagrant machines, vagrant status gives output about which machines a...")
 
 
(16 intermediate revisions by the same user not shown)
Line 1: Line 1:
==Example executable dynamic inventory script==
==What is a dynamic inventory script==


Let's walk through an example.
In order to remotely connect to machines and run commands to configure the machines, Ansible needs information about how to connect to each remote machine.


'''List of hosts:'''
The There are two ways to provide this information:


For vagrant machines, vagrant status gives output about which machines are running. This is information we want to provide to the dynamic inventory file to create a list of hosts.
* a static inventory file, in which details are given by hand (this can be streamlined using groups and variables)
* a dynamic inventory file, in which a script provides Ansible with details about nodes using a dynamic resource (API, database, etc.)
 
==How to use dynamic inventory script==
 
The dynamic inventory script must accept to command line flags, for the two ways Ansible will call this dynamic inventory script:
 
<pre>
--host=<hostname>  show host details
--list              list groups
</pre>
 
For example, Ansible will call the inventory script like so:


<pre>
<pre>
vagrant status --machine-readable
$ ./dynamic.py --host=vagrant2
</pre>
</pre>


which is a lot of info in CSV format that can be parsed.


Next, we can use the <code>vagrant ssh-config</code> command, which outputs information that is formatted for an SSH config file, and parse it using pamiko (a Python library for doing SSH-related things - in this case, parsing an SSH config file and turning it into a python dictionary).


<pre>
==Example: Vagrant Dynamic Inventory Script==
#!/usr/bin/env python
 
# Adapted from Mark Mandel's implementation
See [[Ansible/Vagrant/Dynamic Inventory]] for a sample dynamic inventory script for use with Vagrant.
# https://github.com/ansible/ansible/blob/stable-2.1/contrib/inventory/vagrant.py
 
# License: GNU General Public License, Version 3 <http://www.gnu.org/licenses/>
==Example: EC2 Dynamic Inventory Script==
import argparse
 
import json
There is a very thorough EC2 dynamic inventory script in the Ansible Github repository: https://raw.githubusercontent.com/ansible/ansible/devel/contrib/inventory/ec2.py
import paramiko
 
import subprocess
This script also has an .ini configuration file associated with it: https://raw.githubusercontent.com/ansible/ansible/devel/contrib/inventory/ec2.ini
import sys
 
Let's run through an overview of how it works.
 
===Environment variables===
 
This dynamic inventory script uses lots of environment variables. The most important are:
 
* <code>AWS_ACCESS_KEY_ID</code> to set your AWS API access key (for boto)
* <code>AWS_SECRET_ACCESS_KEY</code> to set your AWS API access secret (for boto)
* <code>AWS_PROFILE</code> to specify a boto profile
* <code>EC2_INSTANCE_FILTERS</code> to filter the AWS instances returned on various criteria. Extremely detailed API reference for filtering is [https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstances.html#query-DescribeInstances-filters here]
 
Note that each of these can also be specified in the .ini file.
 
===Important===


By default, the ec2.ini file is configured for all Amazon cloud services. You have to turn off the ones you don't want (elasticcache, rds, etc.)


def parse_args():
Ref: https://docs.ansible.com/ansible/latest/user_guide/intro_dynamic_inventory.html#inventory-script-example-aws-ec2
    parser = argparse.ArgumentParser(description="Vagrant inventory script")
    group = parser.add_mutually_exclusive_group(required=True)
    group.add_argument('--list', action='store_true')
    group.add_argument('--host')
    return parser.parse_args()


===Python script===


def list_running_hosts():
The script proper defines a Python object that manages all of the information received from the AWS API.
    cmd = "vagrant status --machine-readable"
    status = subprocess.check_output(cmd.split()).rstrip()
    hosts = []
    for line in status.split('\n'):
        (_, host, key, value) = line.split(',')[:4]
        if key == 'state' and value == 'running':
            hosts.append(host)
    return hosts


The script uses the boto library to interact with the AWS API. The object defines methods for parsing user command line arguments, and implements a number of other methods to do things like ask for a list of nodes, filter nodes by attribute, and store/retrieve information from a cache on disk to prevent everything from going extremely slowly due to slow AWS API responses.


def get_host_details(host):
Key methods defined for the object:
    cmd = "vagrant ssh-config {}".format(host)
    p = subprocess.Popen(cmd.split(), stdout=subprocess.PIPE)
    config = paramiko.SSHConfig()
    config.parse(p.stdout)
    c = config.lookup(host)
    return {'ansible_host': c['hostname'],
            'ansible_port': c['port'],
            'ansible_user': c['user'],
            'ansible_private_key_file': c['identityfile'][0]}


* read input file: https://github.com/ansible/ansible/blob/devel/contrib/inventory/ec2.py#L304
* parse cli: https://github.com/ansible/ansible/blob/devel/contrib/inventory/ec2.py#L523
* get instances: https://github.com/ansible/ansible/blob/devel/contrib/inventory/ec2.py#L593
* get route names for instances from route 53: https://github.com/ansible/ansible/blob/devel/contrib/inventory/ec2.py#L1468
* get a dictionary with instance information: https://github.com/ansible/ansible/blob/devel/contrib/inventory/ec2.py#L1489
* process API call returns: https://github.com/ansible/ansible/blob/devel/contrib/inventory/ec2.py#L1542


def main():
The call order of the script, when run, is:
    args = parse_args()
    if args.list:
        hosts = list_running_hosts()
        json.dump({'vagrant': hosts}, sys.stdout)
    else:
        details = get_host_details(args.host)
        json.dump(details, sys.stdout)


if __name__ == '__main__':
* object is created https://github.com/ansible/ansible/blob/devel/contrib/inventory/ec2.py#L1709
    main()
* init method is called: https://github.com/ansible/ansible/blob/devel/contrib/inventory/ec2.py#L245
</pre>
* credentials are set up
* cache is loaded (if present)
* data is printed (list of nodes)


=Flags=


==Flags==
[[Category:Python]]


[[Category:Ansible]]
[[Category:Ansible]]
[[Category:Ansible Hosts]]
[[Category:Ansible Hosts]]
[[Category:Infrastructure]]
[[Category:Infrastructure]]
[[Category:Dynamic Inventory]]
[[Category:AWS]]
[[Category:Vagrant]]

Latest revision as of 01:44, 11 November 2018

What is a dynamic inventory script

In order to remotely connect to machines and run commands to configure the machines, Ansible needs information about how to connect to each remote machine.

The There are two ways to provide this information:

  • a static inventory file, in which details are given by hand (this can be streamlined using groups and variables)
  • a dynamic inventory file, in which a script provides Ansible with details about nodes using a dynamic resource (API, database, etc.)

How to use dynamic inventory script

The dynamic inventory script must accept to command line flags, for the two ways Ansible will call this dynamic inventory script: 

--host=<hostname>   show host details
--list              list groups

For example, Ansible will call the inventory script like so: 

$ ./dynamic.py --host=vagrant2


Example: Vagrant Dynamic Inventory Script

See Ansible/Vagrant/Dynamic Inventory for a sample dynamic inventory script for use with Vagrant.

Example: EC2 Dynamic Inventory Script

There is a very thorough EC2 dynamic inventory script in the Ansible Github repository: https://raw.githubusercontent.com/ansible/ansible/devel/contrib/inventory/ec2.py

This script also has an .ini configuration file associated with it: https://raw.githubusercontent.com/ansible/ansible/devel/contrib/inventory/ec2.ini

Let's run through an overview of how it works.

Environment variables

This dynamic inventory script uses lots of environment variables. The most important are:

  • AWS_ACCESS_KEY_ID to set your AWS API access key (for boto)
  • AWS_SECRET_ACCESS_KEY to set your AWS API access secret (for boto)
  • AWS_PROFILE to specify a boto profile
  • EC2_INSTANCE_FILTERS to filter the AWS instances returned on various criteria. Extremely detailed API reference for filtering is here

Note that each of these can also be specified in the .ini file.

Important

By default, the ec2.ini file is configured for all Amazon cloud services. You have to turn off the ones you don't want (elasticcache, rds, etc.)

Ref: https://docs.ansible.com/ansible/latest/user_guide/intro_dynamic_inventory.html#inventory-script-example-aws-ec2

Python script

The script proper defines a Python object that manages all of the information received from the AWS API.

The script uses the boto library to interact with the AWS API. The object defines methods for parsing user command line arguments, and implements a number of other methods to do things like ask for a list of nodes, filter nodes by attribute, and store/retrieve information from a cache on disk to prevent everything from going extremely slowly due to slow AWS API responses.

Key methods defined for the object:

The call order of the script, when run, is:

Flags

Retrieved from "https://charlesreid1.com/w/index.php?title=Ansible/Hosts/Dynamic_Inventory&oldid=25363"