Ansible/EC2/Dynamic Inventory - Revision history

Admin: /* Workflow */

2018-11-15T16:56:03Z

Workflow

@@ Line 174: / Line 174: @@
 * Launch one AWS instance (the Ansible mastermind instance) that has the Ansible playbook bundled up and unpacked as part of the cloud-init process
 * The cloud-init process starts up Ansible and runs the playbook automatically
 =Flags=

Admin: /* Workflow */

2018-11-15T16:54:46Z

Workflow

← Older revision		Revision as of 16:54, 15 November 2018
Line 154:		Line 154:
	The final workflow with this script looks like this:		The final workflow with this script looks like this:

	* ~~Set~~ up 1 production ASG and 1+ sandbox ASGs		Tools:

			* Ansible
			* dynamic inventory script above
			* Terraform

			Preparation:

			* Use terraform to set up 1 production ASG and 1+ sandbox ASGs
	* Use <code><nowiki>{{ target }}</nowiki></code> template in playbook so you can pass in the target ASG from the command line		* Use <code><nowiki>{{ target }}</nowiki></code> template in playbook so you can pass in the target ASG from the command line
			* Compress Ansible playbook directory and base64 encode it (http://jesseszwedko.com/post/ansible-aws-launch-configuration/)

	For local development:		For local development:

Admin: /* Alternative implementation that deals with auto scaling groups */

2018-11-15T16:51:34Z

Alternative implementation that deals with auto scaling groups

@@ Line 71: / Line 71: @@
 }}
-Here is the script:
+==Inventory script==
 <pre>
@@ Line 119: / Line 123: @@
 print(json.dumps(inventory))
 </pre>
 =Flags=

Admin: /* Alternative implementation */

2018-11-15T16:29:37Z

Alternative implementation

← Older revision		Revision as of 16:29, 15 November 2018
Line 57:		Line 57:
	* data is printed (list of nodes)		* data is printed (list of nodes)

	=Alternative implementation=		=Alternative implementation that deals with auto scaling groups=

	An alternative dynamic inventory script is provided by @jszwedko: http://jesseszwedko.com/post/ansible-autoscaling-inventory/		An alternative dynamic inventory script is provided by @jszwedko: http://jesseszwedko.com/post/ansible-autoscaling-inventory/

	This script ~~does something slightly different~~. This script ~~is used~~ to ~~create auto~~-scaling ~~groups on AWS EC2~~.		This script is not the same as the EC2 script provided by Ansible. This script deals with ASGs (Auto-Scaling Groups) and uses their Name tag to return instances. The Ansible-provided EC2 script does not deal with ASGs.

			Note: what is an ASG?

			{{Quote\|
			An Auto Scaling group contains a collection of EC2 instances that share similar characteristics and are treated as a logical grouping for the purposes of instance scaling and management.

			- https://docs.aws.amazon.com/autoscaling/ec2/userguide/AutoScalingGroup.html
			}}

			Here is the script:

	<pre>		<pre>

Admin: /* Alternative implementation */

2018-11-15T16:26:56Z

Alternative implementation

← Older revision		Revision as of 16:26, 15 November 2018
Line 81:		Line 81:
	region = os.environ.get('AWS_REGION', None)		region = os.environ.get('AWS_REGION', None)
	if region is None:		if region is None:
	print "$AWS_REGION must be set"		print("$AWS_REGION must be set")
	sys.exit(1)		sys.exit(1)

Line 90:		Line 90:

	if args.host:		if args.host:
	print "{}"		print("{}")

	if not args.list:		if not args.list:
Line 107:		Line 107:
	inventory[name]['hosts'] += instance_dns_names		inventory[name]['hosts'] += instance_dns_names

	print json.dumps(inventory)		print(json.dumps(inventory))
	</pre>		</pre>








	=Flags=		=Flags=

Admin: /* Flags */

2018-11-15T16:26:22Z

Flags

@@ Line 56: / Line 56: @@
 * cache is loaded (if present)
 * data is printed (list of nodes)
 =Flags=

Admin: /* Managing a dynamic inventory using EC2 nodes */

2018-11-15T16:24:56Z

Managing a dynamic inventory using EC2 nodes

← Older revision		Revision as of 16:24, 15 November 2018
Line 9:		Line 9:
	[[Ansible/EC2/Dynamic Inventory]] - dynamic inventory uses the AWS API to get information about machines that AWS is managing.		[[Ansible/EC2/Dynamic Inventory]] - dynamic inventory uses the AWS API to get information about machines that AWS is managing.

	=Managing a dynamic inventory using EC2 nodes=		=Managing a dynamic inventory using EC2 nodes: example script=

	~~There is a very thorough~~ EC2 ~~dynamic~~ inventory script in ~~the Ansible~~ Github repository: https://raw.githubusercontent.com/ansible/ansible/devel/contrib/inventory/ec2.py		Here we walk through the dynamic EC2 inventory script provided by Ansible in their Github repository: https://raw.githubusercontent.com/ansible/ansible/devel/contrib/inventory/ec2.py

	This script also has an .ini configuration file associated with it: https://raw.githubusercontent.com/ansible/ansible/devel/contrib/inventory/ec2.ini		This script also has an .ini configuration file associated with it: https://raw.githubusercontent.com/ansible/ansible/devel/contrib/inventory/ec2.ini

Admin at 16:40, 11 November 2018

2018-11-11T16:40:57Z

← Older revision		Revision as of 16:40, 11 November 2018
Line 60:		Line 60:

	{{AnsibleFlag}}		{{AnsibleFlag}}

			[[Category:AWS]]

Admin at 16:40, 11 November 2018

2018-11-11T16:40:24Z

← Older revision		Revision as of 16:40, 11 November 2018
Line 56:		Line 56:
	* cache is loaded (if present)		* cache is loaded (if present)
	* data is printed (list of nodes)		* data is printed (list of nodes)

			=Flags=

			{{AnsibleFlag}}

Admin: Created page with "{Main|Ansible/EC2}} This page covers how to use a dynamic inventory script to manage Ansible nodes when using Amazon AWS EC2. =Static vs dynamic inventory= Ansible/EC2/St..."

2018-11-11T16:39:54Z

Created page with "{Main|Ansible/EC2}} This page covers how to use a dynamic inventory script to manage Ansible nodes when using Amazon AWS EC2. =Static vs dynamic inventory= Ansible/EC2/St..."

New page

{Main|Ansible/EC2}}

This page covers how to use a dynamic inventory script to manage Ansible nodes when using Amazon AWS EC2.

=Static vs dynamic inventory=

[[Ansible/EC2/Static Inventory]] - static inventory requires the hosts file (containing the list of machines that Ansible is managing) be kept up to date by hand. This can be a burden if details are changing often or if resources are allocated automatically.

[[Ansible/EC2/Dynamic Inventory]] - dynamic inventory uses the AWS API to get information about machines that AWS is managing.

=Managing a dynamic inventory using EC2 nodes=

There is a very thorough EC2 dynamic inventory script in the Ansible Github repository: https://raw.githubusercontent.com/ansible/ansible/devel/contrib/inventory/ec2.py

This script also has an .ini configuration file associated with it: https://raw.githubusercontent.com/ansible/ansible/devel/contrib/inventory/ec2.ini

Let's run through an overview of how it works.

==Environment variables==

This dynamic inventory script uses lots of environment variables. The most important are:

* <code>AWS_ACCESS_KEY_ID</code> to set your AWS API access key (for boto)
* <code>AWS_SECRET_ACCESS_KEY</code> to set your AWS API access secret (for boto)
* <code>AWS_PROFILE</code> to specify a boto profile
* <code>EC2_INSTANCE_FILTERS</code> to filter the AWS instances returned on various criteria. Extremely detailed API reference for filtering is [https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstances.html#query-DescribeInstances-filters here]

Note that each of these can also be specified in the .ini file.

==Important==

By default, the ec2.ini file is configured for all Amazon cloud services. You have to turn off the ones you don't want (elasticcache, rds, etc.)

Ref: https://docs.ansible.com/ansible/latest/user_guide/intro_dynamic_inventory.html#inventory-script-example-aws-ec2

==Python script==

The script proper defines a Python object that manages all of the information received from the AWS API.

The script uses the boto library to interact with the AWS API. The object defines methods for parsing user command line arguments, and implements a number of other methods to do things like ask for a list of nodes, filter nodes by attribute, and store/retrieve information from a cache on disk to prevent everything from going extremely slowly due to slow AWS API responses.

Key methods defined for the object:

* read input file: https://github.com/ansible/ansible/blob/devel/contrib/inventory/ec2.py#L304
* parse cli: https://github.com/ansible/ansible/blob/devel/contrib/inventory/ec2.py#L523
* get instances: https://github.com/ansible/ansible/blob/devel/contrib/inventory/ec2.py#L593
* get route names for instances from route 53: https://github.com/ansible/ansible/blob/devel/contrib/inventory/ec2.py#L1468
* get a dictionary with instance information: https://github.com/ansible/ansible/blob/devel/contrib/inventory/ec2.py#L1489
* process API call returns: https://github.com/ansible/ansible/blob/devel/contrib/inventory/ec2.py#L1542

The call order of the script, when run, is:

* object is created https://github.com/ansible/ansible/blob/devel/contrib/inventory/ec2.py#L1709
* init method is called: https://github.com/ansible/ansible/blob/devel/contrib/inventory/ec2.py#L245
* credentials are set up
* cache is loaded (if present)
* data is printed (list of nodes)