Securing your Ansible deployment on AWS

Lately I have been involved in a team which is developing software as micro-services. It is very interesting work and has helped me to get introduced to some interesting technologies such as AWS Ansible and Docker in depth (and in practice).

Ansible is a great deployment automation tool. It is made in python, is declarative and agent-less (i.e. it just needs SSH access to your box). SSH it self is very secure if you use key based authentication. But people tend to use it in very insecure manner (i.e they share SSH keys on email, don't delete old SSH keys once they are revoked etc).

In my opinion the best solution to stop sharing of private keys is to generate them on a hardware token from which they cannot be (easily) copied. Fortunately I have access to Yubico Neo on which I managed to generate a public/private gpg key. The private key resides in your gpg card and public key can be put in your Ansible target hosts. There are quite a few guides (Mac, Linux, Windows) for Yubikey and few more for general purpose CCID based key-stores. And then one can use SSH key forwarding with your Gpg agent so that you can use your Yubikey for servers which require you to first login to jumpboxes.

Ansible has good AWS support in form of modules such as ec2, rds etc (backed by python boto). These modules of-course require AWS api access when trying to conjure/terminate nodes or updating a route53 DNS entry. Normally this means you have to generate and use AWS API tokens. I prefer using jump-boxes with IAM roles instead (IAM roles essentially gives permission to an Amazon instance to call Amazon APIs on your behalf, without requiring any extra keys or passwords etc). 

So this entire approach will be useful only if you can easily login to a jumpbox with IAM role. I like the using OpsWorks to create and start such jumpbox instances. The only thing required for such jumpboxes is to have correct IAMroles (which can do all the things that you need to do from your Ansible scripts, i.e EC2.* or S3.*) and your IAM users have the permission to SSH to the jumpbox. IAM users can register their own SSH keys (of-course the one generated on Yubikey) on OpsWorks. And they keys get magically updated on all machines managed by Opsworks.

Following this guide you make it impossible to leak your SSH key or Amazon credentials, since you don't put them in any remote machine/code. And that should be a good baseline for a secure Ansible deployment on AWS.