My First 5 Minutes on a Server (with Ansible)

So today, Bryan Kennedy wrote this nice post about how to set up some basic security on a fresh Ubuntu Lucid box.

Cut to Hacker News, with this extremely helpful comment:

The premise of this thing is not good advice.

1) Your first couple minutes on a server should be used to install a configuration management client, if your bootstrap policies somehow don’t already install one.

This stuff isn’t hard. It’s worth doing right.

Nothing like the express lane from “here’s some practical advice that you can use to secure your server today” and “NOPE, you need to be a big-shot sysadmin to perform basic configuration on the box!”

This helpful statement garnered this reply:

This stuff isn’t hard. It’s worth doing right.

Can you provide an article as equally succinct as the OP’s that provides this information? Your list is painfully devoid of anything of true value. Since it’s not hard, and worth doing right, I imagine something should already be written.

Let’s see if we can find some middle ground.

Introducing Ansible

I LOVE Ansible. It’s a nice little tool that falls somewhere on the spectrum between “Do everything manually” and “Learn a ridiculous configuration language and beat your head against the Puppet manual for three hours in order to make a small change.”

(I don’t think I’m a stupid person, but DAMN do I feel dumb when I’m working with Puppet configs. And before you ask, I worked with Puppet for a year and found it consistently painful.)

Key differences between Ansible and just about anything else:

  • Ansible is small, just a couple thousand lines of well-tested code.
  • There is no central server.
  • There is no software to install on managed servers.
  • There is no configuration database.
  • Configurations are text files that can be read by humans.
  • Ansible connects over SSH and uploads small Python scripts to do its work.

NOT included

I’m not going explain how to install Ansible on your workstation. Sorry, that’s a topic for another post. There’s decent documentation over at the Ansible site.

I’m not going to discuss how to write an Ansible playbook. Again, see documentation and maybe my pedantic playbook example.

I’m not going to talk in depth about my personal security preferences versus the original author’s. My purpose is simply to show that it’s equally easy (and possibly better) to automate server configurations.

(On that note, I’m going to skip firewalling for now, because it’s trivial to add, and because it’s a broader topic. fail2ban is a great start on locking down the server.)

So, without further ado…

First 5 minutes: “use configuration management” edition

Let’s restate the original post’s requirements:

I like to have a single deploy user for people to log in with. This user has a complex password stored somewhere safe. Regular users log in using their SSH keys. No one is allowed to SSH in as root, they can only use sudo as the deploy user.

I want to use fail2ban to stop SSH scanning robots from trying users and passwords all the livelong day.

I also like to automatically install security updates so I don’t get hit unawares by a new exploit, and so that I don’t have to think about updates often.

I like to have logwatch email me every night with a summary of the day’s logs.”

As a bullet list

Here’s what I think you should do to a new server:

  • Set the root password.
  • Create the deploy user.
  • Lock down SSH using fail2ban, disable root access & password authentication.
  • Add my team’s SSH keys to the deploy user.
  • Set up automatic security updates.
  • (Lock down the firewall)
  • Set up logwatch to run nightly (and also configure outbound mail).”

The first thing we have to do manually, but everything else can be easily done with an Ansible playbook.

A playbook?

Think of “playbook” as a shell script that does server deployment. “Do this, then do that. If we changed the ssh config file, then restart the ssh service.”

Ansible uses YAML for its playbooks, which means you can read and edit them in any text editor, even Notepad.

The result end is that the configurations are readable by mere mortals, in fact, it looks very much like the commands the original article prescribed.

Let’s proceed.

Step 1: Set the root password

Run:

yourmachine$ ssh root@server

Enter the initial root password from your hosting provider, then run:

root@server# passwd

Step 2: Fetch the bootstrap recipe.

https://github.com/phred/5minbootstrap/

yourmachine ~$ git clone https://github.com/phred/5minbootstrap.git
yourmachine ~$ cd 5minbootstrap

Step 3: Edit hosts.ini

Ansible needs to know about the servers you want to manage. There is no fancy central database, just a text file with a list of servers. Oh, it’s called an “inventory file.”

Edit the hosts.ini that came with the repository. Replace 127.0.0.1 with your IP address, and :2222 with your SSH port (or leave it off if it’s port 22).

[newservers]
127.0.0.1:2222

For convenience I made a newservers server group. The idea is that when I get a new server, I put it in that group temporarily and run the bootstrap.yml playbook.

Step 4: Update the SSH public key.

yourmachine ~/5minbootstrap$ cp ~/.ssh/id_dsa.pub ./fred.pub

For simplicity I provided my public key in the repo. Unless you want to grant me login access to your server, you probably want to change that. :-)

Step 5: Run the playbook.

This is the needed invocation for Vagrant:

yourmachine ~/5minbootstrap$ ansible-playbook -i hosts.ini bootstrap.yml --ask-pass --sudo

Correction 6 Mar 2013: If you are logging into a fresh Linode, or another sytem where you only have the root user, you need to run this command:

yourmachine ~/5minbootstrap$ ansible-playbook -i hosts.ini bootstrap.yml --user root --ask-pass

I have updated the 5minbootstrap repo with a couple small changes to make that work.

Step 6: Go get a cup of coffee.

You’re DONE. I prefer hand-ground French pressed coffee myself. Tea is also fine.

What?!? are you lying?

A little. Ansible takes a little bit of work to get going locally. But it takes ZERO server-side configuration.

It did take me some time to debug this playbook, about an hour. If it takes “5 minutes” to do the original set up steps, and about 2 minutes to do these… I break even on time investment after ~20 servers.

But when I consider that running commands by hand on 20 servers, the fat fingered mistakes I’d likely make, and that in actuality it might take much longer than 5 minutes to do those simple tasks, it seems worthwhile.

Checklists have been studied in hospitals and they are proven to reduce errors and improve surgical outcomes.

A playbook is a checklist that you can EXECUTE. Even better.

Zealotry

Why aren’t you using (XYZ configuration management tool) for EVERYTHING!!! ZOMG you’re doing it wrong.”

Let me take you aside and say for a minute that doing things by hand IS FINE. Especially if you’re not an expert sysadmin, and you’re only managing N servers, where N is approximately 2.

I’ve found that when N > 2 the task gets miserable, and the amount of work needed multiplies with each server.

Before I did configuration management, I kept meticulous notes (a topic for another post). I had a file called /root/JOURNAL.textile on each server and appended an entry with a date every time that I did something significant. Simple, stupid, reliable and repeatable. Manual management + meticulous notetaking: a fine solution.

After configuration management, I still keep meticulous notes, but now my knowledge is also captured and “re-playable” in an Ansible playbook.

TL;DR: Ansible manages configs, they’re human readable, and (mostly) one file.

Here’s the 64-line playbook that does all of this:

https://github.com/phred/5minbootstrap/blob/master/bootstrap.yml

I think that it’s human readable. Disagree? Hit me up on Twitter.

Tagged as: ansible bootstrap basics configuration management

Hey — thanks for reading!

My name is Fred, and I'm a web developer by trade, Linux sysadmin by necessity. I want you to win at hosting your own web applications.

Server administration doesn't have to come with a side of stomach ulcer.

As a developer you've got most of the skills you need, all you need are some practical ways to up your server game.

Questions? Email me.