Automate VM setup with Ansible

This post will cover setting up Ansible, I'll note a couple of different ways to run Ansible and SSH
Important note, please only use Ansible playbooks if you are confident in what you are doing.
I highly encourage setting up a couple of VMs to use to test running playbooks on.

If you are already confident with SSH, you can skip to the Ansible focused section

You will need a separate system to install and run Ansible on, I use a VM that is dedicated to Ansible and is always on, so if I schedule playbooks to run Automatically they will. Using your laptop or Desktop is also fine and these can be macOS, Linux or WSL

Once you have decided where Ansible will be installed, the next decision needs to be how to handle the SSH connection to the machines you're going to use Ansible on. I use key pairs that are loaded into my VMs via the cloud init loader on my VMs created from my templates, but I do have devices with their own key pair and these I created and pass in through the command I run, I do avoid this method if I can just send the key to the device, but It's not always safe to do so.
You can also just pass the user and password into the command when you run a playbook the same as if you were SSHing to a machine using password based authentication (not my preference).
If you are doing this on VMs that have all been created with the cloud init method it won't be necessary but if you do need or want separate SSH keys run the below command on the VM/Machine you plan on using to administer your Ansible Playbooks

sudo ssh-keygen -t ed25519 -C "ansible"

If you prefer using rsa encryption, use the below, and edit 2048 to what ever bit size you like.
sudo ssh-keygen -b 2048 -t rsa -C "ansible"

when you run either of the above commands you will be prompted with where to save the keys, you should keep them in the default location, but we need to name the files so use /home/username/.ssh/ansible (Change username to match yours) this will name the key pair ansible and allows them to be used separate to your other keys

Now if you are going to use key pairs as I do for when you SSH to a machine you will need to send the public key to the machines you intend of administering with Ansible
Run the below to copy the ansible public key created above to your remote machines

ssh-copy-id -i /home/username/.ssh/ansible [email protected]
Make sure to change username in the file path to your current user and the remoteusername to a user on the remote machine with SSH privileges, also the IP address should match your remote machine's IP
You will be prompted for the remote users password
#note - For macOS replace home with Users in the path

Now you can test if the key is successfully store on the remote system by running
ssh -i /home/username/.ssh/ansible [email protected]
The reason we are passing in a file using the -i flag is due to use creating the SSH key ansible which is not the standard id_ed25519 if you are just using your systems generated rsa or ed25519 key then you won't need to pass through the ansible key as it will just log in, same applies if you are using the cloud init VM template method as these machines with already have matching key pairs
#note - For macOS replace home with Users in the path

You should now be logged in without needing a password, and you can exit the SSH session and continue.

If you don't want to use key pairs and prefer password based authentication with SSH then run sudo apt update then sudo apt install sshpass this will allow us to run the ansible playbooks by passing the --ask-pass parameter into the command.
#note - For macOS run brew install esolitos/ipa/sshpass

Now it's time to Install Ansible

to install Ansible, it will exist in most distros as a package and the below will work run sudo apt update then sudo apt install ansible -y
#note for macOS run brew install ansible

Now if you are on an older distro like Ubuntu LTS 22.04 you may want to add the Ansible repo manually to ensure you get the latest version
sudo add-apt-repository ppa:ansible/ansible
then run sudo apt update then sudo apt install ansible -y
macOS will be fine using brew to get the latest version
You can always verify the Ansible version that was installed by running ansible --version and checking the version against the Ansible Releases page

Checkout TechnoTim's guide on installing the latest version using Pip if that is something you would prefer, it's based on Debian when looking at the removal but is python from then on

Now we have Ansible installed, let's create a list of our remote machines this is called an inventory file it tells ansible what the remote machine ip address is and what group it belongs to. Ansible playbooks can be run on all machines in the inventory file or just a particular group, and example would be if you are running macOS or BSD based playbooks you don't want them running on Linux machines, or you may have Dev/Prod machines so you may only want to deploy to dev machines before rolling out to Prod

I like to create ansible folder in my home directory to keep everything neat, but you're more than welcome to create all files in the base of your home directory
To ensure you're in the home directory, run cd ~/ now to create a folder run sudo mkdir ansible as we are creating the inventory file, let's create a folder for that too
cd ansible sudo mkdir inventory
Now we will create the inventory file, I use vs-code mostly by nano works fine for me too
sudo nano inventory you can change inventory to what ever name you like, could be hosts, servers or systems doesn't matter as long as it means something to you

Now we need to list the remote machines we will be administering with ansible
this is an ini format config file, so that is the format this needs to be in.
For me, I will list a couple of test machines, one macOS and one Ubuntu 24.04.
You can use DNS name for the machines as long as they can be resolved

[macOS]
192.168.1.123

[ubuntu]
192.168.1.124

If using nano control x then y to confirm, otherwise use the save method of your editor

Now the inventory file is created, let's test the connection to the remote machines before moving on to creating a playbook
run
ansible --key-file /home/username/.ssh/ansible -i /home/username/ansible/inventory/inventory groupname -m ping --user remoteuser
change username to your user, change groupname to a group in the inventory file or to all to run against all listed machines, change remoteuser to the user being used for the SSH connection
If successful you should see an output like the below
#note - For macOS replace home with Users in the path

192.168.1.123 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    },
    "changed": false,
    "ping": "pong"
}
192.168.1.124 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    },
    "changed": false,
    "ping": "pong"
}

A simple playbook I like to use on all my VMs is for installing the qemu-guest-agent, I run, this following the provisioning of any new VMs
Playbooks are in yaml format
I like to store playbooks in a folder called playbooks within the ansible folder
sudo mkdir /home/username/ansible/playbooks
change into the directory
cd /home/username/ansible/playbooks
Now we create the playbook
sudo nano qemu-guest-agent.yml
#note - For macOS replace home with Users in the path

- name: Install and start qemu-guest-agent
  hosts: all
  tasks:
    - name: Install qemu-guest-agent
      apt:
        name: qemu-guest-agent
        state: present
        update_cache: true
      become: true

    - name: Ensure qemu-guest-agent is started and enabled
      systemd:
        name: qemu-guest-agent
        state: started
        enabled: yes
      become: true

The yaml is a basic one but the jist is
name - what you want the playbook to be called, generally call it what it does to keep it simple
hosts - all means all hosts in the inventory file will have the task run on it, if you want the playbook to only apply to certain machine groups that change it suit
tasks - is all the tasks that the playbook will run, in this case it is only one task
name - name of task running in the playbook, best to be descriptive with this so you can identify which task failed when troubleshooting
apt - is the Ansible module being used
state - checks the final state has been achieved, in this case has the qemu-guest-agent been installed
update_cache - tells Ansible whether or not to run the apt update
become - whether or not to run commands as sudo
name - next tasks name more descriptive
systemd - app to run a command with
name - name of task basic
state - checks the final state has been achieved, in this case has the qemu-guest-agent service started
enabled - tells ansible we are wanting the qemu-guest-agent service started
become - run the command on the remote machine with sudo

As I only want to run this on the Ubuntu machine in my inventory file, I would change the hosts in the playbook to hosts: ubuntu
run the playbook with the below
ansible-playbook --key-file /home/username/.ssh/ansible -i /home/username/ansible/inventory/inventory /home/username/ansible/playbooks/qemu-guest-agent.yml --user remoteuser
replace username with your user, replace remoteuser with the SSH user to connect with
#note - For macOS replace home with Users in the path

The output will look like the below if successful

PLAY [ubuntu] *********************************************************************

TASK [Gathering Facts] *********************************************************
ok: [192.168.1.123]

TASK [install qemu-guest-agent] *********************************************
changed: [192.168.1.123]

PLAY RECAP *********************************************************************
192.168.1.12               : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Now that a basic playbook has been made, use these resources to understand the power of Ansible and what can be done with a playbook
https://docs.ansible.com/ansible/latest/collections/ansible/builtin/file_module.html#parameters

Make Ansible commands shorter

To avoid needing to key so much into the CLI every time you want to run a playbook you can edit /etc/ansible/ansible.cfg to set your inventory file location and ssh key to use
change these line and ensure the # is removed from the beginning of the line
inventory = /home/username/ansible/inventory/inventory
private_key_file = /home/username/.ssh/ansible
by doing this, I now can cd into the playbooks folder and simple run
ansible-playbook qemu.guest.agent.yml --user remoteuser
#note - For macOS replace home with Users in the path
On macOS Ansible config is kept in /private/etc/ansible/ansible.cfg

Make Ansible Commands even shorter

not recommended due to increasing your secuirty risk

This can be simplified even more if the local user executing the commands exists on the remote pc
Create a user called ansible
sudo adduser ansible
enter a password and press enter for the other fields asked to complete
sudo usermod -aG sudo ansible
Logout of the current user and log in as ansible
You can run these commands on all the machines being administered by Ansible manually or using a playbook to create the user
For Ansible to be able to create a user, it needs the user's password to be hashed
Run the below command to generate a hashed password for the playbook replace password with what ever you want the password to be
openssl passwd -1 password
Create a playbook in the playbooks folder with nano create.ansible.user.ssh.key.yml
copy the below into the yaml then ctrl+x and enter to save

---
- name: Create ansible user and add SSH key
  hosts: ubuntu
  become: true

  tasks:
    - name: Ensure ansible user is present
      user:
        name: ansible
        password: "Replace hashed password from previous command"  # change hash to the one output from before
        groups: sudo
        state: present
        create_home: yes

    - name: Install public keys
      ansible.posix.authorized_key:
        user: ansible  # Change to the created user 'ansible'
        state: present
        key: "{{ lookup('file', '~/.ssh/ansible.pub') }}"  # Ensure the public key file exists

    - name: Change sudoers file
      ansible.builtin.lineinfile:
        path: /etc/sudoers
        state: present
        regexp: '^%sudo'
        line: '%sudo ALL=(ALL) NOPASSWD: ALL'
        validate: /usr/sbin/visudo -cf %s

Replace hashed password from previous command with the output of the openssl passwd -1 ansible run previously

Run the below to execute the playbook and add the user to each system, and it will be part of the sudo group
ansible-playbook create.ansible.user.ssh.key.yml --user remoteuser --ask-become-pass
Now if you log in as ansible copy the inventory and playbooks from the previous user into the same structure and be able to run all playbooks with just
ansible-playbook playbooknamehere.yml
No user, password or become password needed anymore in the command line when running playbooks.

The user doesn't need to be ansible it can be what ever user you like even the current user if you set the VM up to just be the Ansible server.

Using SSH with keys is the better option than using just a password.
The above method to remove the need to use a password when running the playbooks is great for convenience, but if the Ansible server gets compromised, then every machine with the key applied will be able to be sshed into without a password.
Use two factor with SSH for extra security
This link has the instructions on how to set it up with Google authenticator app

Additional Tips
to select just one group from the inventory file use
ansible-playbook playbook.yml -l 'group'
or for multiple groups from the inventory file
ansible-playbook playbook.yml -l 'group1','group2