How to secure the ansible key

When we use RedHat Ansible Automation Platform (RHAAP) or AWX, we need an ansible private
key on all systems managed by RHAAP or AWX. If poorly configured we can potentially
risk to give root permissions to any user with access to RHAAP or AWX.

With the ansible private key there is a lot of power available to the potential hacker/user. In many organizations it gives access to systems and even more to
the root privileges.

In this document we describe the risks, and the mittigation of those risks, using
ansible and RHAAP.

The risk

As most system admins know, with a private_ssh_key exposed, we have a serious problem.
Even more, when this key gives accces to all hosts and passwordless sudo.
This would compromise the entire production environment. In my experience there are many
companies at risk.

The documentation of RHAAP tells us the key can't be decrypted from the controller or a
playbook running, that is correct. But the key is also present on the system being
targetted by a playbook. And if I can run playbook as the ansible user (wich is mostly the
case), I can create a playbook to:

  • copy the private_keyfile to /tmp
  • make the file readable to the world
  • cat the file

This way I get the content off the private key file in the output of my playbook to copy
anyware. If I copy this key to a linux host I have access to, I can configure ssh so, that
this key will be used to connect to other linux hosts as the "ANSIBLE" user with root access.
without ever being asked for a password.....

The Fix

To add several layers of security, follow the steps below, ensuring your ansible setup keeps working.

Add a complex passphrase to the ansible private key

As the ansible keypair is often only created once (and never replaced), this should be done right
the first time. Using a complex passphrase wil enhance security greatly. RHAAP can use this out of
the box, adding the passphrase to the configuration of the ansible credential.
In configuration as code this is done by the ssh_key_unlock option (see the example below for more info).

  • ssh_key_unlock

When we now extract the private key with an ansible playbook, we still get the key, but we won't
get the passphrase to unlock the key. Using the key has become virtually impossible, even when we
could brute force the passphrase.

Ensure the ansible account has a "complex" password.

Ensure the complex password setting for the ansible user, be sure to make the password different
than the passphrase for the key, it would add nothing if You use the same password.
Only the complex password doesn't help without changing a bit of configuration in the system.
The sudoers file for the ansible user must (as per hardening guideline) never contain the "NOPWD"
directive. So, if its there, remove it!

Configure the new passwords in your RHAAP/AWX environments, ensuring each environment in Your DTAP
has an unique set.

To configure this through the infra.controller_configuration:

  - name: ansible
    description: This is a secure ansible credential using passwords on all options
    credential_type: Machine
    organization: Default
    inputs:
      become_method: sudo
      become_username: ''
      ssh_key_data: !vault |
          93284210394338921830912487394834908490484109
          32094812348921234894679847894728974892748466
          13838930293839218393801980102198301380380381
          93284210394338921830912487394834908490484109
          32094812348921234894679847894728974892748466
          13838930293839218393801980102198301380380381
          93284210394338921830912487394834908490484109
          32094812348921234894679847894728974892748466
          13838930293839218393801980102198301380380381
          93284210394338921830912487394834908490484109
          32094812348921234894679847894728974892748466
          13838930293839218393801980102198301380380381
          93284210394338921830912487394834908490484109
          32094812348921234894679847894728974892748466
          13838930293839218393801980102198301380380381
          3294890381293081203
      username: ansible
      ssh_key_unlock: !vault |
          09238430297893823092138398130921381209381903
          12098232109381203912830913810938210938109380
          23098487093481
      become_password: !vault |
          93284210394338921830912487394834908490484109
          32094812348921234894679847894728974892748466
          13838930293839218393801980102198301380380381
          3294890381293081203

Ensure all data is vaulted in your git repositories, it helps with security.