Add a secrets vault

Many organization use a vault to obscure secrets from git and other automation platforms.
As we are building a lab that should be enterprise like, we also need a secrets vault.

We chose to use an openbao container for this: - It is lightweight
- Industry standard (HashiCorp opensource)
- Easy to deploy and use

As we are using the containeized version, we need to install a docker machine first, nothing fancy, just docker..
In this docker machine, we need to apply the following configuration (standard docker):

We use rocky linux (rhel compliant)

install docker

Add docker repo

Create a repo file for the docker repository, with the folowing content vi /etc/yum.repos.d/docker.repo

content:

[docker-ce-stable]
name=Docker ce stable
baseurl=https://download.docker.com/linux/rhel/9/x86_64/stable
gpgcheck=0
enabled=1

packages:

  • docker-ce
  • docker-ce-cli
  • containerd.io
  • docker-buildx-plugin
  • docker-compose-plugin
  • nfs-utils
  • python3.11
  • python3.11-requests

Some of the packages are not needed for docker,but are essential when using ansible against this host.

Ensure some modules will be loaded by default

vi /etc/modules-load.d/docker.conf

content:

ip_tables
ip_conntrack
iptable_filter
ipt_state

Enable docker service

sudo systemctl enable docker sudo systemctl start docker

Install openbao container

From your home directory, create a new directory "openbao"
In the openbao directory, create a directory "config"

Now we create the following files:

openbao/config/bao.hcl

This is the configuration file for openbao.

ui = true

storage "file" {
  path = "/openbao/data/"
}

listener "tcp" {
  address     = "0.0.0.0:8200"
  tls_disable = 1
}

api_addr = "http://0.0.0.0:8200"

openbao/compose.yml

This is the container configuration for the docker container running openbao.
Ensure that the user id (1000) is NOT used on your system, if so, configure an other free id. If this id is in use, openbao will not work correctly. 

---
services:
  openbao:
    image: openbao/openbao
    container_name: openbao
    restart: always

    user: "100:1000"

    ports:
      - '8200:8200'

    command: server -config=/openbao/config/bao.hcl

    volumes:
      - openbao-data:/openbao/data
      - ./config/bao.hcl:/openbao/config/bao.hcl:ro
    cap_add:
      - IPC_LOCK

volumes:
  openbao-data: {}

Start the container docker compose up -d The UI is now reachable on the ip of the docker host and port 8200.

IMPORTANT
Go to the following directory:
/var/lib/docker/volumes
Here you will find the 'openbao' volume, set the access rights correctly:
chown -R 100:<used_id> <openbao-volume-name>

Now openbao will work for you.

Configure openbao

On first startup Openbao will ask (on the ui) to generate the master key and the number of unlock key parts.
When you first try to test openbao, it may be wise to set both at 1. This way you get a single key to unlock(unseal) your openbao server.
The openbao service start in sealed mode on every restart and you will have to unlock it. Start a browser and provide the unlock key, you vault will be unlocked now. As this is safe, in a homelab this is not a real issue, we want the vault to unlock automaticly (I do, as I have separated my lab from the internet).

This crontab option is NOT the preferred way to open the secrets vault, better ways can be found in the openbao documentation.

Other possibillities are: - an ansible playbook, using vaulted secrets - a kubernetes/openshift secret

But the crontab option is good enough for my homelab, as there are no real company secrets in there.

Unlock through crontab on the host: You need 2 files: - key.json - open_vault.sh

key.json

In this json file the unlock key is saved, to be provided to the api through curl

{
  "key": "<your-key>"
}

open_vault.sh

In this file you specify the curl command to be run on startup of the host.

#!/bin/bash
sleep 20
curl --request POST --data @key.json http://127.0.0.1:8200/v1/sys/unseal

Ensure that the script is executable: chmod +x open_vault.sh

The script sleeps for 20 seconds to give openbao the opportunity to startup before we try to unseal the vault. Adapt the commands to your distribution, or install bash and sleep packages.

crontab

Edit the crontab for root with sudo crontab -e and add the following line:

@reboot /root/open_vault.sh

Using Approles from vault