Adding a new team to the Controller
In the previous chapter, we have seen how and what we can manage in the controller by populating the files that are in a repository for a team. But how do we achieve that? We don't want to have to prepare and create everything by hand every time, that would easily take 2 days of work per team. So we're going to automate this, in this chapter we're going to explain what we need to do and we're going to give the code to do it as a gift. The code does what it's supposed to do, but there is always room for improvement, so do your thing and adapt it for your own organization. To add a team that can work autonomously, we need to add it to the basic configuration, after all, that's where the data that determines whether a team (read: organization) exists. This concerns the following data: * Team name * The Admin account * The admin password * LDAP/AD config (if applicable) * De ansible key
Just changing the basic configuration is not enough to give a new team a flying start. Basically, what we want is to make sure that the new team at the time they're added to the controller, they have a configuration in a repository with a pipeline that they can customize. After each change, the pipeline must start running and make the change to the controller of the development environment. The ansible key does not have to be in the base config, but it is better to do so in terms of security, after all, you give away "root" access, you would put it in the user repository, the value can be deciphered with the vault password and secure root access is no longer guaranteed. Once this is taken care of, the way to the successor environments has become easy.
What we're going to do is make a two-stage rocket: * The first stage modifies the base config * The second takes care of the repository and pipeline
We will discuss these step by step.
Customizing the basic configuration
The code in this chapter will add a new organization to the existing configuration of the controller. We do this with code exactly as a human would, i.e. using the GitOps (configuration as code) method.
The steps are as follows: * Clone de base config repository * Updates the files with the new data * Commit and push the changes * Let the pipeline do its job
Once this pipeline has done its job, the new organization is known in the controller of
the development environment and the organization can be populated by the
organization admin. If AD/LDAP is used, users can log in to a hitherto empty
organization.
To do this in the following environments, merge requests must be executed by the
management team of the controller environments.
How do we do this in code:
Variables
Almost all the variables needed to build this are included in the env_vars.yml file.
Please note! This contains passwords, make sure they are encrypted with
vault!
The vault password for this file is added to the controller as a credential, so it can
only be executed from the controller. Partly because a second staircase has to be
added to the back, we want to arrange that in the same workflow.
The workflow will eventually ask for a few values via a survey: * organization_short_name * team_password
The above variables will be used to generate the names and the password will be used to encrypt the secrets with vault. For security reasons, we recommend that the password be at least 15 characters, with a base64 character set. The base64 is needed because gitlab asks for it if you want to be able to mask secrets in pipelines, since we are going to give this password to the pipeline, we don't want it to show up in the logs.
In the case of a multi-AD/LDAP setup, a third parameter must be asked in which LDAP configuration it should be added.
- ldap_number
This value is in the env_vars by default, but if this configuration is necessary, this value should be removed there and added in the survey. The code can be used with or without LDAP configuration. If no LDAP is in use, set the "add_ldap" variable to "false" in the env_vars.yml, if there is an LDAP, set it to true.
Contents of the env_vars.yaml:
---
# put your vars in here and make sure this file is ALWAYS vault encrypted
# the values in this file will be encrypted and used in the config files.
organization_long_name: 'ORG_{{ organization_short_name }}'
gitlab_protocol: 'https://'
gitlab_url: '{gitlab_url}/'
project_name: {project_name_for_base_config}
gitlab_user: {service_account}
gitlab_password: {serviceaccount_passwd}
gitlab_group: {group_for_config_as_code}
add_ldap: true
ldap_number: '1'
ldap:
- name: 1
ldap_pre_str: 'CN=G-AAP-'
ldap_post_str: ',OU=AUTM,OU=Groups,OU=Company,DC=example,DC=com'
- name: 2
ldap_pre_str: 'CN=G-AAP-'
ldap_post_str: ',OU=AUTM,OU=Groups,OU=Another,DC=example,DC=com'
- name: 3
ldap_pre_str: 'CN=G-AAP-'
ldap_post_str: ',OU=AUTM,OU=Groups,OU=Another,DC=example,DC=com'
- name: 4
ldap_pre_str: 'CN=G-AAP-'
ldap_post_str: ',OU=AUTM,OU=Groups,OU=Another,DC=example,DC=com'
- name: 5
ldap_pre_str: 'CN=G-AAP-'
ldap_post_str: ',OU=AUTM,OU=Groups,OU=Another,DC=example,DC=com'
global_credentials_vars:
all:
credentials:
- name: "{{ organization_short_name }}_gitlab"
description: 'SCM credential'
credential_type: Source Control
encrypt: |
-----BEGIN OPENSSH PRIVATE KEY-----
-----END OPENSSH PRIVATE KEY-----
username: AAP_user
Dev:
credentials:
- name: "{{ organization_short_name }}_ansible"
description: 'Machine credential development'
credential_type: Machine
username: ansible
encrypt: |
-----BEGIN OPENSSH PRIVATE KEY-----
-----END OPENSSH PRIVATE KEY-----
test:
credentials:
- name: "{{ organization_short_name }}_ansible_test"
description: 'Machine credential test'
credential_type: Machine
username: ansible
encrypt: |
-----BEGIN OPENSSH PRIVATE KEY-----
-----END OPENSSH PRIVATE KEY-----
The playbook
The heart of this part of the configuration customization, the playbook. We're going to go through the steps of the playbook here, so it's here in pieces, but all of these pieces make up 1 playbook. So with some cutting and pasting, you can have the whole playbook in no time.
---
- hosts: localhost
gather_facts: false
pre_tasks:
- name: Get vars
ansible.builtin.include_vars: env_vars.yml
- name: Clone the new gitlab repository
ansible.builtin.git:
repo: "https://{{ gitlab_user }}:{{ gitlab_password }}@{{ gitlab_url }}/{{ gitlab_group }}/{{ project_name }}.git"
dest: "/tmp/{{ project_name }}"
version: dev
clone: true
update: true
In part 1 of the playbook, we bring in the variables and based on the variables, we clone the repository in which the basic configuration is stored. We always clone the repository again, because we have no guarantee that it will survive a pipeline run. If the pipeline runs in an image, the image will be "fresh" every time.
tasks:
- name: Set execute bit on script
ansible.builtin.file:
path: ./encrypt.sh
mode: u+x,g-rw,o-rw
- name: Create the Organization
ansible.builtin.blockinfile:
path: "/tmp/{{ project_name }}/group_vars/all/organization.yaml"
insertbefore: ...
marker: "# {mark} ANSIBLE MANAGED BLOCK {{ organization_long_name }}"
marker_begin: "# BEGIN BLOCK {{ organization_long_name }}"
marker_end: "# END BLOCK {{ organization_long_name }}"
block: |
{% filter indent(width=2, first=true) %}
- name: {{ organization_long_name }}
description: Organization for team {{ organization_short_name }}
{% endfilter %}
We add the new name to the organization.yaml in the all directory of the config as code, so we can be sure that it will exist in any environment where it is emerged. We also make sure that this block is uniquely marked, so that code can also be created to remove it again.
- name: Create the user for the ORG
ansible.builtin.blockinfile:
path: "/tmp/{{ project_name }}/group_vars/all/users.yaml"
insertbefore: ...
marker: "# {mark} ANSIBLE MANAGED BLOCK {{ organization_long_name }}"
marker_begin: "# BEGIN BLOCK {{ organization_long_name }}"
marker_end: "# END BLOCK {{ organization_long_name }}"
block: |
{% filter indent(width=2, first=true) %}
- username: CaC_admin_{{ organization_short_name }}
password: {{ team_password }}
email:
first_name: admin
last_name: admin for {{ organization_short_name }}
auditor: false
superuser: false
update_secrets: false
{% endfilter %}
We add a new user to the users.yaml in the all directory of the config as code, so we can be sure that it will exist in any environment where it is emerged. We also make sure that this block is uniquely marked, so that code can also be created to remove it again. This user will become the new admin of the organization added in the previous task.
- name: Make that user ORG_ADMIN
ansible.builtin.blockinfile:
path: "/tmp/{{ project_name }}/group_vars/all/roles.yaml"
insertbefore: ...
marker: "# {mark} ANSIBLE MANAGED BLOCK {{ organization_long_name }}"
marker_begin: "# BEGIN BLOCK {{ organization_long_name }}"
marker_end: "# END BLOCK {{ organization_long_name }}"
block: |
{% filter indent(width=2, first=true) %}
- user: CaC_admin_{{ organization_short_name }}
organization: {{ organization_long_name }}
role: admin
{% endfilter %}
We make sure that the newly added user gets the admin role for the new organization. We do this by adding it to the roles.yaml in the all directory of the config as code, so we can be sure that it will exist in any environment where it is emerged. We also make sure that this block is uniquely marked, so that code can also be created to remove it again.
- name: Configure LDAP groups if enabled
ansible.builtin.include_tasks:
file: add_LDAP_team.yml
when: add_ldap
If LDAP is set to true, we bring in a few extra tasks to put the parameters for this in the configuration. This is explained below.
- name: Loop over the envs to add credentials
ansible.builtin.include_tasks:
file: add_creds.yml
Vars:
curr_env: "{{ gcreds.key }}"
env_vars: "{{ gcreds.value }}"
loop: "{{ global_credentials_vars | dict2items }}"
loop_control:
loop_var: gcreds
no_log: true
- name: Push the updated GitLab repository
ansible.builtin.shell: |
git config --global user.name "{{ gitlab_user }}"
git config --global user.email "{{ gitlab_user }}@rivm.nl"
git add --all
git commit -m "Admin for {{ organization_short_name }} added"
git push origin dev
args:
chdir: "/tmp/{{ project_name }}"
changed_when: false
- name: Delete the tempory directory
ansible.builtin.file:
path: /tmp/{{ project_name }}
state: absent
- name: Wait for 10 secs
ansible.builtin.pause:
seconds: 10
All files have now been modified to reflect the new organization in the basic configuration. For this we have to make sure that everything is neatly recorded in GIT according to the git process, this will trigger the pipeline and put the files in the controller as a configuration.
- name: GitLab Post | Obtain Access Token
ansible.builtin.uri:
url: "https://gitlab.localdomain/oauth/token"
method: POST
validate_certs: false
body_format: json
headers:
Content-Type: application/json
body: >
{
"grant_type": "password",
"username": "{{ gitlab_user }}",
"password": "{{ gitlab_password }}"
}
register: gitlab_access_token
- name: Store the token in var
ansible.builtin.set_fact:
token: "{{ gitlab_access_token.json.access_token }}"
- name: Check the pipeline until it has run
ansible.builtin.uri:
url: "https://gitlab.localdomain/api/v4/projects/CaC%2Faap_cac_base/jobs"
validate_certs: false
headers:
Authorization: "Bearer {{ token }}"
register: _jobs_list
until: _jobs_list.json[0].pipeline.status == "success"
retries: 20
delay: 15
We want to be sure that everything has been properly implemented in gitlab and that the pipeline has run successfully, for this we regularly check whether this is the case. If so, this part is complete and we can start the next step for the organization configuration.
add_LDAP_team.yml
If the variable add_ldap is set to true, these additional tasks will also be performed
within the play, which will add the ldap configuration for this particular team to the
settings.yml in the all directory of the base configuration.
This takes care of 2 things:
* The groups are linked to the organization
* The LDAP groups are assigned to specific teams within the organization
---
- name: Set ldap facts
ansible.builtin.set_fact:
ldap_pre: "{{ ldap | selectattr('name', 'match', ldap_number) |
map(attribute='ldap_pre_str') | join() }}"
ldap_post: "{{ ldap | selectattr('name', 'match', ldap_number) |
map(attribute='ldap_post_str') | join() }}"
- name: "Add a block of text to the settings.yml file AUTH_LDAP_{{ ldap_number
}}_ORGANIZATION_MAP"
ansible.builtin.blockinfile:
path: "/tmp/{{ project_dir }}/group_vars/all/settings.yaml"
block: |
{% filter indent(width=6, first=true) %}
{{ organization_long_name + ':' }}
admins: {{ ldap_pre + organization_short_name + '-A' + ldap_post }}
remove_admins: false
remove_users: true
users: {{ ldap_pre + organization_short_name + '-U' + ldap_post }}
{% endfilter %}
marker: "# {mark} ANSIBLE MANAGED BLOCK {{ organization_short_name }}
ORGANIZATION_MAP"
marker_begin: "# BEGIN ANSIBLE MANAGED BLOCK {{ organization_short_name }}"
marker_end: "# END ANSIBLE MANAGED BLOCK {{ organization_short_name }}"
create: true
backup: false
insertafter: '(?m)^.*AUTH_LDAP_{{ ldap_number
}}_ORGANIZATION_MAP[\s\S].*value:.*'
- name: "Add a block of text to the settings.yml file AUTH_LDAP_{{ ldap_number
}}_TEAM_MAP"
ansible.builtin.blockinfile:
path: "/tmp/{{ project_dir }}group_vars/all/settings.yaml"
block: |
{% filter indent(width=6, first=true) %}
{{ 'LDAP_' + organization_short_name + '_Admins:' }}
organization: {{ organization_long_name }}
remove: true
users: {{ ldap_pre + organization_short_name + '-A' + ldap_post }}
{{ 'LDAP_' + organization_short_name + '_Developers:' }}
organization: {{ organization_long_name }}
remove: true
users: {{ ldap_pre + organization_short_name + '-D' + ldap_post }}
{{ 'LDAP_' + organization_short_name + '_Operators:' }}
organization: {{ organization_long_name }}
remove: true
users: {{ ldap_pre + organization_short_name + '-O' + ldap_post }}
{% endfilter %}
marker: "# {mark} ANSIBLE MANAGED BLOCK {{ organization_short_name }} TEAM_MAP"
marker_begin: "# BEGIN ANSIBLE MANAGED BLOCK {{ organization_short_name }}"
marker_end: "# END ANSIBLE MANAGED BLOCK {{ organization_short_name }}"
create: true
backup: false
insertafter: '(?m)^.*AUTH_LDAP_{{ ldap_number
}}_TEAM_MAP[\s\S].*value:.*'
add_creds.yml
In the main playbook, the credentials for the ansible user and the git user are not created, the playbook calls the tasks from add_creds.yml, so these credentials are stored vault encrypted. In turn, add_creds.yml invokes a secure_loop.yml which in turn executes a shell script. Not quite great, but it works great, below you can see all the scripts and playbooks:
---
- name: "Loop to write secure credentials"
ansible.builtin.include_tasks:
file: secure_loop.yml
loop: "{{ env_vars.credentials }}"
loop_control:
loop_var: creds
when: env_vars.credentials | length > 0
no_log: true
secure_loop.yml:
---
- name: Be sure to clean the file with unencrypted value
block:
- name: Write token to file
ansible.builtin.copy:
content: "{{ creds.encrypt }}"
dest: /tmp/file
mode: "0600"
- name: Encrypt a var and register result
ansible.builtin.command:
argv:
- ./encrypt.sh
- /tmp/file
- dummy
- "{{ base_vault_password }}"
register: _vaulted
changed_when: false
always:
- name: Clean the unencrypted file
ansible.builtin.file:
path: /tmp/file
state: absent
- name: Write the encrypted credential to file
ansible.builtin.blockinfile:
path: "/tmp/{{ project_name }}/group_vars/{{ curr_env }}/credentials.yaml"
insertbefore: ...
marker: "# {mark} ANSIBLE MANAGED BLOCK {{ creds.name }}"
marker_begin: "# BEGIN BLOCK {{ organization_long_name }}"
marker_end: "# END BLOCK {{ organization_long_name }}"
block: |
{% filter indent(width=2, first=true) %}
- name: {{ creds.name }}
{% if cred.description is defined %}
description: {{ creds.description }}
{% else %}
description:
{% endif %}
credential_type: {{ creds.credential_type }}
organization: {{ organization_long_name }}
inputs:
{% if creds.credential_type == 'Source Control' %}
ssh_key_data: {{ _vaulted.stdout }}
username: {{ creds.username }}{% endif %}
{% if creds.credential_type == 'Machine' %}
become_method: sudo
become_username: ''
ssh_key_data: {{ _vaulted.stdout }}
username: {{ creds.username }}{% endif %}
{% endfilter %}
encrypt.sh
#!/bin/bash
# encrypt ansible vars, usage:
echo ${3} > /tmp/pwd
cat ${1}|ansible-vault encrypt_string --name ${2} --vault-password-file /tmp/pwd
rm -f /tmp/pwd
That's all it takes to add to the basic configuration.
Basically, the team can now log in to the controller and start using it via the UI. However, this is not what we want from a safeguarding perspective (GitOps). Everything one does in the UI is not recorded in GIT.
That's why we go a step further and have a follow-up project that we run through as
step 2 after this one in the workflow.
Create Team git repository
Where we assumed that the repository already existed when we changed the basic
configuration, we now have to assume that this is not the case, and that we have to
create it.
If the repository isn't there, there's no base set of files. This requires a little
more work.
There is 1 piece that is used by multiple tasks and that is a shell script to encrypt
vault passwords, which is used in multiple places and by placing it in the main
repository, it can be used by all playbooks.
encrypt.sh:
#!/bin/bash
# encrypt ansible vars, usage:
echo ${3} > /tmp/pwd
cat ${1}|ansible-vault encrypt_string --name ${2} --vault-password-file /tmp/pwd
rm -f /tmp/pwd
The script is written in such a way, that it doesn't matter if you're going to encode a single line string or a whole ssh-key, the answer is usable in all cases. Variables In the basic configuration we used the env_vars.yml, which we will use again here. He seems to be the same, but he certainly isn't! There is a certain overlap in the file and the variables.
env_vars.yml:
---
# put your vars in here and make sure this file is ALWAYS vault encrypted
# the values in this file will be encrypted and used in the config files.
organization_long_name: 'ORG_{{ organization_short_name }}'
gitlab_protocol: 'https://'
gitlab_url: '<gitlab_url>'
gitlab_user: {service_account}
gitlab_password: {serviceaccount_passwd}
gitlab_group: {group_for_config_as_code}
project_name: '<pre_str>_{{ organization_short_name | lower }}'
aap_env:
dev:
controller_hostname: <dev_controller_url>
automationhub_hostname: <dev_automationhub_url>
controller_admin_password: <dev_password>
test:
controller_hostname: <rest_controller_url>
automationhub_hostname: <test_automationhub_url>
controller_admin_password: <test_password>
accp:
controller_hostname: <accp_controller_url>
automationhub_hostname: <accp_automationhub_url>
controller_admin_password: <accp_password>
Prod:
controller_hostname: <prod_controller_url>
automationhub_hostname: <prod_automationhub_url>
controller_admin_password: <prod_password>
Since this also contains passwords, make sure that the vaiables in this file are always vault encrypted.
The content of the variables will be easy to guess and we won't go into that any
further.
Just as the env_vars.yml the environment has variables in sight, another file of
variables is needed. This determines the content of the repository that will be made
available to the teams. Below is an overview.
other_vars.yml:
---
# put your vars in here and make sure the encrypt vars in this file are ALWAYS
vault encrypted, the encrypt var is used to encrypt passwords, tokens or keys.
# the values in this file will be encrypted and used in the config files.
code_environment_vars:
all:
credentials:
- name: "{{ organization_short_name }}_gitlab"
credential_type: Source Control
encrypt: |
-----BEGIN OPENSSH PRIVATE KEY-----
This is a placeholder, use the vaulted key here
-----END OPENSSH PRIVATE KEY-----
username: AAP_user
inventories:
- name: "{{ organization_short_name }}_demo_inventory"
description: 'Demo inventory, not functional'
organization: "{{ organization_long_name }}"
inventory_sources:
- name: "{{ organization_short_name }}_demo_inventory"
description: 'Just a demo, not functional'
organization: "{{ organization_long_name }}"
organizations:
- description: 'Description'
projects:
- name: "{{ organization_short_name }}_demo_play"
description: Demo base OS project
scm_url: "git@{{ gitlab_url }}:example_group/example_play.git"
roles: []
teams:
- use: here
templates:
- use: here
dev:
credentials:
- name: "{{ organization_short_name }}_ansible"
credential_type: Machine
encrypt: |
-----BEGIN OPENSSH PRIVATE KEY-----
This is a placeholder, use the vaulted key here
-----END OPENSSH PRIVATE KEY-----
- name: "{{ organization_short_name }}_automation_hub_token_published"
credential_type: Ansible Galaxy/Automation Hub API Token
auth_url: ''
url: "https://{{ aap_env['dev'].automationhub_hostname }}/api/galaxy/"
- name: "{{ organization_short_name }}_automation_hub_token_community"
credential_type: Ansible Galaxy/Automation Hub API Token
auth_url: ''
url: "https://{{ aap_env['dev'].automationhub_hostname
}}/api/galaxy/content/community/"
- name: "{{ organization_short_name }}_automation_hub_token_rh_certified"
credential_type: Ansible Galaxy/Automation Hub API Token
auth_url: ''
url: "https://{{ aap_env['dev'].automationhub_hostname
}}/api/galaxy/content/rh-certified"
inventories: []
inventory_sources: []
organizations: []
projects:
- name: "{{ organization_short_name }}_demo_inventory"
description: inventory project
scm_url: "git@{{ gitlab_url }}:example_group/example_inventory.git
roles:
- team: "LDAP_{{ organization_short_name }}_Developers"
teams: []
templates: []
test:
credentials: []
inventories: []
inventory_sources: []
organizations: []
projects: []
roles: []
teams: []
templates: []
accp:
credentials: []
inventories: []
inventory_sources: []
organizations: []
projects: []
roles: []
teams: []
templates: []
Prod:
credentials: []
inventories: []
inventory_sources: []
organizations: []
projects: []
roles: []
teams: []
templates: []
Based on the above variables, the various files are populated with data that will be loaded into the development environment via the pipeline. These variables are particularly used by the role create_group_vars. Before we start screening the roles, we will first need to have a folder in which we can write. So we're going to go through the main playbook first and then we'll discuss the roles.
The playbook
Below is the "big fairy tale book" to create the repository and fill it with the necessary data. We're not going to interrupt to explain, but add comments in the text.
main.yml:
---
- hosts: localhost
gather_facts: false
pre_tasks:
# Get the vars from the file into the play
- name: Get vars
ansible.builtin.include_vars: env_vars.yml
# Create a new repository in gitlab for the team
# and ensure that the pipeline must succeed and commits are not squashed
- name: "Create GitLab Project in group {{ gitlab_group }}"
community.general.gitlab_project:
api_url: "{{ gitlab_protocol }}{{ gitlab_url }}"
validate_certs: true
api_username: "{{ gitlab_user }}"
api_password: "{{ gitlab_password }}"
name: "{{ project_name }}"
group: "{{ gitlab_group }}"
default_branch: dev
only_allow_merge_if_pipeline_succeeds: true
squash_option: never
shared_runners_enabled: true
initialize_with_readme: true
state: present
# In the new repository, create the nessecery branches for the
# controller environments, these are defined in the env_vars.yml
- name: Create the environment branches on repository
community.general.gitlab_branch:
api_url: "{{ gitlab_protocol }}{{ gitlab_url }}"
api_username: "{{ gitlab_user }}"
api_password: "{{ gitlab_password }}"
project: "{{ gitlab_group }}/{{ project_name }}"
branch: "{{ curr_env }}"
ref_branch: dev
state: present
vars:
curr_env: "{{ cenv.key }}"
loop: "{{ aap_env | dict2items }}"
loop_control:
loop_var: cenv
no_log: true
# On the new repository set the vault secret for the pipeleine in de CI/CD
variables
- name: Set vault_secret CI/CD variables
community.general.gitlab_project_variable:
api_url: "{{ gitlab_protocol }}{{ gitlab_url }}"
api_username: "{{ gitlab_user }}"
api_password: "{{ gitlab_password }}"
project: "{{ gitlab_group }}/{{ project_name }}"
purge: false
variables:
- name: VAULT_PASSWORD
value: "{{ team_password }}"
masked: true
protected: false
environment_scope: '*'
# make sure the encrypt shell script is executable, its copied every time this
is run.
- name: Set execute bit on script
ansible.builtin.file:
path: ./encrypt.sh
mode: u+x,g-rw,o-rw
# now we will clone the new (and empty) repository to a new directory in /tmp
# so we can fill this with the files we want.
- name: Clone the new gitlab repository
ansible.builtin.git:
repo: "{{ gitlab_protocol }}{{ gitlab_user }}:{{ gitlab_password }}@{{
gitlab_url }}/{{ gitlab_group }}/{{ project_name }}.git"
dest: "/tmp/{{ project_name }}"
version: dev
clone: true
update: true
tasks:
# Get the additional vars into the playbook to fill the files in the repository
- name: Get other vars
ansible.builtin.include_vars:
file: other_vars.yml
# Checkout a new branch to push and merge
- name: Checkout the branch to push
ansible.builtin.shell: |
git config --global user.name "{{ gitlab_user }}"
git config --global user.email "{{ gitlab_user }}@rivm.nl"
git checkout -b initial
args:
chdir: "/tmp/{{ project_name }}"
# Create the needed directories in the gitlab repository for the
# group_vars
- name: Create group_var directories
ansible.builtin.file:
path: "/tmp/{{ project_name }}/group_vars/{{ curr_env }}"
state: directory
mode: '0755'
loop: "{{ env_list }}"
loop_control:
loop_var: curr_env
- name: Create host_var directories
ansible.builtin.file:
path: "/tmp/{{ project_name }}/host_vars/controller_{{ curr_env }}"
state: directory
mode: '0755'
loop: "{{ env_list }}"
loop_control:
loop_var: curr_env
when: curr_env != 'all'
- name: Template the pipeline inventory
ansible.builtin.template:
src: repo_inventory.yaml.j2
dest: "/tmp/{{ project_name }}/inventory.yaml"
mode: "0644"
# We need to template the contoller_auth.yml files
# The secrets will be vaulted, so we will loop.
- name: "Render controller_auth.yaml files {{ curr_env }}"
ansible.builtin.include_tasks:
file: controller_loop.yml
loop: "{{ env_list }}"
loop_control:
loop_var: curr_env
when: curr_env != 'all'
- name: Template the controller_env.yml
ansible.builtin.template:
src: controller_env.yml.j2
dest: "/tmp/{{ project_name }}/host_vars/controller_{{ curr_env1
}}/controller_{{ curr_env1 }}.yaml"
mode: "0644"
loop: "{{ env_list }}"
loop_control:
loop_var: curr_env1
when: curr_env1 != 'all'
- name: Copy the main.yml playbook
ansible.builtin.copy:
src: main.yml
dest: "/tmp/{{ project_name }}/main.yml"
mode: "0644"
- name: Template the pipeline
ansible.builtin.template:
src: gitlab-ci.yml.j2
dest: "/tmp/{{ project_name }}/.gitlab-ci.yml"
mode: "0644"
# We now create the list of files to put in each group_vars directory
- name: Create the list of templates
ansible.builtin.set_fact:
template_files:
- credentials
- inventory
- notifications
- organization
- projects
- roles
- schedules
- teams
- templates
- users
- workflows
# joint he files with the environments to generate a list we can itterte
# just once, without the need for a double loop construct
- name: Create the loop var
ansible.builtin.set_fact:
template_loop: |-
[
{%- for env in code_environment_vars -%}
{%- for file in template_files -%}
'{{ env }},{{ file }}'
{%- if not loop.last -%},
{%- endif -%}
{%- endfor -%}
{%- if not loop.last -%},
{%- endif -%}
{%- endfor -%}
]
- name: "Template files loop"
ansible.builtin.template:
src: "{{ curr_file.split(',')[-1] }}.yml.j2"
dest: "/tmp/{{ project_name }}/group_vars/{{ curr_file.split(',')[0] }}/{{
curr_file.split(',')[-1] }}.yml"
lstrip_blocks: true
mode: "0640"
loop: "{{ template_loop }}"
loop_control:
loop_var: curr_file
# Add the vaulted credentials to the files
- name: "Loop to write secure credentials"
ansible.builtin.include_tasks:
file: secure_group.yml
vars:
curr_env2: "{{ lcred[0] }}"
creds: "{{ lcred[1] }}"
loop_control:
loop_var: lcred
loop: |-
[
{%- for key, env_item in code_environment_vars.items() -%}
{%- if env_item.credentials != [] -%}
{%- for cred in env_item.credentials -%}
['{{ key }}',{{ cred }}],
{%- endfor -%}
{%- endif -%}
{%- endfor -%}
]
when: creds.credential_type != 'Machine'
no_log: true
# Push the new branch to gitlab
- name: Push the updated GitLab repository
ansible.builtin.shell: |
git add --all
git commit -m 'initial config'
git push origin initial
args:
chdir: "/tmp/{{ project_name }}"
changed_when: false
- name: Create Merge Request from initial to dev
community.general.gitlab_merge_request:
api_url: "{{ gitlab_protocol }}{{ gitlab_url }}"
api_username: "{{ gitlab_user }}"
api_password: "{{ gitlab_password }}"
project: "{{ gitlab_group }}/{{ project_name }}"
source_branch: initial
target_branch: dev
title: "Initial config from code"
description: "First org configuration"
state_filter: "opened"
remove_source_branch: true
state: present
# leave the pipeline environment with a clean tmp
- name: Delete the tempory directory
ansible.builtin.file:
path: /tmp/{{ project_name }}
state: absent
# This block depends on the presence of the gitlab python extension, if this
runs without an error
# The message is not displayed. In most cases you will need a execution
environment that has the extension installed.
# The play will fail as the message is displayed
- name: Protected branch block
block:
- name: Protect created branches
community.general.gitlab_protected_branch:
api_url: "{{ gitlab_protocol }}{{ gitlab_url }}"
api_username: "{{ gitlab_user }}"
api_password: "{{ gitlab_password }}"
project: "{{ gitlab_group }}/{{ project_name }}"
name: "{{ benv.key }}"
merge_access_levels: maintainer
push_access_level: nobody
loop: "{{ aap_env | dict2items }}"
loop_control:
loop_var: benv
rescue:
- name: Print only in case of protection error
ansible.builtin.debug:
msg:
- "The branches for the repository were not protected, due to an
error."
- "The error is that the account used to create the repository is not
"
- "the owner of the repository."
- "Please set the protection manually...."
- name: Fail the playbook due to error.
ansible.builtin.fail:
msg: "Creation was succesfull, only the branch protection was not set,
see message above"
# After pushing and creating a merge request we must check if it runs
# first we need a token
- name: GitLab Post | Obtain Access Token
ansible.builtin.uri:
url: "{{ gitlab_protocol }}{{ gitlab_url }}/oauth/token"
method: POST
validate_certs: false
body_format: json
headers:
Content-Type: application/json
body: >
{
"grant_type": "password",
"username": "{{ gitlab_user }}",
"password": "{{ gitlab_password }}"
}
register: gitlab_access_token
- name: Store the token in var
ansible.builtin.set_fact:
token: "{{ gitlab_access_token.json.access_token }}"
# We check if the merge request is there
- name: Check the merge request
ansible.builtin.uri:
url: "{{ gitlab_protocol }}{{ gitlab_url }}/api/v4/projects/{{ gitlab_group
}}%2F{{ project_name }}/merge_requests"
validate_certs: false
headers:
Authorization: "Bearer {{ token }}"
register: _requests
# To ensure we have no interaction with gitlab to create this, we set the
request to auto merge
# So when the pipeline finishes successfull, it will merge.
- name: Set the merge request to automerge
ansible.builtin.uri:
url: "{{ gitlab_protocol }}{{ gitlab_url }}/api/v4/projects/{{ gitlab_group
}}%2F{{ project_name }}/merge_requests/{{ _requests.json[0].iid
}}/merge?merge_when_pipeline_succeeds=true"
method: POST
validate_certs: false
headers:
Authorization: "Bearer {{ token }}"
# Keep checking the pipline, the result is used to finish the play with the
correct status
- name: Check the pipeline until it has run
ansible.builtin.uri:
url: "{{ gitlab_protocol }}{{ gitlab_url }}/api/v4/projects/{{ gitlab_group
}}%2F{{ project_name }}/jobs"
validate_certs: false
headers:
Authorization: "Bearer {{ token }}"
register: _jobs_list
until: _jobs_list.json[0].pipeline.status == "success"
retries: 20
delay: 10
The playbook is fairly simple for the complex whole that the result actually is. The real complexity lies in the roles used to generate the contents of the repository. We are now going to go through and explain each of them.
files/main.yml
This is the playbook that will be used by the repository to load the variables added by the team into the controller.
---
- hosts: "{{ instance }}"
gather_facts: false
connection: local
pre_tasks:
- name: Set credentials_var
ansible.builtin.set_fact:
controller_credentials: "{{ controller_credentials_all +
controller_credentials_dev }}"
when: branch_name == 'dev'
- name: Set credentials_var
ansible.builtin.set_fact:
controller_credentials: "{{ controller_credentials_all +
controller_credentials_test }}"
when: branch_name == 'test'
- name: Set credentials_var
ansible.builtin.set_fact:
controller_credentials: "{{ controller_credentials_all +
controller_credentials_accp }}"
when: branch_name == 'accp'
- name: Set credentials_var
ansible.builtin.set_fact:
controller_credentials: "{{ controller_credentials_all +
controller_credentials_prod }}"
when: branch_name == 'prod'
- name: Set the controller vars
ansible.builtin.set_fact:
controller_inventories: >
{{ controller_inventories_all |
community.general.lists_mergeby( vars['controller_inventories_' +
branch_name],
'name', recursive=true, list_merge='append' ) }}
controller_inventory_sources: >
{{ controller_inventory_sources_all |
community.general.lists_mergeby( vars['controller_inventory_sources_' +
branch_name],
'name', recursive=true, list_merge='append' ) }}
controller_notifications: >
{{ controller_notifications_all |
community.general.lists_mergeby( vars['controller_notifications_' +
branch_name],
'name', recursive=true, list_merge='append' ) }}
controller_organizations: >
{{ controller_organizations_all |
community.general.lists_mergeby( vars['controller_organizations_' +
branch_name],
'name', recursive=true, list_merge='append' ) }}
controller_projects: >
{{ controller_projects_all |
community.general.lists_mergeby( vars['controller_projects_' +
branch_name],
'name', recursive=true, list_merge='append' ) }}
controller_roles: >
{{ controller_roles_all |
community.general.lists_mergeby( vars['controller_roles_'+ branch_name],
'role', recursive=true, list_merge='append' ) }}
controller_schedules: >
{{ controller_schedules_all |
community.general.lists_mergeby( vars['controller_schedules_' +
branch_name],
'name', recursive=true, list_merge='append' ) }}
controller_teams: >
{{ controller_teams_all |
community.general.lists_mergeby( vars['controller_teams_' + branch_name],
'name', recursive=true, list_merge='append' ) }}
controller_templates: >
{{ controller_templates_all |
community.general.lists_mergeby( vars['controller_templates_' +
branch_name],
'name', recursive=true, list_merge='append' ) }}
controller_workflows: >
{{ controller_workflows_all |
community.general.lists_mergeby( vars['controller_workflows_' +
branch_name],
'name', recursive=true, list_merge='append' ) }}
roles:
- infra.controller_configuration.dispatch
As you may have noticed, this one is exactly the same as the one in the previous chapter. The layout is also the same as that of the basic configuration. In the base configuration, only other variables are merged before the collection is called. We need some additional playbooks to run the complete config. We need to add vaulted credentials to the files, the playbooks below will do just that.
controller_loop.yml
This piece of playbook uses the shell scripting bit mentioned at the beginning to encrypt passwords. The first step is to write a file containing the password to be encrypted, the second step is to use that file together with the vault password to perform the encryption and we store the result in a variable. Whatever happens, we want to make sure that the temporary file is erased, so we actively discard it. We write that resulting variable here in the template.
---
- name: Be sure to clean file with credential
block:
- name: Write token to file
ansible.builtin.copy:
content: "{{ team_password }}"
dest: /tmp/file
mode: "0644"
- name: Encrypt a var and register result
ansible.builtin.command:
argv:
- ./encrypt.sh
- /tmp/file
- dummy
- "{{ team_password }}"
register: _vaulted
changed_when: false
always:
- name: Clean the unencrypted file
ansible.builtin.file:
path: /tmp/file
state: absent
- name: Write encrypted the credential to file
ansible.builtin.template:
src: controller_auth.yml.j2
dest: "/tmp/{{ project_name }}/host_vars/controller_{{ curr_env
}}/controller_auth.yaml"
lstrip_blocks: true
mode: "0644"
secure_group.yml
This piece of the playbook will add the vaulted credentials to the group_vars, using the same shell script as the above play.
---
- name: Be sure to clean the file with unencrypted value
when: creds.encrypt is defined
block:
- name: Write token to file
ansible.builtin.copy:
content: "{{ creds.encrypt }}"
dest: /tmp/file
mode: "0600"
- name: Encrypt a var and register result
ansible.builtin.command:
argv:
- ./encrypt.sh
- /tmp/file
- dummy
- "{{ team_password }}"
register: _vaulted
changed_when: false
always:
- name: Clean the unencrypted file
ansible.builtin.file:
path: /tmp/file
state: absent
- name: Write the encrypted credential to file
ansible.builtin.blockinfile:
path: "/tmp/{{ project_name }}/group_vars/{{ curr_env2 }}/credentials.yml"
insertafter: EOF
marker: ''
block: |
{% filter indent(width=2, first=true) %}
- name: {{ creds.name }}
{% if cred.description is defined %}
description: {{ creds.description }}
{% else %}
description:
{% endif %}
credential_type: {{ creds.credential_type }}
organization: {{ organization_long_name }}
inputs:
{% if creds.credential_type == 'Ansible Galaxy/Automation Hub API Token' %}
auth_url: ''
token: {{ '"{{ ahub_token }}"' }}
url: '{{ creds.url }}'
update_secrets: true
{% endif %}
{% if creds.credential_type == 'Source Control' %}
ssh_key_data: {{ _vaulted.stdout }}
username: {{ creds.username }}
{% endif %}
{%- endfilter -%}?
templates/controllers_auth.yml.j2
In this file, the authentication for the infrastructure collection is recorded for the environment that needs to be configured.
---
controller_hostname: {{ env_vars.controller_hostname }}
controller_validate_certs: false
controller_username: CaC_admin_{{ organization_short_name }}
controller_password: {{ _vaulted.stdout }}
ahub_hostname: {{ env_vars.automationhub_hostname }}
Please note that the admin user has a fixed structure for an organization. The password is written as a vaulted value in this file and is the encrypted password for this user. The rest of the variables mentioned may have a known content.
templates/gitlab-ci.yml.j2
The pipeline for the new repository.
# Pull the ansible config as code image
image: localhost:5000/ansible-image:1.0
# List of pipeline stages
stages:
- lint
- Configure RHAAP Controller from merge
- Recover RHAAP Controller
lint_new_additions:
tags:
- AutomRunner
stage: lint
rules:
- if: '$CI_COMMIT_BRANCH != "dev"
&& $CI_COMMIT_BRANCH != "test"
&& $CI_COMMIT_BRANCH != "accp"
&& $CI_COMMIT_BRANCH != "prod"
&& $CI_PIPELINE_SOURCE == "push"'
script:
- echo "Start linting on '$CI_COMMIT_BRANCH'"
- ansible-lint
--exclude host_vars/
# Normal stage when a merge request is done..
configure_rhaap_controller_from_merge_request:
tags:
- AutomRunner
stage: Configure RHAAP Controller from merge
rules:
- if: '$CI_COMMIT_BRANCH == "dev"
&& $CI_PIPELINE_SOURCE == "push"
&& $CI_COMMIT_MESSAGE =~ /Merge branch/i'
- if: '$CI_COMMIT_BRANCH == "test"
&& $CI_PIPELINE_SOURCE == "push"
&& $CI_COMMIT_MESSAGE =~ /Merge branch/i'
- if: '$CI_COMMIT_BRANCH == "accp"
&& $CI_PIPELINE_SOURCE == "push"
&& $CI_COMMIT_MESSAGE =~ /Merge branch/i'
- if: '$CI_COMMIT_BRANCH == "prod"
&& $CI_PIPELINE_SOURCE == "push"
&& $CI_COMMIT_MESSAGE =~ /Merge branch/i'
script:
- echo "Perform merge to '$CI_COMMIT_BRANCH' Environment"
- TOKEN_VAR=$(echo "AUTOMATION_HUB_TOKEN_${CI_COMMIT_BRANCH}" | tr
[:lower:] [:upper:])
- ansible-playbook main.yml
-i inventory.yaml
-e instance=controller_$CI_COMMIT_BRANCH
-e ahub_token=$(printenv $TOKEN_VAR)
-e branch_name=$CI_COMMIT_BRANCH
--vault-password-file <(echo ${VAULT_PASSWORD})
# This runs in case of a trigger from another project ( like recovery
process )
configure_controller_from_trigger:
tags:
- AutomRunner
stage: Recover RHAAP Controller
rules:
- if: '$CI_PIPELINE_SOURCE == "pipeline"'
script:
- echo "Pipeline triggered by '$CI_PIPELINE_SOURCE' ref"
- TOKEN_VAR=$(echo "AUTOMATION_HUB_TOKEN_${CI_COMMIT_REF_NAME}" | tr
[:lower:] [:upper:])
- echo "From pipeline - Start controller recovery on
'$CI_COMMIT_REF_NAME' Environment"
- ansible-playbook main.yml
-i inventory.yaml
-e instance=controller_$CI_COMMIT_REF_NAME
-e branch_name=$CI_COMMIT_REF_NAME
-e ahub_token=$(printenv $TOKEN_VAR)
--vault-password-file <(echo ${VAULT_PASSWORD})
Again, the pipeline is no different from a standard pipeline that we have seen and
discussed many times in other chapters. We are not going to repeat this here.
templates/inventory.yaml.j2
To run a playbook from a pipeline, we need an inventory, which we create by calling this template.
---
dev:
hosts: controller_dev
test:
hosts: controller_test
accp:
hosts: controller_accp
Prod:
hosts: controller_prod
As you can see, this is not really a template, because everything is already filled in and not really variable. Note: The environments in this file MUST match you environments! templates/credentials.yml.j2 We'll start with a special one, normally quite a few credentials would have to be created in this. We're going to do that too, but through the secure group loop, so they'll be encrypted, so in the template we'll just write the header of the file.
---
{% if env_vars.credentials | length > 0 %}
controller_credentials_{{ curr_env }}:
{% else %}
controller_credentials_{{ curr_env }}: []
{% endif %}
templates/inventory.yml.j2
---
{% if env_vars.inventories | length > 0 %}
controller_inventories_{{ curr_env }}:
{% for item in env_vars.inventories %}
- name: {{ item.name }}
description: {{ item.description }}
organization: {{ organization_long_name }}
{% endfor %}
{% else %}
controller_inventories_{{ curr_env }}: []
{% endif %}
{% if env_vars.inventory_sources | length > 0 %}
controller_inventory_sources_{{ curr_env }}:
{% for item in env_vars.inventory_sources %}
- name: {{ item.name }}
description: {{ item.description }}
organization: {{ organization_long_name }}
source: scm
source_project: {{ organization_short_name }}_demo_inventory
source_path: hosts.ini
inventory: {{ organization_short_name }}_demo_inventory
update_on_launch: true
overwrite: true
{% endfor %}
{% else %}
controller_inventory_sources_{{ curr_env }}: []
{% endif %}
...
templates/notifications.yml.j2
---
controller_notifications_{{ curr_env }}: []
...
templates/organization.yml.j2
---
{% if env_vars.organizations | length > 0 %}
controller_organizations_{{ curr_env }}:
{% for item in env_vars.organizations %}
- name: {{ organization_long_name }}
description: {{ item.description }}
galaxy_credentials:
- {{ organization_short_name }}_automation_hub_token_rh_certified
- {{ organization_short_name }}_automation_hub_token_community
- {{ organization_short_name }}_automation_hub_token_published
assign_galaxy_credentials_to_org: true
{% endfor %}
{% else %}
controller_organizations_{{ curr_env }}: []
{% endif %}
...
templates/projects.yml.j2
---
{% if env_vars.projects | length > 0 %}
controller_projects_{{ curr_env }}:
{% for item in env_vars.projects %}
- name: {{ item.name }}
description: {{ item.description }}
organization: {{ organization_long_name }}
scm_type: git
{% if item.scm_url is defined %}
scm_url: {{ item.scm_url }}
{% endif %}
scm_credential: {{ organization_short_name }}_gitlab
scm_branch: master
scm_clean: false
scm_delete_on_update: false
scm_update_on_launch: true
scm_update_cache_timeout: 0
allow_override: false
timeout: 0
{% endfor %}
{% else %}
controller_projects_{{ curr_env }}: []
{% endif %}
...
templates/roles.yml.j2
---
{% if env_vars.roles | length > 0 %}
controller_roles_{{ curr_env }}:
{% for item in env_vars.roles %}
- team: {{ item.team }}
job_templates:
- {{ organization_short_name }}_demo_os_base
role: execute
- team: {{ item.team }}
credentials:
- {{ organization_short_name }}_gitlab
- {{ organization_short_name }}_ansible
- {{ organization_short_name }}_automation_hub_token_rh_certified
- {{ organization_short_name }}_automation_hub_token_community
- {{ organization_short_name }}_automation_hub_token_published
role: use
- team: {{ item.team }}
inventory: {{ organization_short_name }}_demo_inventory
role: use
- team: {{ item.team }}
projects:
- {{ organization_short_name }}_demo_os_base
- {{ organization_short_name }}_demo_inventory
role: use
{% endfor %}
{% else %}
controller_roles_{{ curr_env }}: []
{% endif %}
...
templates/schedules.yml.j2
---
controller_schedules_{{ curr_env }}: []
...
templates/teams.yml.j2
---
{% if env_vars.teams | length > 0 %}
controller_teams_{{ curr_env }}:
{% for item in env_vars.teams %}
- name: LDAP_{{ organization_short_name }}_Admins
description: Admin users
organization: {{ organization_long_name }}
- name: LDAP_{{ organization_short_name }}_Developers
description: Development users
organization: {{ organization_long_name }}
- name: LDAP_{{ organization_short_name }}_Operators
description: Operator users
organization: {{ organization_long_name }}
{% endfor %}
{% else %}
controller_teams_{{ curr_env }}: []
{% endif %}
...
templates/templates.yml.j2
---
{% if env_vars.templates | length > 0 %}
controller_templates_{{ curr_env }}:
{% for item in env_vars.templates %}
- name: {{ organization_short_name }}_demo_os_base
description:
organization: {{ organization_long_name }}
project: {{ organization_short_name }}_demo_os_base
inventory: {{ organization_short_name }}_demo_inventory
playbook: main.yml
job_type: run
fact_caching_enabled: false
credentials:
- {{ organization_short_name }}_ansible
concurrent_jobs_enabled: false
ask_scm_branch_on_launch: false
ask_tags_on_launch: false
ask_verbosity_on_launch: false
ask_variables_on_launch: false
extra_vars:
execution_environment: Default execution environment
survey_enabled: false
survey_spec: {}
{% endfor %}
{% else %}
controller_templates_{{ curr_env }}: []
{% endif %}
...
templates/workflows.yml.j2
---
controller_workflows_{{ curr_env }}: []
...