Creating a workflow over multiple organizations
As the configuration is now separated in multiple organizations, we have teams that are in full controll of their credentials, projects, templates and workflows.
So each team can perform their actions automated by running a job_template or workflow. This is the start of a fully automated deployment, but we want more.
I will describe the workflow creation for two teams:
- Infrastructure team (creating VM's)
- Linux team (configure the VM and Patching)
Infrastructure team
First the Infrastructure team has created a project to automate the deployment of virtual machines. This project is used on a job_template with the following definition in the configurationas code for the INFRA organization.
- name: INFRA_vm_create
description: Create and register VM's
organization: ORG_INFRA
project: INFRA_infra
inventory: Default_inventory
playbook: main.yml
job_type: run
fact_caching_enabled: false
credentials:
- INFRA_ansible
- INFRA_vault_deploy
concurrent_jobs_enabled: false
ask_scm_branch_on_launch: false
ask_tags_on_launch: false
ask_verbosity_on_launch: false
ask_variables_on_launch: true
extra_vars:
instances: host or group
execution_environment: ee-vm-handling
survey_enabled: false
survey_spec: {}
The definition of this template is quite simple and working for the Infrastructure team. As they pass the hostname for the new VM, the creation starts.
For the record, the specs for this host (like memory and disks are in the host_vars file in the inventory, not shown here).
Linux team
The Linux team has created 2 separate projects:
- OS configuration (configure NTP, sshd, selunix, users, etc)
- OS patching (apply the latest patches)
These are defined in their config as code repository as follows:
- name: LNX_configure_OS
description: Configure OS settings
organization: ORG_LNX
project: LNX_rhel
inventory: Default_inventory
playbook: main.yml
job_type: run
fact_caching_enabled: false
credentials:
- LNX_ansible
- LNX_vault_deploy
concurrent_jobs_enabled: false
ask_scm_branch_on_launch: false
ask_tags_on_launch: false
ask_verbosity_on_launch: false
ask_variables_on_launch: true
extra_vars:
instances: host or group
execution_environment: Default execution environment
survey_enabled: false
survey_spec: {}
- name: LNX_Patch_OS
description:
organization: ORG_LNX
project: LNX_maintenance
inventory: Default_inventory
playbook: run_dnf_update.yml
job_type: run
fact_caching_enabled: false
credentials:
- LNX_ansible
concurrent_jobs_enabled: false
ask_scm_branch_on_launch: false
ask_tags_on_launch: false
ask_verbosity_on_launch: false
ask_variables_on_launch: false
extra_vars:
instances: host or group
execution_environment: Default execution environment
survey_enabled: false
survey_spec: {}
ALso these templates work for the Linux team and can be run separately to do their stuff.
For the patching template they have implemented a schedule to keep all systems up-to-date, according company policy.
If you want to create a workflow using all these templates, as rhaap is configured at the moment, this will not work. We need to change some thing to make this work.
To make things a little clearer, I have created a separate organization in rhaap, with its own users and organization repository, called ORG_WFL.
We will configure the access rights needed to create the workflow step-by-step.
As each team (INFRA and LNX) is in control of what they create and share, they must configure the job_templates to share for 'use' and to who they give this right, conforming to RBAC.
Infrastructure access for ORG_WFL
The infrastructure team updates the following file in the configuration as code repository they own to give the WFL admins team access to the INFRA_vm_create job template:
/group_vars/{env}/controller_roles:
- job_templates:
- INFRA_vm_create
teams:
- LDAP_WFL_Admins
- LDAP_WFL_Developers
- LDAP_WFL_Operators
roles:
- execute
- organization: ORG_INFRA
teams:
- LDAP_WFL_Admins
roles:
- read
As you can see here, there are 2 items to be added, this is what they do:
The job_templates variable gives the teams in the WFL organization the right to execute the job_template, but they cannot read or see it. So if they want to create a workflow using this job_template, they wouldn't find it and it will fail.
This is where the second access right comes in, this gives only the admins for the WFL organization the right to see things in the INFRA organization, so they can find the job_template and use it to create a workflow, that the other team in the organization can use.
LNX access for ORG_WFL
For the Linux team, almost the exact same changes must be done to accomplish the sharing of the 2 job_templates they made:
/group_vars/{env}/controller_roles:
- job_templates:
- LNX_configure_OS
- LNX_Patch_OS
teams:
- LDAP_WFL_Admins
- LDAP_WFL_Developers
- LDAP_WFL_Operators
roles:
- execute
- organization: ORG_LNX
teams:
- LDAP_WFL_Admins
roles:
- read
As the pipelines are run to configure the new options into rhaap the access rights as specified are granted.
WFL workflow definition
As you can see in the examples above, to accomplish this over many team/organizations requires a strict naming convention, so it is clear and deductable what needs to be configured.
The workflow definition for the WFL organization using all three job_templates, would look like this:
controller_workflows_dev:
- name: Deployment workflow
description: A basic workflow for system deployment
extra_vars:
instances: host or group
allow_simultaneous: true
ask_variables_on_launch: true
inventory: Default_inventory
limit:
scm_branch: master
ask_inventory_on_launch: false
ask_scm_branch_on_launch: false
ask_limit_on_launch: false
webhook_service: ''
webhook_credential:
organization:
name: ORG_WFL
workflow_nodes:
- all_parents_must_converge: false
identifier: INFRA-000
unified_job_template:
name: INFRA_vm_create
type: job_template
organization:
name: ORG_INFRA
related:
success_nodes:
- workflow_job_template:
name: Deployment workflow
identifier: OS-config
- all_parents_must_converge: false
identifier: OS-config
unified_job_template:
name: LNX_configure_OS
type: job_template
organization:
name: ORG_LNX
related:
success_nodes:
- workflow_job_template:
name: Deployment workflow
identifier: OS-patch
- all_parents_must_converge: false
identifier: OS-patch
unified_job_template:
name: LNX_Patch_OS
type: job_template
organization:
name: ORG_LNX
survey_enabled: false
survey_spec: {}
If the access rights are configured correctly, the workflow is created successfully and can be started by the users is the WFL organization. Full controll of each job_template and used credentials in these templates, stays in the teams that exposed the templates.
Also the projects are not shared and version management of all content is still under the controll of the originating team.
This way each team is autonomous, but the organization can make use of the products they create and share.
This example can be extended/changed as your organization needs are different than a lab setup.
Going further than only the OS installation
For most organizations it will not end with the installation of the OS itself, an application will not install itself if the server is availlable. So the chain must be extended from here, we can do this in various ways:
- create multiple, specific workflows one per application
- create separate playbooks/job_templates (we need them anyway)
- extend the above workflow in a flexible way to read the next step from the host_vars
The creation of multiple workflows is just a matter of copy/paste the above workflow and add the steps that are needed and save this as a new name.
The playbooks/job_templates to install the applications, we need anyway, so that would be the first step to full automation.
When the job templates for the installation of the applications are ready and have been entered into rhaap, we can start extending the workflow.
For this example we assume the WEB team has created a playbook/job_template in their organization with the name: WEB_apache and the MON(intoring) team has done the same for their monitoring tool and has created the MON_agent template. These templates are shared with the WFL organization, as are the INFRA ans LNX templates.
The workflow playbook
In order to extend the deployment workflow dynamicly, we create a simple playbook that will read the vars from the hostvars and act on these through itteration.
The playbook below is added in the deployment workflow as the last step in the flow.
This will check the hostvars of the machine for the definition of the buidingblocks variable.
And if this variable is not found, will end the play, otherwise it wil start itterating through the list.
main.yml
---
- name: Run buildingblock jobs
hosts: "{{ instances |default('dummy') }}"
gather_facts: false
pre_tasks:
- name: Stop play if nothing to do
ansible.builtin.meta: end_play
when: buildingblocks is not defined
delegate_to: localhost
tasks:
# Start the job template in a separate file, because we want to fail the
# play if the job fails.
- name: Loop over the defined steps
ansible.builtin.include_tasks:
file: job_launch.yml
args:
apply:
delegate_to: localhost
loop: "{{ buildingblocks }}"
loop_control:
loop_var: template_name
when: buildingblocks is defined
For every item in the list it wil import the tasks below and run these, what will result in an API call to rhaap, starting the job_template in the configured organization.
job_launch.yml
- name: Debug launch params
ansible.builtin.debug:
var: template_name
- name: Launch job template "{{ template_name }}"
ansible.builtin.include_role:
name: infra.aap_configuration.controller_job_launch
vars:
controller_launch_jobs:
- name: "{{ template_name['name'] }}"
organization: "{{ template_name['organization'] }}"
extra_vars:
instances: "{{ inventory_hostname }}"
buildingblock: "{{ template_name }}"
wait: true
register: job
The hostvars
The hostvars definition holds the key to the installation of products on the system:
host_name: webserver.homelab
ip_address: 10.1.2.12
cpu_cores: 2
disk_size: 4
memory_size: 1024
swap_size: 256
id: 1331
type: qemu
proxmox_node: proxmox02
os:
name: LNX_rhel
version: master
buildingblocks:
- name: WEB_apache
organization: ORG_WEB
- name: MON_agent
organization: ORG_MON
In this case, there will be 2 products installed, directly after the machine configuration:
- an apache webserver
- a monitoring agent
And nobody touches the system to do anything, this is really Look Mammy, No hands!
The forementioned example is ofcourse just that, an example, how you might implement this, is entirely up to you.