Automated Installation of Openshift Origin using Ansible

Redhat provided Ansible Playbook to automate installation of Openshift, but the installation is not straight forward and lack of documentation available for successful installation and working version of it.

In this article, I will go through the Containerized Installation of Openshift Origin Version 3.9 (which was latest at the time of writing). I have used vmware workstation to build 3 CentOS 7.5 hosts, 1 master and two nodes.

Step 1: Build a BIND dns server using Ansible Role ( https://galaxy.ansible.com/bertvv/bind/)

Step2: Configure the primary interface (e.g. eth0) of the openshift hosts (both master and nodes) to use it’s own ip as primary name server and the dns server above as secondary name server. Also change the search order for the domain, cluster.local is mandatory, replace vm.local with your own domain

e.g

nmcli con mod ens33 ipv4.domain “cluster.local vm.local”

nmcli con mod ens33 ipv4.dns “192.168.23.30 192.168.23.16”

Here ens33 is the primary interface name of the host, 192.168.23.30 is the master ip and 192.168.23.16 is the bind dns ip

Step3: Enable the primary interface of the openshift hosts to be Network Manager Controlled and enable PEERDNS -

e.g.

grep -q -F ‘NM_CONTROLED=yes’ /etc/sysconfig/network-scripts/ifcfg-ens33 || (echo ‘NM_CONTROLED=yes’ | sudo tee -a /etc/sysconfig/network-scripts/ifcfg-ens33)

grep -q -F ‘PEERDNS=yes’ /etc/sysconfig/network-scripts/ifcfg-ens33 || (echo ‘PEERDNS=yes’ | sudo tee -a /etc/sysconfig/network-scripts/ifcfg-ens33)

Step4: Clone the Openshift Ansible repository and change to use release 3.9

git clone https://github.com/openshift/openshift-ansible.git
git checkout release-3.9

Step5: Run Ansible Playbook to configure the Openshift hosts, after cloning my repo below and change the hosts name in the Inventory according to your hostname or ip -

controller ansible_ssh_host=oc-master.vm.local

node1 ansible_ssh_host=oc-node1.vm.local

node2 ansible_ssh_host=oc-node2.vm.local

Step6: Run the playbook —

ansible-playbook -i inventory playbook.yml -vv

playbook.yml contains the following -

---
- import_playbook: ../openshift-ansible/playbooks/prerequisites.yml
- import_playbook: ../openshift-ansible/playbooks/deploy_cluster.yml

Step7: Wait for the playbook to show failure, this is due to 99-origin-dns.sh failure to configure dns accordingly. Follow the steps below and run the playbook again as in Step 6:

a. sudo cp /etc/resolv.conf /etc/origin/node/resolv.conf

b. “echo server=192.168.23.16” | sudo tee -a /etc/dnsmasq.d/origin-upstream-dns.conf

c. sudo systemctl restart dnsmasqd

Replace 192.168.23.16 with your own BIND dns server ip created in step 1

WebConsole

After the installation is complete the webconsole can be accessed from -

https://oc-master.vm.local:8443

Replace your master domain with oc-master above. The default cred for login is system/admin.

Explanation of Step7 -

Openshift Origin from 3.6 uses skydns for its internal name resolution and run on ip 127.0.0.1:53, dnsmasq run on node ip e.g in master node above it is running on 192.168.23.30:53 and forwards all requests of cluster.local to skydns which it forwards to bind dns server that we configured in step 1.

Therefore the flow is -

dnsmasq -> skydns -> bind(external dns)

Every node requires dns search order setup in Step2 to resolve internal services e.g. registry.

Uninstallation

To uninstall, run the following from the inventory repo -

ansible-playbook -i inventory \
../openshift-ansible/playbooks/adhoc/uninstall.yml -vv