Introduction
OKD is the upstream and community-supported version of the Red Hat OpenShift Container Platform (OCP). OpenShift expands Kubernetes on an Enterprise level
NSX Advanced load balancer (AVI) is a 100% software-defined multi-cloud application services platform with Software Load Balancers, Intelligent WAF (iWAF), and Container Ingress.
Ansible is a radically simple IT automation tool. It handles configuration management, application deployment, cloud provisioning, ad-hoc task execution, network automation, and multi-node orchestration. Ansible makes complex changes like zero-downtime rolling updates with load balancers easy.
All modern applications use load balancing nowadays. Load balancing is the process of distributing network traffic across multiple servers.
In this blog, I am using three control-plane servers and two worker-nodes, since I am using multiple servers, it makes sense to have a load balancer in front.
AKO (Avi Kubernetes Operator) can be installed to be used to configure routes and services of type Loadbalancer. (Not covered in this blog. Click here on how to install AKO on OpenShift.)
The goal of this blog
I used a very good guide to install OKD 4 by Craig Robinson. You can find it here.
Craig uses HA-PROXY to do the load balancing.
My goal is to use AVI instead of HA-PROXY.
I am going to use Ansible to make the AVI config. (Click here on how to use AVI ansible collections)
I will not go very deep on how to install AVI, OKD, or Ansible
An overview of the IPs and Hostnames I am using
| Ip address | Host |
| 100.64.35.99 | okd4-bootstrap |
| 100.64.35.100 | cm-master01 |
| 100.64.35.101 | cm-master02 |
| 100.64.35.102 | cm-master03 |
| 100.64.35.103 | cm-worker01 |
| 100.64.35.104 | cm-worker02 |
Installing
Ansible will be using AVI ansible collection. (click here on how to install)
1. Download git files
$ git clone https://github.com/chrismentjox/ako-ansible.git
Cloning into 'ako-ansible'...
remote: Enumerating objects: 9, done.
remote: Counting objects: 100% (9/9), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 9 (delta 3), reused 9 (delta 3), pack-reused 0
Unpacking objects: 100% (9/9), 2.40 KiB | 1.20 MiB/s, done.
2. Change vars
You need to change the vars to match your setup.
$ vi build_lb_vars.ymlIf you are in the progress of bootstrapping OKD, it is important to remove “enabled”: “false” in the different pool members. When you are done bootstrapping add “enabled”: “false” again to the pool members
3. Running ansible-playbook
$ ansible-playbook deploy.yml
PLAY [localhost] ************************************************************************************************************************************************************************************************
TASK [Gathering Facts] ******************************************************************************************************************************************************************************************
ok: [localhost]
TASK [Create local load balancing pool] *************************************************************************************************************************************************************************
changed: [localhost] => (item={'pool_name': 'okd4_https_ingress_traffic_be', 'healthmonitor_name': 'System-TCP', 'default_server_port': 443, 'pool_members': [{'ip': {'addr': '100.64.35.103', 'type': 'V4'}, 'hostname': 'cm-worker01'}, {'ip': {'addr': '100.64.35.104', 'type': 'V4'}, 'hostname': 'cm-worker02'}]})
changed: [localhost] => (item={'pool_name': 'okd4_http_ingress_traffic_be', 'healthmonitor_name': 'System-TCP', 'default_server_port': 80, 'pool_members': [{'ip': {'addr': '100.64.35.103', 'type': 'V4'}, 'hostname': 'cm-worker01'}, {'ip': {'addr': '100.64.35.104', 'type': 'V4'}, 'hostname': 'cm-worker02'}]})
changed: [localhost] => (item={'pool_name': 'okd4_k8s_api_be', 'healthmonitor_name': 'System-TCP', 'default_server_port': 6443, 'pool_members': [{'ip': {'addr': '100.64.35.99', 'type': 'V4'}, 'hostname': 'okd4-bootstrap', 'enabled': 'false'}, {'ip': {'addr': '100.64.35.100', 'type': 'V4'}, 'hostname': 'cm-master01'}, {'ip': {'addr': '100.64.35.101', 'type': 'V4'}, 'hostname': 'cm-master02'}, {'ip': {'addr': '100.64.35.102', 'type': 'V4'}, 'hostname': 'cm-master03'}]})
changed: [localhost] => (item={'pool_name': 'okd4_machine_config_server_be', 'healthmonitor_name': 'System-TCP', 'default_server_port': 22623, 'pool_members': [{'ip': {'addr': '100.64.35.99', 'type': 'V4'}, 'hostname': 'okd4-bootstrap', 'enabled': 'false'}, {'ip': {'addr': '100.64.35.100', 'type': 'V4'}, 'hostname': 'cm-master01'}, {'ip': {'addr': '100.64.35.101', 'type': 'V4'}, 'hostname': 'cm-master02'}, {'ip': {'addr': '100.64.35.102', 'type': 'V4'}, 'hostname': 'cm-master03'}]})
PLAY [localhost] ************************************************************************************************************************************************************************************************
TASK [Gathering Facts] ******************************************************************************************************************************************************************************************
ok: [localhost]
TASK [Create vsvip] *********************************************************************************************************************************************************************************************
changed: [localhost] => (item={'vsvip_name': 'okd4-vip', 'vs_name': 'okd4_http_ingress_traffic_fe', 'pool_name': 'okd4_http_ingress_traffic_be', 'service_ports': [{'port': '80'}]})
changed: [localhost] => (item={'vsvip_name': 'okd4-vip', 'vs_name': 'okd4_https_ingress_traffic_fe', 'pool_name': 'okd4_https_ingress_traffic_be', 'service_ports': [{'port': '443'}]})
changed: [localhost] => (item={'vsvip_name': 'okd4-vip', 'vs_name': 'okd4_k8s_api_fe', 'pool_name': 'okd4_k8s_api_be', 'service_ports': [{'port': '6443'}]})
changed: [localhost] => (item={'vsvip_name': 'okd4-vip', 'vs_name': 'okd4_machine_config_server_fe', 'pool_name': 'okd4_machine_config_server_be', 'service_ports': [{'port': '22623'}]})
PLAY [localhost] ************************************************************************************************************************************************************************************************
TASK [Gathering Facts] ******************************************************************************************************************************************************************************************
ok: [localhost]
TASK [Create virtualservice] ************************************************************************************************************************************************************************************
changed: [localhost] => (item={'vsvip_name': 'okd4-vip', 'vs_name': 'okd4_http_ingress_traffic_fe', 'pool_name': 'okd4_http_ingress_traffic_be', 'service_ports': [{'port': '80'}]})
changed: [localhost] => (item={'vsvip_name': 'okd4-vip', 'vs_name': 'okd4_https_ingress_traffic_fe', 'pool_name': 'okd4_https_ingress_traffic_be', 'service_ports': [{'port': '443'}]})
changed: [localhost] => (item={'vsvip_name': 'okd4-vip', 'vs_name': 'okd4_k8s_api_fe', 'pool_name': 'okd4_k8s_api_be', 'service_ports': [{'port': '6443'}]})
changed: [localhost] => (item={'vsvip_name': 'okd4-vip', 'vs_name': 'okd4_machine_config_server_fe', 'pool_name': 'okd4_machine_config_server_be', 'service_ports': [{'port': '22623'}]})
PLAY RECAP ******************************************************************************************************************************************************************************************************
localhost : ok=6 changed=3 unreachable=0 failed=0 skipped=0 rescued=0 ignored=04. Check the VIPs
5. Change DNS
Do not forget to change DNS to the correct IP address.
In my setup, I am using “okd.lab.fqdn.nl” as a domain.
The following domains need to be changed to the new IP:
console-openshift-console.apps.okd.lab.fqdn.nl
oauth-openshift.apps.okd.lab.fqdn.nl
api.okd.lab.fqdn.nl
api-int.okd.lab.fqdn.nl
You can find the AVI assigned IP in the GUI
6. Happy OKD
Going to my console URL https://console-openshift-console.apps.okd.lab.fqdn.nl shows me the dashboard.
Conclusion
That concludes this short blog.
AVI and automation make it very easy to set up load balancing.
If you have any questions or comments, let me know.