remote-lab.net learn by doing

Deploying all-in-one OKD 3.11 with Let's Encrypt SSL certificates

15 Jan 2019

I’ve been experimenting with OpenShift lately and in the following post I’d like to document the steps required to get an OKD 3.11 environment up and running. For those who are not familiar with the OKD abbreviation it is “The Origin Community Distribution of Kubernetes that powers Red Hat OpenShift”. The deployment is going to be set up on a single node and configured to use Let’s Encrypt SSL certificates for the API endpoint/console and HTTPS routes.

The main purpose of this exercise is to:

  • get myself familiar with the openshift-ansible installer process
  • get an OpenShift environment connected to the Internet up and running that I can experiment with

Let’s get started.

Prerequisites

Hardware

** Get a beefy CentOS 7 VM ready. For this test I used one with 4 vCPUs, 16 GB of RAM and 50GB of storage. The specs could be lowered but you may need to disable specific checks the installer runs.

DNS configuration

** I’m going to use ‘containers.remote-lab.net’ as the domain name for this OpenShift installation. Consequently I’ve got to set up the following DNS records in my domain’s zone file to be able to reach the environment from any client. The wildcard mask entry is required for the OpenShift routes:

containers.remote-lab.net.	   IN	A	148.251.139.92
*.containers.remote-lab.net.       IN	A	148.251.139.92

Prepare

  • Set up the hostname
## SSH to the CentOS VM
[[email protected] ~]$ sudo hostnamectl set-hostname containers.remote-lab.net
[[email protected] ~]$ sudo hostnamectl set-hostname containers.remote-lab.net --transient
[[email protected] ~]$ hosts=$(echo "$(ip a s dev eth0 | awk '/inet / {split($2, ary, /\//); print ary[1]}') $(hostname -f) $(hostname -s)")
[[email protected] ~]$ sudo sh -c "echo $hosts >> /etc/hosts"
  • Set up the OKD 3.11 repos
[[email protected] ~]$ sudo sh -c 'cat > /etc/yum.repos.d/openshift.repo << EOF
[openshift-3.11]
name=OpenShift Origin 3.11
baseurl=https://cbs.centos.org/repos/paas7-openshift-origin311-candidate/\$basearch//os/
enabled=1
gpgcheck=0

[openshift-common]
name=OpenShift Common
baseurl=https://cbs.centos.org/repos/paas7-openshift-common-candidate/\$basearch//os/
enabled=1
gpgcheck=0
EOF'
  • Install openshift-ansible and enable NetworkManager
[[email protected] ~]$ sudo yum install -y openshift-ansible NetworkManager pyOpenSSL
[[email protected] ~]$ sudo systemctl start NetworkManager
[[email protected] ~]$ sudo systemctl enable NetworkManager
  • Fix small dependency issue if still necessary
[[email protected] ~]$ curl -L https://github.com/openshift/openshift-ansible/pull/10812/commits/0d7608fedc522a8d034191c4e07d880b82e07006.diff | sudo patch -d /usr/share/ansible/openshift-ansible/ -p1
  • Generate Let’s Encrypt certificates. I’m using Cloudflare for hosting my domain DNS zone so the example below will call the Cloudflare script
[[email protected] ~]$ sudo yum install -y git
[[email protected] ~]$ git clone https://github.com/Neilpang/acme.sh.git
[[email protected] ~]$ cd acme.sh/
## set your API key and email address in dnsapi/dns_cf.sh
[[email protected] acme.sh]$ vi dnsapi/dns_cf.sh
[[email protected] acme.sh]$ ./acme.sh --issue -d containers.remote-lab.net -d *.containers.remote-lab.net --dns dns_cf
## copy certificate and key that got created in the previous step
[[email protected] acme.sh]$ sudo cp /home/centos/.acme.sh/containers.remote-lab.net/containers.remote-lab.net.cer /etc/pki/tls/certs/containers.remote-lab.net.cer
[[email protected] acme.sh]$ sudo cp /home/centos/.acme.sh/containers.remote-lab.net/containers.remote-lab.net.key /etc/pki/tls/private/containers.remote-lab.net.key
[[email protected] acme.sh]$ sudo cp /home/centos/.acme.sh/containers.remote-lab.net/ca.cer /etc/pki/tls/certs/containers.remote-lab.net.ca.cer
  • Create Inventory file
[[email protected] ~]$ cat okd-inventory

[OSEv3:children]
masters
nodes
etcd

[OSEv3:vars]
ansible_connection=local
openshift_deployment_type=origin

openshift_master_identity_providers=[{ 'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider'}]
openshift_master_htpasswd_users={'marius': '$apr1$fQGIonSg$Y6vvC8Q4Lowzp0EQl.mlE1'}

openshift_master_cluster_hostname=containers.remote-lab.net
openshift_master_cluster_public_hostname=containers.remote-lab.net
openshift_master_default_subdomain=containers.remote-lab.net

openshift_master_overwrite_named_certificates=true
openshift_master_named_certificates=[{"certfile": "/etc/pki/tls/certs/containers.remote-lab.net.cer", "keyfile": "/etc/pki/tls/private/containers.remote-lab.net.key", "names": ["containers.remote-lab.net"], "cafile": "/etc/pki/tls/certs/containers.remote-lab.net.ca.cer"}]

openshift_cluster_monitoring_operator_install=false
openshift_metrics_install_metrics=false
ansible_service_broker_install=false

[masters]
containers.remote-lab.net

[etcd]
containers.remote-lab.net

[nodes]
containers.remote-lab.net openshift_node_group_name='node-config-all-in-one'

[oo_all_hosts:children]
masters
nodes
etcd

Deploy!

  • Run prerequisites playbook
[[email protected] ~]$ sudo ansible-playbook -i okd-inventory /usr/share/ansible/openshift-ansible/playbooks/prerequisites.yml
  • Run deploy playbook
[[email protected] ~]$ sudo ansible-playbook -i okd-inventory /usr/share/ansible/openshift-ansible/playbooks/deploy_cluster.yml
  • After running the deploy_cluster playbook the OpenShift setup should be up and running and reachable from the Internet.
[[email protected] ~]$ sudo oc status
In project default on server https://containers.remote-lab.net:8443

https://docker-registry-default.containers.remote-lab.net (passthrough) (svc/docker-registry)
  dc/docker-registry deploys docker.io/openshift/origin-docker-registry:v3.11.0 
    deployment #2 deployed 14 minutes ago - 1 pod
    deployment #1 deployed 26 minutes ago

svc/kubernetes - 172.30.0.1 ports 443->8443, 53->8053, 53->8053

https://registry-console-default.containers.remote-lab.net (passthrough) (svc/registry-console)
  dc/registry-console deploys docker.io/cockpit/kubernetes:latest 
    deployment #1 deployed 26 minutes ago - 1 pod

svc/router - 172.30.42.93 ports 80, 443, 1936
  dc/router deploys docker.io/openshift/origin-haproxy-router:v3.11.0 
    deployment #2 deployed 15 minutes ago - 1 pod
    deployment #1 deployed 26 minutes ago

View details with 'oc describe <resource>/<name>' or list everything with 'oc get all'.
  • There is one more step that we have to do post deployment. This replaces the current router certficate with the Let’s Encrypt certificate that we created earlier.
[[email protected] ~]$ cat /etc/pki/tls/certs/containers.remote-lab.net.cer /etc/pki/tls/private/containers.remote-lab.net.key /etc/pki/tls/certs/containers.remote-lab.net.ca.cer > containers.remote-lab.net.pem
[[email protected] ~]$ sudo sh -c "oc secrets new router-certs tls.crt=/home/centos/containers.remote-lab.net.pem tls.key=/etc/pki/tls/private/containers.remote-lab.net.key -o json --type='kubernetes.io/tls' --confirm | oc replace -f -"
[[email protected] ~]$ sudo oc delete pod $(sudo oc get pods | awk '/router/ {print $1}')

Enjoy!

The OpenShift console can now be reached @ https://containers.remote-lab.net:8443.

Credit goes to Marcos Entenza for sharing the Let’s Encrypt certificate instructions.

Related Posts

comments powered by Disqus