I'd like to share my story about migration an application to Openshift. Also, as a result, I will compare some of the most popular solutions and tools for managing your application inside Openshift. It is the transcription of my presentation at kubernetes SPB meetup #3.

Let's deploy to Openshift

What should we do?

First of all, let's talk about our application. It is out of box enterprise solution, it supports different databases, application servers and integration interfaces with third-party systems. Usually, our clients were installing on dedicated servers, however, we faced the issue. We had to tun the application inside Openshift.

Prerequisites

The application is the product with a long history, it should work out of the box in completely different environments. As a result, there are a lot of page in our installations guides. However, the top level schema is easy as pie, you just should:

• Apply DB schema.
• Prepare application server configuration.
• Install your license.
• Initialize the application.

Unfortunately, the world is cruel, there were some important prerequisites.

• We could build the application only at a special Jenkins slave, because of security restrictions
• There was no access from the client's Openshift installation to the private developers' docker registry.
• We were not able to re-use existing docker images, because they were created for developing & testing needs only.
• There were ansible playbooks for application deployment on VMs.

Ansible-container demo

Ansible Container is an open source project that aims to enable the automation of the entire container build, deployment and management process. Best of all, it uses the same simple, powerful and agentless Ansible automation language that you're already using, ensuring you can automate the entire application lifecycle.

We already had written some Ansible roles for installing the application at VMs, so we reused them with ansible-container. Ansible container is a toolset for building containers. I’m not sure that it’s really good toolset, however it allows:

• Create containers the same way we deploy our servers.
• Stop chaining together shell commands in Dockerfiles.

There was no major issue with ansible-container because Openshift creating images guidlines is awesome. However, I'd like to notice:

• We modified our ansible roles, because Docker + systemd = pain.
• There is no ability to use chroot, sudo inside Openshift by default, because of security. Just read CVE-2019-5736.
• For security reasons Openshift be default also generate random UID for each container, in other words openshift ignores the USER option from a Dockerfile.

The main point is that ansible-container helped us to create a demo very fast, because of reusing.

Multiple containers demo

The first demo container was built via ansible-container. It was good enough for the demo, however, we decided not to use it. We split the monolith container into different:

1. We used the original Openshift PostgreSQL container without any modifications.
2. We built the application stateless container.

However, it wasn't clear to initialize the database? We found a great article about PODs life inside kubernetes. So, we decided to use init container for database initialization.

Initialize the application

As I mentioned before, the application should work out of the box in completely different environments, support different application servers/databases and integration interfaces with third-party systems.

There are a lot of ways to initialize the application:

1. Pass configuration via environment variables. This means add all our documentation/knowledge about how to initialize the application for each use case into each container. It doesn't sound good.
2. Use start hook, this is approximately the same as the first one.
3. Initialize during provision to Openshift.
4. Use an external container with individual configuration for each use case.

We chose the last one, we created additional replication controller for initializing the application? Really?

We read the documentation again.

A pod (as in a pod of whales or pea pod) is a group of one or more containers (such as Docker containers), with shared storage/network, and a specification for how to run the containers.

POD is a group of the containers. As a result, we decided to run 3 containers in an application POD

1. Init container for a PostgreSQL initialization.
2. The application container.
3. Application initialization container.

This approach allows to store our configuration as a code, there are two interesting results: the application configuration is testable and reproducible.

Tools

There a lot of already exists for managing openshift.

• Openshift templates
• bash/python/ruby + yml templates
• Terraform k8s provider
• Ansible-container
• Ansible + k8s module
• Ansible Playbook Bundle
• Ansible bender
• operator
• source2image
• kustomize
• helm
• Automation broker

During the migration, I've tested some of them. I'd like to share my results.

Openshift templates

Openshift templates

Pros:

• Native. It has awesome documentation.

Cons:

• Yet another YAML templates.
• Log horrible YAML file.
• You care about services dependencies.

Scripts and template

Pros:

• Scripting.

Cons:

• Kludges.

Terraform k8s provider

Terraform k8s provider

Pros:

• Don’t care about creating order.
• Re-use code as modules.
• You can add initialization logic.

Cons:

• No Openshift support, only k8s.
• Sometimes outdated.
• Yet another tool.

Ansible-container

Ansible-container

Pros:

• Make CM, no bash
• Re-use existing roles. role=image.
• One toolset for everything.

Cons:

• Huge images, because of a single layer.
• Looks abandon & out of date.

Ansible container was replaced by Ansible bender.

Ansible k8s module

Ansible + k8s module

Pros:

• Single playbook for all.
• Re-use code as roles.
• You can add initialization logic.

Cons:

• No proxy support.
• You care about removing. If you want to remove something, then you must declare it.
• You care about creating order. You should deploy the application before initialization.

Ansible Playbook Bundle

An Ansible Playbook Bundle (APB) is a lightweight application definition (meta-container). They are used to define and deploy complex groups of applications, deployment configs, deployments, and services to an OpenShift Origin cluster running the Ansible Service Broker. APBs offer more power and simple configuration by leveraging the power of Ansible. APBs have the following features:

The main idea is that you pack all needed thing into a container and run the container inside Openshift. Ansible Playbook Bundle

Pros:

• Bundles everything.
• Testable and reproducible.
• Service catalogue integration.

Cons:

• You need admin permissions.
• Documentation sometimes is out of date.

Result

One one hand I don't want to be the final authority, but on the other hand, I'd like to share my point of view. There is no silver bullet exists.

• if you don't plan provide your application as a service then Ansible k8s module is your choice.
• if you are going to provide your application as a service then you should go deeper in automation broker and Ansible Playbook Bundle.

P.S.

• Slides (EN)
• It is crosspost
• Russian Version