Llamas Band and Green/Blue Deployments

Overview

The last two jobs indicated they were interested in or already doing green/blue deployments. There are multiple methods of deploying in this manner. Kubernetes has two strategies in the deployment context in order to gradually roll out a new image and roll back in the event of a problem. Argo provides a Rollout deployment process that provides additional deployment strategies via controller and CRDs.

Kubernetes Strategies

There are two update strategies that are available in Kubernetes by default. The Recreate strategy and the RollingUpdate strategy.

The Recreate strategy essentially deletes all existing pods before starting up the new pods.

The RollingUpdate strategy uses maxUnavailable and maxSurge to do an rolling update. Default values for both are 25% although you can set it to an absolute number that define the number of pods.

Argo Rollouts Strategies

There are two update strategies that are provided by the Argo Rollouts tools. The blueGreen strategy and the Canary strategy. Note the Kubernetes documentation does provide information on how you can perform a Canary update in Kubernetes. In Argo Rollouts, Canary is an actual strategy.

blueGreen Strategy

The blueGreen strategy from Argo Rollouts has two services. An Active service and a Preview service. When an upgrade is made to the Rollout context, new images are started and the Preview service points to the new ReplicaSet. You can use the Analysis task which verifies the new ReplicaSet is ready for traffic. Once tests pass, the Active service is promoted to the new ReplicaSet and the old ReplicaSet is called down to zero.

Canary Strategy

The Canary strategy lets you configure the rollout so that a small percentage of the new Replicaset is available to users. You define steps in the strategy that must complete before migrating more users to the new ReplicaSet. The steps can be simple pause statements where you wait a specific period of time before migrating more users or checks for the number of active pods. You can even have it pause until someone manually promotes the ReplicaSet.

Green/Blue Process

The problem with these strategies is they don’t address the need for testing in the same environment. Some customers don’t have access to lower environments so having a second one in all environments means new changes can be tested before they go live. So if the requirement is to have two sites up and running, one live and one test, then the process needs something different to accommodate that requirement.

For the Llamas band website, there are two namespaces, a llamas-blue and a llamas-green. If we identify llamas-blue as the live site, the ingress context would have llamas.internal.pri as the ingress URL. But for testing, we might define the ingress URL as llamas-green.internal.pri in the llamas-green namespace. In this manner, we can access https://llamas-green.internal.pri, test new features, and ensure it continues to work as expected in a production environment. When ready to activate the llamas-green namespace, we update the llamas-blue ingress context for the URL to be llamas-blue.internal.pri and update the llamas-green ingress context for the URL to be llamas.internal.pri.

This gives us the capability of immediately switching back to the llamas-blue project in case something goes wrong. Otherwise all future updates now apply to the llamas-blue project.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: llamas
  namespace: llamas-blue
  annotations:
    kubernetes.io/ingress.class: haproxy
spec:
  rules:
  - host: llamas.internal.pri
    http:
      paths:
      - backend:
          service:
            name: llamas
            port:
              number: 80
        path: /
        pathType: Prefix
  tls:
  - hosts:
    - llamas.internal.pri

And the green ingress route.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: llamas
  namespace: llamas-green
  annotations:
    kubernetes.io/ingress.class: haproxy
spec:
  rules:
  - host: llamas-green.internal.pri
    http:
      paths:
      - backend:
          service:
            name: llamas
            port:
              number: 80
        path: /
        pathType: Prefix
  tls:
  - hosts:
    - llamas-green.internal.pri

References

• Kubernetes Deployments
• Argo Rollouts
• Argo BlueGreen Strategy
• Argo Canary Strategy