Kubernetes Multi-Container Patterns

A classic example of multi container pods is when you have your Main app Container separated from a Container that has helper code. That way you separate responsibility and the Helper Container can be reused in other pods.

Multi Container patterns

Depending on the role of each container in a Pod, best practices (or design patterns) emerged. All patterns presented here are based on a pattern called Sidecar, which is simply having two Containers in one Pod: A Main Container and a Sidecar Container.

We can further specify whether the Sidebar Container has to run to completion before(!) the Main Container starts. If yes, then we call this an **Init Container **or Init Pattern. If not, both app containers are running concurrently and we call it Sidecar pattern.

Init Pattern

One use case for this pattern is to have the Sidecar pull down code from a git repository initially(!) and only after that is done, the web server Container will start using that code.

To make this happen we have to configure the Pod accordingly. Kubernetes even has a special configuration property initContainers for that:

1
apiVersion: v1
2
kind: Pod
3
metadata:
4
  name: git-syncer
5
  labels:
6
    app: git-syncer
7
spec:
8
  initContainers:
9
  - image: k8s.gcr.io/git-sync:v3.1.5
10
    name: init-sync-ctr
11
    volumeMounts:
12
    - name: html
13
      mountPath: /tmp/git
14
    env:
15
    - name: GIT_SYNC_REPO
16
      value: https://github.com/your-username/your-repo.git
17
    - name: GIT_SYNC_BRANCH
18
      value: master
19
    - name: GIT_SYNC_DEPTH
20
      value: "1"
21
    - name: GIT_SYNC_DEST
22
      value: "html"
23
    - name: GIT_SYNC_ONE_TIME  # only get the git files initially and then quit
24
      value: "true"
25
  containers:
26
  - image: nginx
27
    name: web
28
    volumeMounts:
29
    - name: html
30
      mountPath: /usr/share/nginx
31
  volumes:
32
  - name: html
33
    emptyDir: {}
34
---
35
apiVersion: v1
36
kind: Service
37
metadata:
38
  name: git-syncer
39
spec:
40
  selector:
41
    app: git-syncer
42
  ports:
43
    - port: 80
44
  type: LoadBalancer

Kubernetes runs those Init Containers in order and sequential (not parallel) and guarantees that they are run before the Main Container.

But what if we want to continue to update our running app, even after the initial git sync? Well, we create a similar configuration and put them in a (non-init) sidecar container:

1
apiVersion: v1
2
kind: Pod
3
metadata:
4
  name: git-syncer
5
  labels:
6
  app: git-syncer
7
spec:
8
  containers:
9
  - image: nginx
10
    name: web
11
    volumeMounts:
12
    - name: html
13
      mountPath: /usr/share/nginx/
14
  - image: k8s.gcr.io/git-sync:v3.1.5
15
    name: init-sync-ctr
16
    volumeMounts:
17
    - name: html
18
      mountPath: /tmp/git
19
    env:
20
    - name: GIT_SYNC_REPO
21
      value: https://github.com/your-username/your-repo.git
22
    - name: GIT_SYNC_BRANCH
23
      value: master
24
    - name: GIT_SYNC_DEPTH
25
      value: "1"
26
    - name: GIT_SYNC_DEST
27
      value: "html"
28
  volumes:
29
  - name: html
30
  emptyDir: {}
31
---
32
apiVersion: v1
33
kind: Service
34
metadata:
35
  name: git-syncer
36
spec:
37
  selector:
38
    app: git-syncer
39
  ports:
40
  - port: 80
41
  type: LoadBalancer

Another use case is if you want to prepare environment variables or wait for the availability of a service before the Main Container executes. In the following example we execute a command that continuously looks up (1 seconds intervals) if the service my-service is available:

1
apiVersion: v1
2
kind: Pod
3
metadata:
4
  name: ps-init
5
  labels:
6
    app: initializer
7
spec:
8
  initContainers:
9
  - name: init-ctr
10
    image: busybox
11
    command: ['sh', '-c', 'until nslookup my-service; do echo waiting for my-service; sleep 1; done; echo Service found!']
12
  containers:
13
    - name: web-ctr
14
      image: myrepo/web-app:1.0
15
      ports:
16
        - containerPort: 8080

If any Init Container fails, the pod restarts (unless restart policy it set to ‘never’) and the Init Containers run again! That’s why it is important that your init container are idempotent, meaning they should have the same predictable outcome every time they are started. Keep resource limits low for Init Containers.

Service Mesh pattern

Service Mesh pattern is another Sidecar pattern. It serves as a Pod’s single entry point. It allows you to add additional features/services like monitoring, encrypting, telemetry, filtering, logging of your Pod.

Adapter pattern

The Adapter pattern takes non standard output and transforms it to standardised output via a Adapter Container, so that an External system can process it.

For example, Prometheus is a monitoring tool that uses this pattern.

Ambassador pattern

The Ambassador pattern proxies traffic from the main container to an external system: