A Journey into Helm

A-Journey-into-Helm.html

<section>
	<h1>A Journey into Helm</h1>
	<h2>Grant Sherrick</h2>
</section>
<section data-transition="linear">
	<section id="where-we-started">
		<h2>We started with a git repo.</h2>
		<br>
		<pre><code>the-greatest-kafka-consumer
├── Dockerfile
├── README.md
├── package.json
├── src
│   └── index.js
├── test
│   └── index_test.js
├── dev
│   ├── the-greatest-deployment.yml
│   ├── the-greatest-cm.yml
│   └── the-greatest-job.yml
├── local
│   ├── the-greatest-deployment.yml
│   ├── the-greatest-cm.yml
│   └── the-greatest-job.yml
└── the-greatest-gif.gif</code></pre>
	</section>
	<section id="issues-in-our-repo">
		<h2>We had some issues in that repo.</h2>
		<ul>
			<li>Not DRY</li>
			<li>No local development environment</li>
			<li><b>Dirty</b>, persistent Kafka</li>
			<li>Manual, inconsistent deployments</li>
		</ul>
	</section>
	<section id="templating-options">
		<h2>We looked into some options</h2>
		<ul>
			<li class="fragment">Bash script all the things</li>
			<li class="fragment">A Custom Templating Service in Golang</li>
			<li class="fragment">Jsonnet</li>
			<li class="fragment">Helm</li>
		</ul>
	</section>
	<section id="helm-overview">
		<h2>Helm</h2>
		<span class="fragment">
			<p>Helm is a tool for managing Kubernetes charts.</p>
			<p>Charts are packages of pre-configured Kubernetes resources.</p>
		</span>
	</section>
	<section id="k8s-resources-explanation">
		<h3>Kubernetes Resources are:</h3>
		<ul class="fragment" style="float: left; padding-left: 6.5em;">
			<li>pods</li>
			<li>deployments</li>
			<li>jobs</li>
			<li>configmaps</li>
			<li>secrets</li>
			<li>and many more</li>
		</ul>
	</section>
	<section id="k8s-manifests-explanation">
		<h3>Kubernetes Manifests are:</h3>
		<ul class="fragment">
			<li>pod.yaml</li>
			<li>deployment.yaml</li>
			<li>job.yaml</li>
			<li>configmap.yaml</li>
			<li>secret.yaml</li>
			<li>and all of the other k8s yamls</li>
		</ul>
	</section>
	<section id="what-is-a-chart">
		<h2>So...what is a Helm chart?</h2>
		<br>
		<h3 class="fragment" style="text-align: left; padding-left: 2.4em;">A Helm chart contains:</h3>
		<ul>
			<li class="fragment" style="text-align: left;">a set of templated kubernetes manifests.</li>
			<li class="fragment" style="text-align: left;">collective versioning information about the chart.</li>
			<li class="fragment" style="text-align: left;">a list of charts that this chart depends upon.</li>
		</ul>
	</section>
	<section id="the-greatest-kafka-consumer-chart">
		<h2>We created a helm chart for <code>the-greatest-kafka-consumer</code>:</h2>
		<br>
		<pre><code>the-greatest-kafka-consumer
├── Chart.yaml
├── NOTES.md
├── bin
│   ├── build-and-push-image
│   ├── run-integration-tests
│   ├── deploy-from-helm-repo
│   └── push-to-helm-repo
├── charts
│   ├── local-kafka-0.2.1.tgz
│   ├── the-greatest-kc-dev-0.1.1.tgz
│   ├── the-greatest-kc-local-0.1.1.tgz
│   ├── the-greatest-kc-prod-0.1.1.tgz
├── local-charts
│   ├── the-greatest-kc-dev
│   │   ├── Chart.yaml
│   │   └── values.yaml
│   ├── the-greatest-kc-local
│   │   ├── Chart.yaml
│   │   └── values.yaml
│   └── the-greatest-kc-prod
│   │   ├── Chart.yaml
│   │   └── values.yaml
├── requirements.lock
├── requirements.yaml
├── templates
│   ├── _helpers.tpl
│   ├── the-greatest-deployment.yaml
│   ├── the-greatest-cm.yaml
│   └── the-greatest-job.yaml
└── values.yaml</code></pre>
	</section>
	<section id="the-greatest-kafka-consumer-templates">
		<h2>All of our k8s manifests were templated!</h2>
		<br>
		<pre><code>apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: {{ template "name" . }}
  annotations:
    checksum/theGreatestConfigMap: {{ include (print $.Template.BasePath "/the-greatest-cm.yaml") . | sha256sum }}
  labels:
    app: {{ template "name" . }}
    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
    release: {{ .Release.Name }}
    heritage: {{ .Release.Service }}
spec:
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 0
      maxSurge: 3
  replicas: {{ .Values.theGreatestKafkaConsumer.replicaCount }}
  template:
    metadata:
      annotations:
        app: {{ template "name" . }}
      labels:
        app: {{ template "name" . }}
        release: {{ .Release.Name }}
    spec:
      restartPolicy: {{ .Values.theGreatestKafkaConsumer.restartPolicy | quote }}
      imagePullSecrets:
      - name: {{ .Values.theGreatestKafkaConsumer.imageSecretName | quote }}
      containers:
      - image: {{ .Values.theGreatestKafkaConsumer.image | quote }}
        imagePullPolicy: Always
        name: the-greatest-kafka-consumer
        volumeMounts:
        - mountPath: /root/.aws
          name: aws-creds
        env:
        - name: BOOTSTRAP_SERVERS
          value: {{ .Values.theGreatestKafkaConsumer.kafkaServers | quote }}
        - name: SCHEMA_REGISTRY_URL
          value: {{ .Values.theGreatestKafkaConsumer.schemaRegistryUrl | quote }}
        - name: CONNECT_REST_ADVERTISED_HOST_NAME
          value: {{ template "name" . }}
        ports:
        - containerPort: 4563
          protocol: TCP
      volumes:
      - secret:
          defaultMode: 420
          secretName: {{ .Values.theGreatestKafkaConsumer.s3SecretName | quote }}
        name: aws-creds</code></pre>
	</section>
	<section id="the-greatest-kafka-consumer-manifests">
		<h2>Our live k8s resources were versioned!</h2>
		<h3>using the helm server (Tiller)</h3>
		<br>
		<pre><code>$ kubectl get deployments
NAME                  DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
the-greatest-kc       1         1         1            1           23h
tiller-deploy         1         1         1            1           1d

$ helm upgrade --install the-greatest-kc-dev ./the-greatest-kafka-consumer --set tags.the-greatest-kafka-consumer-dev-values=true

Release "the-greatest-kc-dev" has been upgraded. Happy Helming!
LAST DEPLOYED: Tue Jan 30 13:52:55 2018
NAMESPACE: data-processing
STATUS: DEPLOYED

RESOURCES:
==> v1beta1/Deployment
NAME                                     DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE
the-greatest-kc                          1        1        1           1          21h

==> v1/ConfigMap
NAME                                     DATA  AGE
the-greatest-cm                          1     21h

$ helm ls
NAME                    REVISION         UPDATED                        STATUS          CHART                                    NAMESPACE
the-greatest-kc-dev     2                Mon Jan 29 23:06:46 2018       DEPLOYED        the-greatest-kafka-consumer-0.1.1        data-processing</code></pre>
	</section>
	<section id="the-greatest-kafka-consumer-local-kafka">
		<h2>We got a <b>clean</b>, ephemeral, local Kafka platform!</h2>
		<br>
		<pre><code>local-kafka
├── Chart.yaml
├── templates
│   ├── _helpers.tpl
│   ├── kafka-deployment.yaml
│   ├── kafka-service.yaml
│   ├── rest-proxy-deployment.yaml
│   ├── rest-proxy-service.yaml
│   ├── schema-registry-deployment.yaml
│   ├── schema-registry-service.yaml
│   ├── zookeeper-deployment.yaml
│   └── zookeeper-service.yaml
└── values.yaml
		</code></pre>
	</section>
	<section id="the-greatest-kafka-consumer-versioned-charts">
		<h2>Our templates were versioned!</h2>
		<h3>via Chart.yaml</h3>
		<br>
		<pre><code>apiVersion: v1
description: A Helm chart for the Greatest Kafka Consumer
name: the-greatest-kafka-consumer
version: 0.2.1</code></pre>
	</section>
	<section id="prism-long-term-storage-env-specific-charts1">
		<h2>We got environment specific, templated k8s resources!</h2>
		<br>
		<pre><code>$ cat local-charts/prism-lts-local/values.yaml
exports:
  environment-specific-data:
    theGreatestKafkaConsumer:
      kafkaServers: "local-kafka:9092"
      schemaRegistryUrl: "http://local-schema-registry:8081"
      s3SecretName: "s3-creds"
      restartPolicy: "Always"
    theGreatestJob:
      name: "health-metrics-kc-job"
      s3BucketName: "local-kc-job"</code></pre>
	</section>
	<section id="the-greatest-kafka-consumer-env-specific-charts2">
		<h2>We got packaged, environment specific, templated k8s resources!</h2>
		<br>
		<pre><code>$ cat requirements.yaml
dependencies:
- name: local-kafka
  version: "0.2.0"
  repository: "file://../local-kafka"
  tags:
    - the-greatest-kafka-consumer-local-values
- name: the-greatest-kafka-consumer-local
  version: "0.1.1"
  repository: "file://local-charts/the-greatest-kafka-consumer-local"
  tags:
    - the-greatest-kafka-consumer-local-values
  import-values:
    - environment-specific-data
- name: the-greatest-kafka-consumer-dev
  version: "0.1.1"
  repository: "file://local-charts/the-greatest-kafka-consumer-dev"
  tags:
    - the-greatest-kafka-consumer-dev-values
  import-values:
    - environment-specific-data
- name: the-greatest-kafka-consumer-prod
  version: "0.1.1"
  repository: "file://local-charts/the-greatest-kafka-consumer-prod"
  tags:
    - the-greatest-kafka-consumer-prod-values
  import-values:
    - environment-specific-data</code></pre>
	</section>
	<section id="prism-long-term-storage-env-specific-charts3">
		<h2>We got easily modifiable, packaged, environment specific, templated k8s resources!</h2>
		<br>
		<p>There is a priority order for setting templated values:
		<ol>
			<li>--set or -f  <i>other-values.yaml</i> using the <code>helm</code> CLI tool</li> 
			<li><i>values.yaml</i>  in required charts (including other envs)</li>
			<li><i>values.yaml</i>  in the main chart</li>
		</ol>
	</section>
	<section id="using-a-chart1">
		<h2>Not only can we chart up our stuff, there's public charts too</h2>
		<br>
		<code class="fragment">helm install stable/prometheus</code>
	</section>
	<section id="using-a-chart2">
		<h2>How do I find out what is configurable?</h2>
		<br>
		<h3 class="fragment" style="text-align: left; padding-left: 1.5em; padding-bottom: 0.5em;">1. <b>look at its values.yaml file</b></h3>
		<pre class="fragment"><code>nodeExporter:
  ## If false, node-exporter will not be installed
  ##
  enabled: true

  # Defines the serviceAccountName to use when `rbac.create=false`
  serviceAccountName: default

  ## node-exporter container name
  ##
  name: node-exporter</code></pre>
	</section>
	<section id="using-a-chart2">
		<h2>How do I configure a public helm chart?</h2>
		<br>
		<code class="fragment" style="text-align: left;">helm install stable/prometheus --set nodeExporter.enabled=false</code>
		<br><br>
		<code class="fragment" style="text-align: left;">helm install stable/prometheus -f other-values.yaml</code>
	</section>
	<section id="helm-disadvantages">
		<h2>What doesn't Helm do well?</h2>
		<br>
		<p>NOTE: all of these are my opinions</p>
		<ul>
			<li class="fragment"><code>helm upgrade</code></li>
			<li class="fragment">Doesn't follow the Unix philosophy</li>
			<li class="fragment">Obfuscates some kubectl errors and logs</li>
			<li class="fragment">Conventions abound (chart versioning, checksum, exports, helper functions)</li>
		</ul>
	</section>
	<section id="helm-advantages">
		<h2>What does Helm do well?</h2>
		<br>
		<p>NOTE: all of these are my opinions</p>
		<ul>
			<li class="fragment">templating (especially across environments)</li>
			<li class="fragment">intra-chart checksum-based redeployment</li>
			<li class="fragment">a great set of docs and a very active community</li>
			<li class="fragment">dependency resolution between projects</li>
		</ul>
	</section>
	<section>
		<h2>Thanks!</h2>
		<h1>Have a helm of a Day!</h1>
	</section>
</section>