ElasticSearch and Kibana Deployment on Kubernetes

 

In a post 2 years ago, I've specified steps to deploy ElasticSearch and TLS on kubernetes. It was back then, when ElasticSearch and kubernetes have only started dating, and the integration was not simple. But now ElasticSearch deployment is built-in part of the product, and much simpler.

This post is based mostly on the Elastic Cloud documentation.

CRDs

Like most of the operator based deployments, CRDs are required:

kubectl create -f https://download.elastic.co/downloads/eck/2.6.1/crds.yaml

Operator

Once CRDs are applied we can deploy the Elastic Cloud operator.

kubectl apply -f https://download.elastic.co/downloads/eck/2.6.1/operator.yaml

Kind and Ingress

In case the elastic is used on a local development machine using kind, we will need kind ingress to access the services.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: elastic-ingress-kind
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
  rules:
    - host: elastic
      http:
        paths:
          - pathType: ImplementationSpecific
            backend:
              service:
                name: elastic-es-http
                port:
                  number: 9200
    - host: kibana
      http:
        paths:
          - pathType: ImplementationSpecific
            backend:
              service:
                name: kibana-kb-http
                port:
                  number: 5601

In addition, we need to add to /etc/hosts the entries:

127.0.0.1 elastic

127.0.0.1 kibana

Elastic

To install ElasticSearch, we apply the elastic custom resource, and let the operator handle it.

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: elastic
spec:
  http:
    tls:
      selfSignedCertificate:
        disabled: true
  version: 8.6.0
  nodeSets:
    - name: default
      count: 1
      config:
        node.store.allow_mmap: false

Notice that we have disabled HTTPs, so we can access the service easily from the kind ingress.

To view the ElasticSearch password, we can use the following:

Password=$(kubectl get secret elastic-es-elastic-user -o go-template='{{.data.elastic | base64decode}}')
echo "Password is ${Password}"

And we can access the ElasticSearch using curl:

curl -k -u "elastic:${Password}" "http://elastic"

We can also view the ElasticSearch status using the command:

kubectl get elasticsearch

A simple script that uses all these to check that the ElasticSearch is ready is below.

#!/bin/bash

set -e
cd "$(dirname "${BASH_SOURCE[0]}")"

function isReady(){
  kubectl get elasticsearch
  kubectl get pods
  count=$(kubectl get elasticsearch | grep green | wc -l)
  if [[ "${count}" == "0" ]]; then
      return 1
  else
      return 0
  fi
}

function waitForReady(){
  until isReady
  do
    echo "waiting for elastic to be green"
    sleep 5
  done

  echo "elastic is ready"
}

function sendRequest(){
  Password=$(kubectl get secret elastic-es-elastic-user -o go-template='{{.data.elastic | base64decode}}')
  echo "Password is ${Password}"
  set -x
  curl -k -u "elastic:${Password}" "http://elastic"
}


waitForReady
sendRequest

Kibana

To install kibana, we apply the following custom resource:

apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: kibana
spec:
  http:
    tls:
      selfSignedCertificate:
        disabled: true
  version: 8.6.0
  count: 1
  elasticsearchRef:
    name: elastic

Here as well, we turn off the HTTPS access to make access though kind easy.

An we can wait for kibana using another simple script:

#!/bin/bash

set -e
cd "$(dirname "${BASH_SOURCE[0]}")"

function isReady(){
  kubectl get kibana
  kubectl get pods
  count=$(kubectl get kibana | grep green | wc -l)
  if [[ "${count}" == "0" ]]; then
      return 1
  else
      return 0
  fi
}

function waitForReady(){
  until isReady
  do
    echo "waiting for kibana to be green"
    sleep 5
  done

  echo "kibana is ready"
}


waitForReady

Cleanup

To remove the deployments, we use the following:

kubectl delete kibana --all
kubectl delete elastic --all
kubectl delete -f ingress.yaml

Final Note

This progress is great, I really like the operator made by Elastic. Notice that the operator also handles upgrades, and more.