Containers: How to build ELK Stack on Linux on IBM Z

by Alice Frosi

With the increase of logging data, the need of robust logging tool has become a must. One of the most spread tool is the ELK stack from elastic. The ELK stack is composted by 3 main components:

• Elasticsearch: distributed, RESTful search and analytics engine
• Logstash: the data processor
• Kibana: UI for elasticsearch

An additional component, called Beats, can be used as data shipper to elasticsearch or logstash.

Currently, there aren't any official container images available for ELK stack for IBM Z. However, you can use the Dockerfiles available at https://github.com/linux-on-ibm-z/dockerfile-examples in order to build the images from source code.

The configuration files and docker-compose are available also on the github repo s390x-container-logging.

Here, the steps how to build the container images.

Elasticsearch:

$ mkdir -p dockerfiles/elasticsearch/
$ wget -O Dockerfile https://raw.githubusercontent.com/linux-on-ibm-z/dockerfile-examples/master/Elasticsearch/Dockerfile 
$ docker build --no-cache -t elastisearch:7.3.0 -f dockerfiles/elasticsearch/Dockerfile .

Using this Dockerfile for elasticsearch, you could encounter a "Permission denied" issue. For this reason, you can use this additional Dockerfile to correct the ownership of the folder.

dockerfiles/elasticsearch/Dockerfile-user:

FROM elasticsearch:7.3.0

RUN chown -R elasticsearch /usr/share/elasticsearch \
  && mkdir -p /usr/share/elasticsearch/data/nodes \
  && chown -R elasticsearch:elasticsearch /usr/share/elasticsearch/data

USER elasticsearch

then, build the new container image:

$ docker build -t elasticsearch-user:7.3.0 -f dockerfiles/elasticsearch/Dockerfile-user .

Logstash:

$ mkdir -p dockerfiles/logstash/
$ wget -O dockerfiles/logstash/Dockerfile https://raw.githubusercontent.com/linux-on-ibm-z/dockerfile-examples/master/Logstash/Dockerfile
$ wget -O dockerfiles/logstash/dockerfile_netty_tcnative https://raw.githubusercontent.com/linux-on-ibm-z/dockerfile-examples/master/Logstash/dockerfile_netty_tcnative
$ wget -O dockerfiles/logstash/dockerfile_openssl_dynamic https://raw.githubusercontent.com/linux-on-ibm-z/dockerfile-examples/master/Logstash/dockerfile_openssl_dynamic
$ docker build --no-cache -t logstash:7.3.0 -f dockerfiles/logstash/Dockerfile .

Kibana:

$ mkdir -p dockerfiles/kibana/
$ wget -O dockerfiles/kibana/Dockerfile https://raw.githubusercontent.com/linux-on-ibm-z/dockerfile-examples/master/Kibana/Dockerfile
$ docker build -t kibana:7.3.0 -f dockerfiles/kibana/Dockerfile .

Beats:

$ mkdir -p dockerfiles/beats/
$ wget -O dockerfiles/beats/Dockerfile https://raw.githubusercontent.com/linux-on-ibm-z/dockerfile-examples/master/Beats/Dockerfile
$ docker build --no-cache -t beats:7.3.0 -f dockerfiles/beats/Dockerfile .

Check the images:

$ docker images
REPOSITORY                TAG                                        IMAGE ID            CREATED             SIZE
elastisearch-user         7.3.0                                      9a1690439700        3 days ago          1.4GB
elasticsearch-user        7.3.0                                      9a1690439700        3 days ago          1.4GB
beats                     7.3.0                                      08d107976951        3 days ago          1.43GB
elasticsearch             7.3.0                                      8fa079cb4f10        4 days ago          1.02GB
logstash                  7.3.0                                      679cbd6747e1        4 days ago          633MB
kibana                    7.3.0                                      171eb463de5c        4 days ago          7.45GB

Now, you are able to run ELK stack. Check, the next blog post to have some hints how to run it.