An ELK Stack Under $50 ?

Is it possible?

I have been building ELK stacks for almost 4 years and have come to the conclusion that cloud based ELK stacks for complex BI can be built within $50/month budget. Any lower than this could compromise the stability of the stack. With the below mentioned AWS reserved instances, we can reduce the cost to under $30/month

I started out by creating a simple ELK Stack on 2x cloud servers and installing the connecting ElasticSearch first. I encourage you to use the following sequence when installing & configuring the ELK.

  1. ElasticSearch 6.8 or 7.x (Or any other version)
  2. Cerebro for ElasticSearch (Install this to manage the index)
  3. Kibana (The main visualizer)
  4. Excelastic (To import xsl or csv files into ElasticSearch)
  5. Curator (Automate index management by creating rules)
  6. Beats (Transfer log files from source to LogStash -over TCP)
  7. APM (Application Performance Monitoring)

There are many many documents out there with installation instructions for the ELK stack [ https://community.spiceworks.com/how_to/164166-installing-elk-7-elasticsearch-logstash-and-kibana-windows-server-2016-part-1]

ELK Stack For $150

A moderate size ELK stack would cost between $100 to $200 based on the capacity requirements. I have selected a basic $150 Master-Slave architecture with ElasticSearch query processor. This is a special ElasticSearch node with Data=False & Master=False configuration. Such nodes would automatically participate in processing log files.

Moderate Size ElasticSearch Stack

ELK Stack for above $250

ELK Stack configurations can range from as low as $30 in the cloud to thousands of dollars based on the requirements. Such distributions can occupy dozens of servers, if not hundreds in larger analytical organizations. In the following architecture I have highlighted a larger elastic stack comprising of multiple levels of redundancy.

ElasticSearch Stack Large Configuration

This configuration might cost over $330, however when the reserved instance rates are applied, they sharply drop to $175.

Kibana View

What Are Reserved Instances?

These instances cannot be dropped easily without incurring a penalty. Reserved instances are contract based instances for 1year or 3years. Reserved instances are the cheapest among the server industry, however you need to abide by the 1year contract agreement.

Leave a Reply

Your email address will not be published. Required fields are marked *