Log in
Business Monitoring Cloud Cost
Let’s talk

The ELK stack (ElasticSearch, Logstash, Kibana), by Elastic, has gained tremendous popularity in the last several years.

By viewing Kibana graphs derived from the event and log data stored in Elastic, analysts, developers and DevOps can visually get actionable insights in real time.

But what happens when you start to have too many graphs to track? For example, looking at page views and conversion rates from all your users,  grouped by country, user device type and OS would generate thousands of combinations leading to thousands of graphs.

Can you really track and gain insights when the number of interesting graphs increases to thousands and hundred of thousands (and millions in some cases)?

The answer is quite clear – No, this approach doesn’t scale. Unless you can afford hiring an army of experts to look at them.

Data Science to the rescue…

This is where data science in general, and specifically Anodot’s anomaly detection service, scales your monitoring capabilities, without needing to hire that army. Let the machine track the thousands to millions of graphs (aka metrics) for you, automatically learn their normal behavior and how they are related, and alert you when one or more change their pattern and behave abnormally.

Integrating Anodot with ELK in three steps

(These instructions assume that you already have a running ELK stack, and have an active Anodot account – if you don’t, contact: [email protected] or fill out this form)

  1. Follow this great post by Erik Redding and/or this one.  to see how you can send metrics using Graphite protocol with logstash.
  2. Install the Anodot-Relay which supports the graphite protocol.
  3. Add the Anodot relay to your logstash configuration output section as graphite output, set  the host parameter as the relay address.

output {

elasticsearch { host => localhost }

   graphite {

   host => ANODOT_RELAY_IP

   ….

   }

}

That’s all you need to do, and you can start sending metrics to Anodot for immediate analysis.

By adding Anodot as a layer on top of your kibana, you will be alerted to any anomaly, which will dramatically decrease your detection and investigation time.

Enjoy.

Written by Shay Lang

Shay is VP R&D and a founder of Anodot. He has spent his career directing R&D and product engineering teams in the software and cybersecurity space, for companies such as Trustwave, M86 Security, Finjan Software and Voltaire. He holds a bachelor’s degree from Technion – Israel Institute of Technology and an Executive MBA from Tel Aviv University.

You'll believe it when you see it