How Do You Reduce False Positives and False Negatives?

The generation of big data has flourished, but the increased scale has proven challenging to manage.

Business Intelligence (BI) tools are quite useful but require teams to constantly watch over dashboards. Alert settings are also limited, as values have to bypass manually-defined thresholds before triggering notifications. Maintaining these alerts is very time-consuming as thresholds have to be set for each metric (or group of metrics), and manually adjusted over time as the metrics evolve.

This scenario usually leads to (1) alert floods with a high rate of false positives, which often leads users to disregard alerts. This creates opportunities for  incidents to fall through the cracks. The rigid aspect of this system can sometimes lead to (2) false negatives, where real incidents go undetected because the values don’t exceed the static thresholds.

The million-dollar question is: how can you reduce both false positives and false negatives, without constantly having to adjust your thresholds?

Within the fast-growing field of AI analytics, anomaly detection is being enabled by machine learning algorithms to be significantly more accurate.

Algorithms can be trained to independently monitor a very large volume of data to learn how each KPI or metric behaves. The system builds a baseline, a range of expected values for each metric, that fluctuates in response to related metrics and influencing events. Over time, as the algorithms learn behavioral patterns, each metric’s baseline becomes increasingly precise and accurate, which helps filter out false positives and false negatives.

The advantage with machine learning is that you not only increase scale and accuracy, it also frees more of personnel’s time to focus on mission-critical tasks.

The disadvantage is that building an AI-based anomaly detection system takes a considerable amount of time and expertise. For data-driven companies that aren’t willing to gamble a few years of resources to develop anomaly detection in-house, there are off-the-shelf solutions with impressive capabilities, such as contextual alerts and dynamic alert settings.

To demonstrate, consider an eCommerce site that has a very profitable product page. Typically, that page generates 1,000 hourly visits on weekday afternoons. Over time, and with the increased popularity of the website, visits during that time frame reach 10,000 hourly. At some point, due to a technical error, the load time for the site’s pages spikes, and that causes a drop in visitors. This is a situation that warrants an alert, so as to prevent further revenue loss. Let’s also say, for the sake of this example, that an increase in errors and a consequent increase in page load time usually coincide with a recent software release.

It’s unlikely that a team member would find both the anomalous spike in load time and the abnormal drop in visits, especially when there are many product pages to monitor. And if they did, would they be able to trace the cause? This example exemplifies the contextual nature of business metrics.

Anodot Autonomous Business Monitoring is a leading automated solution that incorporates business logic to generate high-fidelity alerts that have a direct impact on the bottom line. Algorithms detect seasonality to project expected number of visits, errors and load times, for each page of each website, for the different times of the day, and different days of the week.

For this time series metric, the blue line represents the load time values for the eCommerce product page. The blue sleeve surrounding this line is the normal range for this metric at any give time – this baseline becomes more precise as algorithms learn the metrics seasonality and patterns. The orange line is an anomalous spike in load time which exceeds the expected range.

Anodot correlates related anomalies across different layers of the business. In the aforementioned example, anomalies in visits, errors and load times are detected and presented in the same alert. The patented root-cause analysis mechanism helps expedite incident investigation and remediation.

Dynamic alert settings is a capability that presents a strong competitive edge over static thresholds. In this case, an Anodot user could also set an alert to fire only when:

• no software release occurred around the time of the incident
• the landing page belongs to a select list of high-volume webpages
• the decrease in users surpasses a percentage based on the anomaly’s delta
  • • For example, when the normal number of hourly visits was 1,000, a decrease of 100 might have warranted an alert, but when the new norm became 10,000 hourly visits, a decrease of 100 might not be worth investigating.

Anodot is an AI-based solution that provides seasonality, contextual analysis and dynamic alert settings to help create more accurate alerting, resulting in shorter time to detection and time to remediation. Users spend less time monitoring dashboards and sifting through false positives, and more time on strategic tasks.

If you’re interested in learning more, this video will illustrate some of these concepts or you can reach out for a demo that fits your use case.