As product managers, you’re ultimately the one held responsible for the entire product. So the last thing you want to assume is that someone else has got monitoring and alerts covered.
In the first days of a release, all eyes are on the new product or latest feature. Just a few months later, when you introduce a brand new feature, the old one might break in the process. At times like these, you want to be ahead of your users, and not hear from your users that something isn’t working.
Developers only cover the tech aspect of monitoring. So who’s monitoring the business funnel? Who looks at conversion rates? Who knows what should be monitored? Take a look in the mirror, it’s you – the product manager.
As funnel ratios change each day, alerts on static thresholds will not do, this requires machine learning-based anomaly detection. Normal behavior needs to be learned, identifying seasonality and alerting when ratios are abnormal.
When Argos had a price glitch posted on a popular site, it ended up crashing their site and leaving many angry customers. They had mistakenly advertised the Xbox One at £89.99, instead of £300.
An anomaly detection system would have tracked that glitch long before the site would have crashed. It might have detected an increase in the number of sales, or it might have detected a decrease in earnings per item, etc.
Products that are not monitored well for performance or accuracy will sooner, rather than later, end up costing companies in lost revenue, and in many cases also bring damage to brand reputation, that is even harder to recover from.
As awesome product owners, you need to make sure to apply the latest monitoring processes as an integral part of the product development requirements.
Here are some tips to make your monitoring easy, consistent and smart.
1. Get Continuous Analysis
Track every feature, no matter how big or small, that is released into production.
In the last few years, continuous deployment has became the standard for high functioning development teams.
This requirement holds for analytics as well. For example, you should track the new user interface functions added in every release. Angulartics is just one example of a library that can automatically track clicks and sends data to the analytics application that you work with. It supports Google analytics, Mixpanel, Segment and many others.
2. Incorporate Automatic Anomaly Detection
Monitoring should work automatically too. Every new event you track should be automatically monitored using a system that builds a baseline for it and monitors whether or not it is within that baseline. In cases where anomalous behavior is identified, or when a threshold is breached the monitoring service should automatically identify the situation, ideally BEFORE hitting the threshold that was set. Such services can be found under Google Analytics, Mixpanel and of course, in Anodot.
3. Get Alerts, In Real Time, But Only For What Matters
When an anomaly is identified or when a static threshold has been breached the monitoring system should reach out and alert the owner – engineer, PM, QA or all of them.
This saves from the actual monitoring time required by the product owner, since an autonomous analytics system only reaches out when necessary and does not otherwise disturb product owners during the day.
Product management is a demanding job, so get a good night sleep, and make sure someone (or better, something) is watching over how your product is performing.
———–
This post was written together with one awesome Product Champ who sleeps like a baby: Omer Gartzman
Written by Amit Levi
You'll believe it when you see it
