Similar to other companies in the entertainment industry, gaming companies typically drive revenue from three sources: in-app purchases, ads, and subscription. A couple of examples of these sources include creating different in-app purchase options for each game and various ad units from multiple ad networks. While this diversity in revenue streams may be advantageous from a business perspective, from a technical standpoint, it creates numerous challenges. In particular, every time an in-app purchase fails, an ad isn’t displayed correctly, or a user isn’t converted to a paying customer, revenue is lost. Navigating Permutational Complexities   Experienced game developers understand each one of their games’ permutational complexities—from the operating systems, user segments, multiple devices, promotional strategies, and more. Each one of these permutations not only presents unique technical challenges but also must be monitored constantly in order to prevent monetization failures. The Limitations of Traditional Monitoring   In the past, many companies have tried traditional monitoring and alerting methods, but the inherent complexities mentioned above often make this unfeasible. Either these dashboards and manual thresholds will miss an anomaly because it is too granular, or the system will generate too many false positives. As we’ll discuss in this guide, autonomous AI-based proactive monitoring is the solution to dealing with the complexities of gaming analytics. What is Autonomous Monetization Monitoring?   As mentioned, proactive monitoring revenue streams of gaming companies often involves tracking thousands of metrics and billions of events each day. Autonomous monitoring allows you to not only observe each individual metric but also automatically learn the normal behavior of each on its own by using a branch of machine learning called unsupervised learning.  As described in our guide on Unsupervised Anomaly Detection: Unsupervised machine learning algorithms, however, learn what normal is, and then apply a statistical test to determine if a specific data point is an anomaly. A system based on this kind of anomaly detection technique is able to detect any type of anomaly, including ones that have never been seen before. In other words, unsupervised learning can be used to monitor 100% of the data, identify anomalies that lead to revenue losses, and alert the relevant team in real-time. Keep in mind that none of these alerts are threshold-based, and instead are constantly changing on their own based on the learned normal behavioral patterns. Similarly, each anomaly is paired with a root-cause analysis of incidents that affect revenue streams or the user experience. This allows the technical team to identify what’s causing the incident and have the fastest possible time-to-resolution. In the context of gaming, Anodot’s autonomous monitoring solution has proven to reduce up to 70% of losses associated with monetization errors for game studios such as King, Gamesys, Outfit7, Moonactive, and more. To do this, the AI-based solution monitors three core monetization channels, including: In-app purchases Ads Subscriptions Now that we’ve discussed what autonomous monetization monitoring is, let’s look at several real-world examples. Use Case: Autonomous Monitoring for Gaming   In this section, we’ll review two use cases of autonomous monitoring for gaming: in-app purchases and drops in ad impressions. Monitoring In-App Purchase Funnels One particular game studio started using autonomous monitoring for its monetization of in-app purchases. Here are a few highlights from their experience: In just the past 6 months, 57 anomalies triggered alerts in real-time based on spikes of purchase failures These purchase failures resulted from various technical bugs, version updates, payment gateway issues, and more With the use of the AI-based correlation analysis, the team was able to remediate the issues within hours instead of days They’ve estimated that because of their faster time-to-resolution, they were able to save $800k in the past 6 months Below, you can see three of the purchase failure rate metrics which are measured by different games, platforms, and versions. The shaded blue area represents the normal range of each metric and each one of the anomalies from this normal behavior is highlighted in orange. As you can see, as the normal range shifts over time, the purchase fail metrics also shift automatically by Anodot’s algorithms. During the same period, there were a total of 64 purchase failure related alerts, which means the detection rate was 92%.     Real-Time Alerts on Drops in Ads Shown Another example of autonomous monitoring for gaming comes from a game studio that has over 500 million monthly active users (MAU). In this case, the company wanted to monitor ad impressions presentations in their games, which is of course tied to the bottom line of the company. Here are a few highlights from their experience implementing autonomous monitoring: In the past 6 months, 18 alerts were triggered indicating a drop in ad impressions across various games, platforms, and ad networks 17 of the 18 alerts were confirmed to be significant revenue impactful incidents, making the detection rate 94.4% The company estimates that they saved $153K USD based on detecting and resolving these 17 incidents in near real-time As you can see below, the blue shaded area represents the normal ad impression range. Here you can see the drop in the ads shown occurred for multiple games and platforms, apart from a single ad provider (Facebook). These anomalies were not only alerted in real-time but were also correlated with a single alert resulting in a faster remediation time and avoiding an alert storm.   Summary: Autonomous Monetization Monitoring   As we’ve discussed, monitoring the various revenue streams of gaming companies is a highly complex undertaking. While experienced developers understand each one of their game’s permutational complexities, monitoring them in real-time still presents a unique challenge. Some companies have tried using manual thresholds in the past, but this either doesn’t have enough granularity or will be triggered based on false-positives, leaving the technical team with an alert storm. Instead, AI-based autonomous monetization monitoring is the only solution that can observe every single metric, learn its normal behavior on its own, and identify anomalies in real-time. Not only does catching and resolving these anomalies drastically enhance the user experience but ultimately it helps companies improve their bottom line.  

Revenue monitoring not only involves monitoring huge amounts of data in real-time but also finding correlations between thousands, if not millions, of customer experience and other metrics.  Are traditional monitoring methods capable of detecting a correlation between a drop in user log-ins and a drop in revenue as it’s happening? For many reasons, the answer is no. The Power of AI-Based Monitoring   To kick off our "Good Catch" series, we're sharing anomalies that Anodot caught for our customers, who flagged each of them as a "good catch" in our system. For an online gaming customer, Anodot alerted them to a drop in log-ins and correlated the anomaly to a spike in command errors, an incident that negatively impacted revenue.  A traditional monitoring system might have been able to catch a drop in revenue as it occurred, but without machine learning, this company would only have caught the connection between these two anomalies had an analyst happened to stumble upon them. An unlikely scenario. The customer managed to release a fix within 3 hours, saving them a significant amount of otherwise lost revenue.     Adapting to Market Changes with AI   Given the subtlety of this alert at the start, using static thresholds would have taken longer for an alert to fire. With the impact of COVID-19 on the travel industry, affected businesses who rely on static thresholds are having to manually adjust those settings to the new norm. They would again need to readjust those settings as travel bookings pick up, although at this time no one can accurately predict when that will be. An AI-based monitoring solution, on the other hand, can adapt to the new normal, without the need for any human intervention. In particular, Anodot’s unsupervised learning algorithms are able to monitor thousands of metrics simultaneously and understand the normal behavior for each individual one. This ability to adapt to changing market conditions and consumer behavior can drastically improve a company's ability to adjust growth and demand forecasts in real-time, both of which can significantly contribute to the bottom line.  As you can see below, the shaded blue area represents the normal range of data. As the COVID-19 closures occurred in mid-March, you can see the AI monitoring solution was able to adjust its normal range and catch up with the global changes in bookings within days. Towards the end of the graph, we can also see there’s an increase towards the original range, which happened without any human intervention or a need to adjust a static threshold:    Real-Time Detection in a Complex Environment   A final example of the difficulty of building your own monitoring system is the fact that you’re dealing with human-generated data, meaning it’s incredibly volatile, irregular, and seasonal. For example, the image below is from a gaming company and you can clearly see the seasonal nature of gamers playing more on the weekends and evenings. In this example, someone on the team released a hot fix, along with a critical bug, that prevented players from completing in-game purchases. Luckily, their anomaly detection solution was able to detect and alert the error in real-time, and root cause analysis led the developers directly to the recent release.  Since there is such a high degree of seasonality in this user-generated data, unlike a traditional BI tool, an AI solution is able to calculate the normal usage depending on each hour and day and adapt accordingly. These incidents give you an under-the-hood look at the complexity in monitoring business metrics, some of which include: an adaptive baseline seasonality granular visibility monitor at scale to correlate related anomalies/events real-time detection In the next post, we'll look at why these aspects also come into play for finding hidden incidents that might otherwise go undetected in your partner networks and affiliate programs.    

While single transaction revenue models are prone to fluctuate based on the seasonality of markets, subscription plans offer more predictable revenues. And while that consistency can certainly be advantageous over one-off transactions, it's notoriously challenging to keep subscribers active. The cornerstone to managing and scaling a subscription-based business is monitoring the KPIs that influence top-line revenue, such as: conversion rate churn rate retention rate One of Anodot's customers runs a subscription-based business with transactions from various countries, in different languages and on different devices. Before working with Anodot, the company experienced an error in the customer sign-up process, where the SMS verification was broken— but only for customers in Russia using Android devices. Without this SMS verification process in place, anybody that fell into this demographic was unable to subscribe to their subscription or process their payment for three weeks. Instead of being able to detect this anomaly in real-time, the bug went unnoticed by their traditional monitoring methods, resulting in three weeks of frustrated customers and hundreds of thousands in lost revenue. As soon as they realized how much time and money could have been saved with the use of machine learning, the company immediately went looking for an AI-based revenue monitoring solution. While you can theoretically use traditional monitoring methods such as statistical models or BI tools, they lack the granularity, scalability, and accuracy to be able to find and alert on an anomaly like the one described above.   What is Revenue Monitoring? If you’re running a subscription-based business, you know that you need to constantly monitor performance metrics such as website traffic, bounce rate, time on site, and many others. The same concept applies to revenue data. Revenue monitoring refers to the process of tracking the KPIs and metrics that influence overall revenue such as conversion rate, subscriber growth, subscribers per location, and so on.  Revenue data is often made up of billions of data points that are influenced by human behavior. In particular, as discussed in our white paper on business monitoring, these metrics pose a unique challenge for three main reasons: Context: Revenue-related business metrics often can’t be evaluated in absolute terms, with set maximum and minimum thresholds. They should be evaluated in relation to a set of changing conditions. Topology: It's much easier to track the relationship between different machines, although the same is not true for business metrics. The relationships and correlations between metrics are dynamic and volatile. Volatility: Business metrics often have irregular sampling rates, which requires a significant adaptation of how data is stored and how the algorithms work. Due to the dynamic nature of revenue-related metrics, manually monitoring that data manually or with static thresholds can easily deluge teams with false positives or, worse yet, allow for false negatives to go unnoticed. So how can companies overcome current hurdles to scalable monitoring and ensure the consistency and predictability of their revenue?  Many companies are enhancing their analytics stack with machine learning, automating their monitoring and anomaly detection. Applying AI to Revenue Monitoring AI-based revenue monitoring means that the solution is able to learn the usual behavior of each performance metric on its own, without providing static thresholds.  Machine learning’s ability to process huge amounts of data and derive insights, patterns, and correlations means that it can provide the granularity and scalability that subscription-based businesses need to effectively monitor their revenue. The AI-based solution can then notify the appropriate team in real-time when anomalous events do occur. Returning to our earlier example of monitoring location-based changes in subscription renewals, below is a visualization of how AI-based anomaly detection can be used for revenue monitoring. In particular, we can see the model is monitoring active subscribers of a particular plan for a given location:   In this example, we can see that the blue line is the actual users’ behavior, and the shaded blue area is the expected behavior based on what the machine learning algorithm has previously learned.  This is an example of unsupervised learning in which the machine learning algorithms derive patterns and structure from unlabeled data. As an example, Anodot uses sequential adaptive learning to learn the normal behavior of each metric, and then each new data point is computed in relation to this behavior going forward. If, however, we wanted to add more metrics to monitor such as each individual traffic source, referrals from affiliates, and so on, you can see how this would simply be too complex for a traditional BI tool to monitor. As highlighted in our guide on Revenue Monitoring with AI, a few reasons why an AI-based solution has so advantageous over traditional monitoring include: AI can learn and monitor each revenue stream by itself: Since the revenue streams from each subscription plan are unique, this means it’s crucial to monitor each one on its own instead of simply monitoring revenue as a single metric. AI can monitor metrics such as traffic and conversion rates simultaneously: If an anomalous event does occur, for example, a significant drop in revenue, being able to monitor not only top-line revenue but also the events that lead up a purchase (such as traffic, conversion rates, etc.) means that you can immediately identify the root cause. AI can correlate metrics and events in real-time for the shortest time to resolution: In the same way that AI can monitor multiple metrics simultaneously, it can also find correlations between metrics and events so that you know exactly what is causing the anomaly and so you have the shortest time to resolution possible. Real World Examples of Revenue Monitoring for Subscription-Based Businesses These are among the most common revenue-related metrics to monitor in a subscription business model.  Churn Rate Monitoring Churn rate helps you identify how many customers you’re losing over a given time period. Monitoring this metric is one example of leveraging customer experience monitoring for subscription-based businesses as a high churn rate indicates that users either aren’t getting enough value from the subscription or don’t know how to use the product properly.   New Subscriber Monitoring New subscriber monitoring is another use case of AI-based revenue monitoring. As you can see below, the anomaly detection solution is constantly monitoring for spikes or drops in registrations. The platform is also fully autonomous so you don’t need to set up and update typical subscriber thresholds. This means that the AI can autonomously monitor billions of events and distills them into a single score, and can also send impact alerts when you need them most. Conversion Rate Monitoring Conversion rate is another metric that has a high impact on a company’s revenue. If there is a sudden drop in conversion rate, this could mean that there is something broken on the website, or, like the example mentioned earlier, there could be a simple translation error. Catching these issues in real-time can often lead to saving a significant amount of potential lost revenue. Aside from conversion rate monitoring for purchase conversions, as discussed in our guide on Use Cases for Machine Learning, subscription-based businesses can also monitor existing customers from the time they login until the time they logout. This allows you to identify anomalous user behavior such as how they’re interacting with features in the product   Revenue Monitoring for Subscription-Based Businesses  As we’ve discussed, revenue streams for subscription-based businesses are highly complex and often fragmented across products, plans, and locations. These revenue streams are also highly susceptible to changes in things like conversion rate, churn rate, and many other performance metrics. The dynamic nature of this revenue data means that traditional monitoring methods, such as BI tools, simply don’t offer the capabilities that are required.  Instead, an AI-based anomaly detection solution can learn the usual behavior of each individual metric on its own, and provide real-time updates when it matters most. Regardless of the number of products offered or the number of active subscribers, being able to catch these incidents in real time can often mean saving a significant amount of otherwise lost revenue.

See how online businesses can use machine learning to more intelligently support teams as they monitor micro-transaction revenue.

I’m very pleased to announce that we’ve just secured $35 million in funding, bringing our total capital raised to $62.5 million.

All the Zoom meetings can get tiresome. Break free from the generic white wall as a background with this fun collection of virtual backgrounds, tried and tested by the Anodot team.

Customer experience is tied to so many different areas of an app - product, customer support, and payments. How do you find small breaks in the chain? Most tools can't. Machine learning solutions are changing that.

Start measuring, monitoring, analyzing, experimenting and improving at the speed of light. Read more here about growth hacks for data-driven marketing.