We’re excited to announce that we've teamed up with Rivery to offer our data pipeline and monitoring solutions in one. Our one-click Data Kits streamline the analytics process, saving teams valuable time so they can act on business incidents fast. 

AI Analytics for eCommerce Digital trade and eCommerce companies are generating transactions in more significant quantities than ever before. In 2020, eCommerce sales made up 19% of all worldwide retail transactions, representing $26.7 trillion in revenue. The cornerstone of any eCommerce company is providing a seamless, reliable experience where customers can log into a clean interface, browse products, and make purchases quickly and on-demand. Increased digitization after the pandemic has only heightened the stakes. The scale, distribution, and speed of today's eCommerce generates millions of daily metrics, including orders, shipping, campaigns, application performance, APIs, log-ins and payment gateways, and countless others. It's simply impossible to notice every potential incident that may harm the customer experience or present an opportunity to increase revenue. That's because old-school analytics reporting, static monitoring, and traditional dashboards can only tell us what's already happened. The data streams which impact the customer experience in eCommerce are far too volatile for businesses to rely on static tools and manual intervention. In today's market, only real-time business monitoring based on AI and machine learning is robust enough to eliminate blind spots and deliver intelligent decision support to achieve competitive business outcomes. What types of issues impact eCommerce the most? For eCommerce companies that rely on online transactions as their bread and butter, incidents are costly. For a company with $1 Billion in annual revenues, annual incident costs can range anywhere from $8-32 million dollars. The total impact depends largely on what type of monitoring solutions they use and how quickly they notice an incident. Let's take a look at some of the most impactful incidents in the eCommerce space: High-volume traffic overloads In retail, seasonality is always critical. Many eCommerce operations rely heavily on limited time windows for the bulk of their revenue. For many businesses, Q4 accounts for more than 50% of annual sales. During these periods, demand can spike dramatically, and the cost of an outage can cripple revenue for the entire year. Issues with Online Shopping Carts One issue that eCommerce companies continue to struggle with is online shopping cart abandonment. There are several reasons why shoppers might forgo their purchase, but companies must know immediately if it's happening because of a technical issue. Unfortunately, traditional monitoring falls short in this area because there are too many metrics and dimensions (products, devices, sessions, campaigns, etc.) to detect issues before impacting the customer experience. Price Glitches Price glitches can price items far above or below their actual price resulting in a loss of revenue. While these issues are comparatively rare today after many high-profile and costly cases, they can still have a tremendous impact when they occur. Mistargeted online advertising Targeted advertising is crucial to eCommerce in today's hyper-competitive marketplace. Mistargeted advertising causes companies to waste opportunities and promotions to fail. Aside from targeting the wrong people with the wrong products, these sorts of issues can also cause companies to miss opportunities to upsell or cross-sell customers based on faulty analytics. Real-time Analytics and Autonomous Business Monitoring Countless metrics describe the customer experience, with each exhibiting multiple dimensions like seasonality. Incidents that impact that experience occur across a vast number of data streams as well. Manual analytics simply can't keep pace because they can't be performed in real-time or the data presented requires time-consuming analysis from a data scientist. Even a robust data science division will have difficulties monitoring and digesting millions of metrics to produce actionable information for business leaders. AI/ML-based analytics is a giant leap forward in capability and can create actionable, contextualized information to get ahead of incidents before they occur, or at a minimum, mitigate their worst impacts. Here are some of the most critical benefits of real-time analytics powered by AI/ML: [CTA id="3509d260-9c27-437a-a130-ca1595e7941f"][/CTA] Use Cases Conversion rate monitoring As conversion rate directly impacts revenue, monitoring for sudden drops can alert a company to errors in their checkout process and save a significant amount of otherwise lost revenue. Revenue monitoring Companies can leverage autonomous monitoring on all revenue-related metrics, including revenue from each acquisition channel, completed purchases and sales velocity. Customer fraud alerts Operators can apply AI/ML analytics to fraud detection and protect merchants against unexpected patterns in user behavior. Techniques Correlation Analysis eCommerce companies can use correlation analysis to reduce time to detection (TTD) and time to remediation (TTR) by guiding mitigation efforts early. Further, correlation analysis helps to reduce alert fatigue by filtering out irrelevant anomalies and grouping multiple anomalies stemming from a single incident into one alert. Anomaly Detection The data points most commonly tracked by eCommerce companies include purchases, page views or unique website visitors, failed payment transactions, or abandoned sales carts. Then, there are additional dimensions like the product category, geographical region, and the device, operating system, or app used for the transaction. All of these metrics are tied directly to eCommerce revenue. AI/ML-based anomaly detection can establish a baseline of expected behavior across all data points and detect anomalies in real-time. Real-time eCommerce Analytics with Anodot Every day that passes with an incident or opportunity undetected has a negative impact. Lost revenue, degraded customer experiences, and failed promotions can add up a horror stories for the business. These problems are often lurking in overlooked eCommerce analytics metrics, missed by overworked operators and data scientists relying on manual methods. The only performance monitoring solution that can meet today's eCommerce challenge can monitor a given metric's dimensions in real-time. Anodot's approach to business monitoring in eCommerce is autonomous. No manual dashboards. No operators sifting through false positives. Real-time analytics with Anodot can help you detect incidents 80% faster and reduce incident costs by over 70%. In this example, it took this company’s internal solution 3 days longer than Anodot to notice a drop in completed purchases, resulting in a loss of more than $200K. Anodot identified the root cause to be a version upgrade in Android devices, an incident that could not be detected with traditional dashboards and manual thresholds. Real-time analytics empowers e-commerce business leaders and teams at every layer of the organization by distilling millions of data points into actionable insights. That's critical because so much e-commerce revenue comes during the holidays when systems are under their heaviest loads of the year. For the companies that embrace it, real-time e-commerce analytics with AI/ML can provide the competitive advantage they need to avoid costly incidents, retain customers, and keep revenue rising.

Anodot’s customers to gain access to advanced cloud cost management and optimization capabilities that can be added to their existing Anodot monitoring services.

Freshly uses Anodot Autonomous Business Monitoring to monitor their prepared meal delivery service in real time for anomalies that can have a material impact on revenue and costs.

With the pandemic, supply chain blockages, and Amazon looming large, what do online retailers have planned for the holidays? Will shoppers buy as much online as they did in 2020? We surveyed thousands of U.S. eCommerce companies and consumers – read the report to learn what trends will dominate holiday shopping in 2021.

Success in today's high-velocity business environments means having the correct information to make the right decisions at the right time. As marketplaces grow more competitive and customer expectations continually rise, the "right time" is often real-time. Every transaction generates a plethora of data. Anomalies within your company's data set can represent opportunities and threats to the business. Real-time detection of anomalies empowers enterprises to make the right decisions to seize revenue opportunities and avoid potential losses. What are the main types of anomalies? There are three commonly accepted types of anomalies in statistics and data science: Global outliers, contextual outliers, and collective outliers. 1. Global outliers represent rare events that likely have never happened before. An example might be if a customer spends no more than $200 per week on e-commerce purchases, suddenly spending $10,000 in a single day. 2. Contextual outliers represent events that fall within the normal range from a global sense but are abnormal in the context of seasonal patterns. If a customer only ever spends $2500 every December on gifts but racks up $2000 in charges in July, it would be considered a contextual anomaly. While that month's spending isn't outside their normal global range, it occurs at an unusual time. 3. Collective outliers represent events that on their own do not fall outside of the standard expected behavior, but when combined, represent an anomaly. A group of customers with a history of order cancellations all canceling their orders at the same time would be a collective outlier. The limitations of manual anomaly detection In the past, when businesses only had a handful of metrics to track across their business, manual monitoring methods were feasible. Now, there are potentially millions of metrics to manage and multiple types of anomalies to consider and evaluate. On top of the inherent complexities, many real-life business anomalies require immediate action. A bad software update could cause a business to lose money every second. And since discovering the problem is the first step in resolving it, eliminating the delay between when the problem occurs and when the problem is detected immediately brings you one crucial step closer to rolling back that update and restoring revenue flow. Manual detection is also insufficient when the anomaly represents an opportunity rather than a problem. For example, an unusual uptick in mobile app installations from a specific geographical area may be due to a successful social media marketing campaign that has gone viral in that region. Given the short lifespan of such surges, your business has a limited time window to capitalize on this popularity and turn all those shares, likes, and tweets into sales. Even when anomalies don't require an immediate response, manual anomaly detection and dashboards can fall tragically short. You can always postpone action on an instant alert, but you can never react to a delayed alert in real-time. [CTA id="3509d260-9c27-437a-a130-ca1595e7941f"][/CTA] The Secrets of Fast and Scalable Autonomous Anomaly Detection If manual anomaly detection is inadequate, then automated anomaly detection must be used to achieve real-time anomaly detection at scale. Incremental machine learning algorithms for anomaly detection have the added benefit of scalability. In anomaly detection, there are two paradigms. In the first, a system detects an anomaly that has already occurred and displays it in a traditional dashboard using batch machine learning algorithms. That's only useful as a retrospective on what's happened in the past to inform future decisions. This has some value with long-term planning, but most online businesses need real-time decision-making to seize opportunities and prevent negative impacts. For example, sudden spikes or dips in purchases could present opportunities for action to generate more sales. The only way to take advantage of real-time trends is to know what's going on at the moment that it's happening. That requires automated, fast, and scalable anomaly detection in real-time. In this example, an ecommerce company was alerted as soon as there was an unusual drop in approval rates for PayPal payments. Fixing the issue quickly prevented a significant loss in revenue. Scaling for Growth Incremental machine learning algorithms are easily scalable, thus making them ideal for the incomprehensibly large data sets of today's businesses. If your company is continuously growing, then scalability is a valid concern. Incremental machine learning algorithms are the best option for companies with more metrics and large data sets. Advanced Anomaly Detection Systems that use advanced anomaly detection are more effective than those that use more straightforward techniques. A Gartner report on advanced anomaly detection explained that "They can detect subtle anomalies that might otherwise escape notice, provide earlier warning of impending problems or more time to capitalize on emerging opportunities, reduce the human effort required to develop a monitoring or measuring application, and reduce the time to solution for implementing complicated anomaly detection systems." Many systems try to find events and data outside the normal range to identify opportunities and threats. Yet, often these systems fail, identifying too many anomalies (false positives) or not enough (false negatives). Applying machine learning to rapidly changing environments means constantly updating and training models incrementally to prevent false positives. Our AI analytics solution offers autonomous analytics with contextualized alerts that minimize false positives. Instead of your data scientists asking many questions and carrying out complex data analysis, Anodot does the work and provides the Autonomous Business Analytics answers to understand why an incident happened. This real customer example illustrates how Anodot immediately detected a drop in completed purchases for an online store and correlated the anomalous events to a specific product and device in order to expedite remediation. AI/ML-based anomaly detection in the real world Scientists already harness the power of AI and machine learning to spot anomalies and the opportunities they present far faster than humans ever could on their own. An AI system developed by NASA's Jet Propulsion Laboratory was able to detect and command an orbital satellite to image a rare volcanic event in Ethiopia – before volcanologists even asked NASA for that satellite to take images of the eruption. When working with thousands or millions of metrics, real-time decision-making requires incremental machine learning algorithms. Whether it's saving your business money or gleaning scientific insights from a brief volcanic eruption, real-time anomaly detection has enormous potential for catching critical deviations in data sets that can have tremendous real-world impacts.

Not everything was gloom and doom. Our own analysis at Anodot confirms that some businesses benefitted from the Facebook outage, while others fretted.

Written by @InterpretableAI & IraIraCohen In the first four parts of our series on correlation analysis, we discussed the importance of this capability in root cause analysis in a number of business use cases, and then specifically in the context of promotional marketing, telco and algorithmic trading. In this blog we walk through how to leverage correlation analysis to address the challenges in ensuring a seamless online payment experience by the end-user. In general, digital payments include:  Online - browser-based purchases of goods/services  In-store - tapping mobile device at a point-of-sale/scanning a code to pay In-app - purchase of goods/services through an app such as features in a mobile game Peer-to-peer - send/receive money through a digital service/platform as exemplified by services such as Venmo, Zelle and the like End-use of digital payment include, but not limited to, BFSI (Banking, financial services and insurance), Healthcare, IT & Telecom, Media & Entertainment, Retail & E-commerce, Transportation.  Unlike matters pertaining to maintaining corporate liquidity, cash is increasingly becoming the not preferred option in the context of consumer-to-business transactions. The COVID-19 crisis and ongoing fears of infection have prompted consumers and businesses to rely, more than ever, on digital and contactless payment options. The infographic below (from McKinsey’s report) highlights the same. In the realm of contactless payment, biometric authentication is expected to gain momentum in the coming years. It is a verification method that involves the biological characteristics of the person. The verifications include facial recognition, fingerprinting scanning, heartbeat analysis, and vein mapping.  Since the founding of PayPal, numerous players have surfaced in the payment space, for example, but not limited to, Square, Stripe, Affirm, Venmo, Chargebee, Zelle, WhatsApp Pay, Novi. As per this whitepaper, 1.7 billion adults globally remain outside of the financial system with no access to a traditional bank, even though one billion have a mobile phone and nearly half a billion have internet access. To this end, payment systems such as Diem, based on blockchain, have been proposed. The Diem Blockchain is a decentralized, programmable database designed to support a low-volatility cryptocurrency that will have the ability to serve as an efficient medium of exchange for billions of people around the world. Novi is a new digital wallet for the Diem payment system. As per the report by Research And Markets, the Global Digital Payment Market size is expected to reach $175.8 billion by 2026, rising at a market growth of 20% CAGR during the forecast period. A sister report by Grand View Research USD breaks down the growth of the global digital payment market size to $236.10 billion by 2028 (a CAGR of 19.4% from 2021 to 2028) by solution type as shown below. Source: Grandview Research Factors driving the growth include, but not limited to, use of smartphones becoming ubiquitous, increasing demand for contactless payments and increasing customer expectations. In addition, factors such as the growing percentage of the global population using banking facilities and the growing adoption of open banking APIs are expected to create new growth avenues for the digital payment market. Headwinds include deteriorating perception of digital payments security over the past year and a growing concern with payments made via social apps and “Internet of Things” devices.” Smooth payment performance is the bedrock of end-user’s e-commerce experience and, consequently, key to contain churn. Performance is no longer limited to an outage. In a report published in October 2020, Gartner highlighted that the traditional KPIs centered around payment performance have been thrown into a tailspin by COVID-19. Going forward, the report calls out the following metrics: Latency: This pertains to the response time in executing a payment. Increase in latency - it can be traced back to a wide variety of reasons such as, but not limited to, gateway or processor issues - could potentially result in timeouts, lost sales and violation of contractual SLAs. Outages (e.g., click here) impacts both the end customer as well as the merchant. Authorization and decline rates: This pertains to how many submitted transactions are approved for payment by the issuer. Typically, approval rates for transactions carried out at the physical point of sale (POS) typically achieve much higher approval rates than the digital commerce counterpart. In light of this, it is important to monitor the metric for POS and digital separately. Also, a sudden shift to digital may warrant a recalibration on the issuer side as well. Anomalies in the digital case can potentially stem from, for example, the fraud detection system being a bit aggressive to contain downside. In an article published last year, Fry highlighted the following flavors of payment acceptance problems: Lack of funds from the payment method                                            “Card not present” transactions                                                  Wrong, missing or expired information                                            Fraud triggers                                                                   Payment requests don’t share the same format                                     Genuine cross-border “foreign” transactions get easily declined                  Currency conversion Fraud detection rates: This pertain to how many submitted transactions successfully make it through the fraud screening tools and are processed for payment. As more and more first-time customers use digital channels, it may result in higher fraud detection rates online. In light of this, it is important to measure the online and offline fraud rates separately - this would help triage the impact on sales and take action accordingly. Payment processing costs: As in-store sales move online, credit and debit card processing fees will apply to a greater portion of your overall sales. Plus, rates are materially less expensive at the physical POS than they are online, thus the credit card processing fees per transaction - which includes interchange fees that are paid to the issuing bank, gateway fees, processor fees, fraud detection costs and more - are likely to increase.  In a similar vein, Kar called out Responsiveness - How quick is the service provider to address issues when a service request is raised by a user? - as one of the key factors impacting user experience in the context of mobile payments - significance value of this hypothesis was found to be less than 0.05 under the model described in therein. Further, the following is recommended: “… enhance the speed at which complaints raised across channels surrounding failures of transactions are addressed ... hugely impact the customer relationship management and impact positively the responsiveness perceived by the customer when any challenge is faced from the use of a digital payment platform.”   Delivery high payment performance is non-trivial. It stems from the complexity of the payments ecosystem as exemplified by the infographics below. Source: GSMA A deep pipeline between the customer and the merchant (refer to the figure below) and potentially a large set of vendors makes the payment experience susceptible to cascading effects. This is akin to the tail latency problem discussed by Dean and Barroso (also see this, this, this). They highlight: “Even rare performance hiccups affect a significant fraction of all requests in large-scale distributed systems.” Source: Deloitte Likewise, one or more factors such as, but not limited to, multiple API handshakes, network hiccups (lost connectivity and/or congestion), availability of one or more intermediaries, directly impact the end-user’s payment experience. Going forward, with blockchain based decentralized architecture gaining momentum, providing resilience against faults would pose an interesting problem. For now, we shall leave the discussion on that front for another time.  Correlation between two or more payment processing KPIs (as discussed in BlueSnap’s article) can help triage the impact to a merchant’s bottomline: Payment Conversion Rate Conversion Rate By Bank Conversion Rate By Payment Method And Card Type   Checkout Abandonment Rate     Landing Page Optimization Rate   Mobile Vs. Desktop Clean Rate (defined as the total percentage of transactions that had a successful outcome) Fraud Rate   Order Rejections Rate  Chargeback Rate (defined as the number of transactions that are disputed by shoppers when they see an unfamiliar charge on their credit card bill) Uptime (what percentage of the day, week, or year is your provider online and processing successfully?) Consider the transactions at an ATM as an example. Potential issues can pertain to, for example, authentication, anomalies in aggregated withdrawal - amount and/or # transactions - per day, abnormally high decline rate owing to the fraud detection system gone wrong. Also, in order to reduce mean time to remediation (MTTR), one may have to slice-and-dice the analysis by, for instance, type of ATM and geography. The two examples below illustrate the importance of anomaly detection and correlations for root cause analysis. In the first example, Anodot’s platform alerts on abnormal drops in payment transaction success rate broken down by payment providers, geography and more. A dip in the success rate for one payment provider in a certain county was correlated to increased errors in one of the payment APIs for one of the banks in that country - this played a key role in rapid remediation of the issue. Interestingly, one would expect the aforementioned to be close to 1.0; however, in practice, we observe that the transaction success rate rises during the day and falls at night (that is, more transactions are successfully completed during the day compared to night time) and are often far from 1.0. The second example comes from monitoring a trading platform. Metrics such as, # transactions and deposits are monitored to surface any potential issues. In the example below, the number of deposits for trading accounts dropped abnormally and was correlated with an increase in latency to the main database handling those transactions - leading to a quick RCA indicating an issue with the DB. Recently, in order to boost the end-user experience, big bank adopted Anodot’s platform for payment and trading monitoring.  To wrap up, complexity of the payments ecosystem  lies at the root of the issues that adversely impact end-user experience. The metrics are diverse and hence, discovering correlations between the metrics can help root cause the issue at hand and drive a reduction in mean time to remediation (MTTR). The benefits to fintech companies include reduction in operational expenses and risk, protection of revenue, and improved customer experience. Continue this series to discover applications of and challenges with applying correlation analysis in: a wider business context ecommerce, specifically promotions fintech, specifically algorithmic trading network performance in telecommunications

For modern businesses faced with increasing volumes and complexity of data, it’s no longer efficient or feasible to rely on analyzing data in BI dashboards. Traditional dashboards are great at providing business leaders with insights into what's happened in the past, but what if they need actionable information in real time? What if they want to use their data to estimate what may happen in the future? Companies are taking notice. In a survey by Deloitte, 67% of executives said they were not satisfied with the benefits of their existing tools and resources. The highest-performing companies use data science empowered by AI/ML to bridge that gap. The Limitations of Dashboards For today's data-driven companies, there are thousands of metrics that can be gathered and collected to gain insights into business performance. As relevant data points grow exponentially, it becomes challenging to use traditional dashboards to track these metrics and make informed timely decisions. Here are some of the most significant limitations of conventional BI dashboards: Lack of real-time anomaly detection prevents proactive incident management Most BI dashboards do not show data in real-time, and when they do, there are so many metrics cluttering screens that users can easily miss the most critical information. Timely intervention is crucial to modern businesses, which often run tightly integrated ecosystems of applications and infrastructure that stretch across multiple departments and process enormous amounts of data. For example, leading adtech platform, Rubicon Project, fields trillions of bid requests per month and needs to analyze data points from millions of potential sources. In an environment like that, every minute can have significant impact. They found that traditional dashboards failed to deliver the real-time detection and response capability necessary to intervene before anomalies impacted their bottom line. According to Gartner, downtime costs the average business more than $300K per hour. Enterprises need the capability to manage these systems proactively rather than reactively. Over-reliance on historical data Most companies configure and use traditional dashboards to track KPIs and other critical business metrics to understand how their business and systems perform. One factor often missed by decision-makers is that the data they view in traditional dashboards describes what has already happened and might not be a reliable indicator of what will happen in the future. Moving from descriptive to predictive modes of thinking requires a deep understanding of the business context and critical thinking, which can be challenging for any person, or even a dedicated team, given the diversity of the data set, new trends, and fluctuating behaviors. Missing small incidents that have a negative impact Some incidents are hard to spot, but that doesn't mean they won't significantly impact the business. When undetected, hard-to-spot incidents can accumulate and can end up having the same impact as more prominent issues. A typical scenario involves incidents affecting only one business component. These isolated issues can easily get lost in KPIs based on a calculated average of multiple metrics. For example, a server cluster might be displaying a 99.99% average uptime. If one server in that cluster is experiencing an anomalously high amount of downtime, it could remain invisible to the dashboard. A single server is a small data point in a data center with thousands of servers, but it could be vital depending on what that server is running. CEO dashboards lack correlation When it comes to business intelligence and gaining real, actionable insights, choosing which metrics to include is more art than science. CEO dashboards only answer the questions that users who configure the system think to ask, but actionable insights can be present in any metric. This limitation is even more problematic because some insights can only be correlated when data is considered across multiple metrics, even if there is no apparent connection. For data to be actionable, it needs to have a holistic view of all relevant information and the impacts of the decision across the business. CEO dashboards fall short when missing data links delay or lead to misinformed decisions that harm the organization. Cluttered dashboards and false positives Sometimes, even with all the necessary information, BI dashboards struggle to present a coherent picture. With CEO dashboards, in particular, there's some guesswork in determining ahead of time what information is important enough to display in the limited real estate available on the screen. When alerts start to pop up, it can be difficult to tell which data is necessary or worth ignoring. The sheer volume and increasing complexity of data can quickly overwhelm the dashboard interface, making it much harder for business leaders to consume in a timely, accurate manner. Lack of intelligent prioritization Collecting thousands of events or alerts every minute from your applications and infrastructure, and presenting that data in a dashboard isn’t analytics. Users apply filters on this data, performing their own analysis and work. Deriving intelligence from data shouldn’t require an end user to define what to look for, or where, or what are the most critical KPIs, or what normal or abnormal is. This is not intelligence because a user is telling the dashboard exactly what data to show. Leverage the Power of AI Analytics Business strategy is only effective if empowered with enough intelligence and agility to outmaneuver the competition. Traditional dashboards don't provide insight fast enough in today's data-driven world, and when a business can lose hundreds of thousands of dollars in a single hour due to a pricing glitch on an e-Commerce site, the stakes are too high. Companies need real-time, actionable insights across all data metrics relevant to performance. The best-performing businesses leverage BI solutions empowered by AI and machine learning to eliminate the need for human correlation across the millions of critical metrics needed to understand business and system performance. Grouping and correlating multiple anomalies by design, Anodot's AI-powered analytics elevates the essential insights first. By learning the normal behavior of millions of metric’s, Anodot detects only the most impactful incidents and alerts relevant teams at the start. In the example above, an online business was alerted immediately to a spike in cost for a Google Ad campaign. It was an incident so granular that it would have been overlooked for days using traditional monitoring tools. As an AI analytics solution with anomaly detection capabilities, Anodot can monitor millions of metrics at a granular level, giving both the detail and scale needed to identify the business incidents that matter. Automated anomaly detection and contextualized alerts break free talented specialists and business leaders from the pain of manually monitoring dashboards so they can step in when it counts.