How AI Found the $12,000/Month Leak in Our Funnel
How AI Found the $12,000/Month Leak in Our Funnel
By Dr. Elena Vasquez, PhD in Artificial Intelligence
The Invisible Cost of Friction
Every SaaS company, e-commerce brand, or digital product team knows the feeling: traffic is up, ad spend is climbing, and yet revenue growth feels sluggish. You’ve optimized your landing pages. You’ve A/B tested your copy. You’ve polished your checkout flow. But revenue still lags. The truth is, most of the money isn’t leaking from one dramatic failure. It’s seeping out through dozens of small, invisible frictions in your funnel—frictions that human analysts miss because the data is too noisy, too high-dimensional, and too subtle to spot by eye.
This is exactly the problem we faced at Meridian Labs, a mid-size B2B analytics platform. Our monthly recurring revenue was plateauing at around $480,000, and our customer acquisition cost (CAC) had crept up 22% over six months. We were spending roughly $12,000 per month in wasted ad spend, abandoned carts, and under-converting trials that we simply couldn’t pinpoint. After three months of manual analysis, we decided to let an AI system dig in. What it found changed how we think about funnel optimization.
Setting the Stage: A Funnel That Looked Healthy on the Surface
Our funnel looked, on the surface, like any healthy SaaS funnel:
Visitors → Sign-ups → Activated Trials → Paid Conversions
128k 14,200 3,800 620Our conversion rates were within industry benchmarks. Our email drip campaigns were firing on schedule. Our support ticket volume was low. Everything looked “fine.” And yet, when we looked at the unit economics, the math didn’t add up. We were acquiring customers at a blended CAC of $310, but our average revenue per account (ARPA) was only $280 in year one. We were losing money on every new account, and the gap was widening.
We knew there was a leak. We just couldn’t find it.
Why Humans Miss Funnel Leaks
The human analyst’s approach to funnel analysis is fundamentally a reductionist one. You look at stage-by-stage conversion rates, you segment by channel or geography, and you look for the biggest drop-off. This works when the leak is large and obvious—say, a broken checkout button or a misconfigured payment gateway. But most funnel inefficiencies are not like that. They are:
Multi-dimensional: a combination of user segment, time of day, device type, and content interaction that only becomes visible when you cross-tabulate five or more variables.
Non-linear: the relationship between a variable and conversion isn’t a clean line. A 2-second delay in page load might cost 4% of conversions for mobile users in the 25–34 age bracket on first visit, but only 1% for desktop users in the 45+ bracket on repeat visit.
Subtle in magnitude: a 0.3% drop in trial activation that only affects a specific product tier in a specific geography.
The human brain can comfortably hold about four to five variables in working memory. A funnel with 40+ tracking events, 8 user segments, 3 device types, and 12 geographic markets produces a combinatorial space of thousands of possible interaction effects. You can’t eyeball that. You need a system that can explore the space systematically.
What We Built: An AI-Powered Funnel Diagnostics System
We didn’t buy a black-box SaaS tool. We built a lightweight pipeline on top of our existing data warehouse (Snowflake) and a set of open-source models. The architecture had three layers:
Layer 1 — Event Ingestion and Normalization. We pulled 14 months of product analytics events (Mixpanel), marketing attribution data (Google Analytics 4 + server-side GTM), billing records (Stripe), and support tickets (Intercom). We normalized timestamps, deduplicated events, and created a unified user-session graph. Each user’s journey became a sequence of events with timestamps, device metadata, segment tags, and a final outcome label (converted / not converted).
Layer 2 — Feature Engineering. This was the critical step. Instead of looking at raw conversion rates, we engineered interaction features:
Time-of-day × device × segment interaction terms
Sequence-based features: e.g., “viewed pricing page within 5 minutes of sign-up” vs. “viewed pricing page 2 days after sign-up”
Cohort-level features: “first-time visitor from organic search, 28–40 age, enterprise tier interest”
Latent behavioral embeddings: we ran a small transformer model (12 layers, 256 hidden units) over user session sequences to learn a 64-dimensional behavioral embedding per user. This captured patterns like “tends to abandon after comparing two pricing tiers” or “engages deeply with documentation before trial activation.”
Layer 3 — Causal Inference and Anomaly Detection. This is where it got interesting. We didn’t just ask the model “what correlates with conversion?” We asked it “what causes conversion differences?” We used a combination of:
Gradient-boosted trees (XGBoost) for feature importance and interaction detection. This gave us a ranked list of variable interactions that most strongly predicted conversion differences.
Causal forest (BART-based) to estimate the conditional average treatment effect (CATE) of specific funnel events. For example: “What is the causal effect of viewing the comparison page on trial activation, segmented by device type and time of day?”
Anomaly detection using an isolation forest over the cohort-level conversion metrics. This flagged cohorts that were underperforming relative to the expected baseline, even when the aggregate number looked normal.
The system ran as a weekly batch job. Every Monday at 06:00 UTC, it ingested the prior week’s data, retrained the models, and generated a diagnostic report. The report was not a dashboard. It was a structured set of hypotheses, each with a confidence score, an estimated revenue impact, and a recommended action.
The Findings: A $12,000/Month Leak in Three Places
After four weeks of running the system, the AI generated a report that identified three distinct leak points. None of them would have been visible to our human analysts using standard funnel dashboards.
Leak 1: The “Pricing Page Bounce” in the 6–9 PM Window
The causal forest identified a specific interaction: users who landed on our pricing page between 6:00 PM and 9:00 PM local time, on mobile devices, in the 25–35 age segment, and who had previously viewed the documentation section (but not the comparison page) had a 4.2 percentage point lower trial activation rate than the baseline. The estimated revenue impact was $4,800/month in lost ARR.
The human analysts had looked at the pricing page conversion rate overall: 38%. They had segmented by device: mobile 34%, desktop 42%. They had segmented by time of day: morning 40%, afternoon 39%, evening 37%. None of these cuts was bad enough to raise an eyebrow. The AI found that the interaction of evening + mobile + age + documentation-viewing behavior created a specific sub-cohort where the pricing page was actually a friction point. The pricing tiers were presented in a layout that was hard to compare on small screens during low-light evening sessions. Users were bouncing because the visual hierarchy was ambiguous on mobile at night.
Action taken: We redesigned the pricing page with a mobile-optimized, high-contrast tier comparison card. We added a “quick compare” toggle that collapsed the three tiers into a single scrollable column. We also added a context-aware tooltip that said “Most teams of your size start with the Growth tier” based on the user’s segment. Trial activation for this specific cohort improved from 34% to 41%.
Leak 2: The “Documentation Depth” Paradox
The behavioral embeddings revealed a counterintuitive pattern. Users who spent more than 8 minutes in our documentation section before starting a trial had a 6.1 percentage point lower conversion rate than users who spent 2–5 minutes. The AI estimated $5,200/month in lost revenue.
Our human team had assumed that more documentation reading = more educated users = better conversion. The AI found that it was the opposite, but with a nuance. Users who read deeply in the documentation were often evaluating us against a competitor. They were looking for specific technical details (API rate limits, SSO support, data residency) that our documentation presented in a way that made the comparison to the competitor’s equivalent feature ambiguous. The documentation was technically accurate but not comparison-oriented.
Action taken: We rebuilt the documentation’s “Features” section as a structured comparison table against our two closest competitors. We added a “Where we differ” callout box on each feature page. We also added a “Compare with [Competitor X]” link at the top of each doc page. The 8+ minute documentation cohort’s conversion improved from 31% to 36%.
Leak 3: The “Trial Email Sequence” Timing Mismatch
The anomaly detection flagged a specific cohort: users who signed up for a trial on a Monday and received our onboarding email at 8:00 AM (server time, not local time) had a 5.8 percentage point lower activation rate than the same cohort receiving the email at 10:00 AM. The impact was $2,000/month.
This was a small leak, but it was a systemic one. Our email system was sending the onboarding sequence at a fixed server-time schedule, not a user-local-time schedule. Users in the East Coast were getting the email at 6:00 AM their time. Users in the West Coast were getting it at 5:00 AM. The email was landing in their inbox at a time when they were less likely to engage.
Action taken: We moved the email scheduling to user-local-time based. The onboarding email now fires at 10:00 AM in the user’s local timezone. Activation for Monday sign-ups improved by 4.5 percentage points.
The Total Impact
Leak 1 (Pricing Page Bounce): $4,800/month recovered
Leak 2 (Documentation Depth): $5,200/month recovered
Leak 3 (Email Timing): $2,000/month recovered
─────────────────────────────────────────────────────
Total: $12,000/month recoveredThat’s $144,000/year in recovered revenue from three relatively low-effort fixes. The total engineering and design effort to implement all three fixes was roughly two engineer-weeks. The ROI was approximately 850%.
What This Teaches Us About AI and Funnel Optimization
The AI didn’t replace our human analysts. It augmented them. The human team still owned the strategic decisions: which leaks to fix first, how to redesign the pricing page, how to restructure the documentation. The AI’s job was to explore the high-dimensional space of interactions that humans can’t eyeball, and to surface the specific sub-cohorts where the friction was concentrated.
Three principles emerged from this project:
1. Funnel optimization is a causal problem, not a correlational one. Knowing that “mobile users convert 8% less than desktop users” is a correlation. Knowing that “mobile users who view the pricing page between 6–9 PM and have previously read documentation convert 12% less than the same users who view the pricing page between 10 AM–2 PM” is a causal insight. The AI’s value is in distinguishing correlation from causation at scale.
2. The leak is often in the interaction, not the main effect. Most funnel problems are not “mobile users convert less.” They are “mobile users in a specific age bracket, at a specific time of day, after a specific behavioral sequence, convert less.” The AI can find these interaction effects. Humans struggle to.
3. The system should generate hypotheses, not just dashboards. A dashboard tells you what is happening. A good AI system tells you why it’s happening and what to do about it. The output should be a ranked list of actionable hypotheses with estimated revenue impact, not a wall of charts.
The Next Iteration
We’re now feeding the AI’s output into a continuous experimentation loop. Every week, the system generates 3–5 new hypotheses. Our product team picks the top 1–2 by estimated impact, designs the fix, and runs a 2-week A/B test. The results feed back into the data warehouse, and the models retrain. It’s a closed loop of hypothesis, experiment, learning, and refinement.
The $12,000/month leak was the first find. The system is already generating the next set of hypotheses. We expect the cumulative recovered revenue to grow non-linearly as the models learn from each experiment.
A Note on Methodology and Transparency
We want to be clear about what the AI did and didn’t do. The transformer model for behavioral embeddings was trained on 14 months of session data. The causal forest was validated against a held-out cohort and cross-checked with 12 manual A/B tests that our team ran independently. The revenue impact estimates are based on historical conversion rates and ARPA, not forward-looking forecasts. We ran the system on a 4-week lookback window, so it captures recent behavioral shifts but not long-term seasonal patterns.
We also want to be clear about the limitations. The AI found three leaks. There may be more that it hasn’t surfaced yet, especially in segments too small to reach statistical significance. The behavioral embeddings capture patterns in the data, not the full causal mechanism. And the system is only as good as the event tracking it’s fed. If you’re not tracking the right events, the AI can only find leaks in the data you’ve chosen to record.
The Bigger Picture
This project was, in many ways, a microcosm of how AI is changing operations work in general. The pattern is the same in marketing, in supply chain, in customer support, in software engineering. Humans are good at asking the right questions, making the right strategic calls, and designing the right fixes. AI is good at exploring the high-dimensional space of interactions that make those questions answerable. The leak in the funnel was not a mystery. It was a pattern in the data that required a system to find.
The $12,000/month leak was a leak. The AI found it. The human team fixed it. And the system is already looking for the next one.
Dr. Elena Vasquez is a researcher and practitioner in applied machine learning, with a focus on causal inference and behavioral analytics. She has spent the last eight years building systems that turn operational data into strategic decisions. She holds a PhD in Computer Science with a specialization in statistical learning theory.
